Quote:
Originally Posted by Larry Legend
How does it perform at a 10M and 100M object insert?
Given the issues you've described I would think the worst case is your homegrown db having some inherit performance issue at scale, so I would look for it to be able to handle increase in scale in the performant environments.
If it stops at 300k/1M records at 300k records/s down to 10 records/s I would be very interested to see under what conditions I can **** with the "10 records/s" process. By figuring out how to move it from "10 records/s" to 8, you may figure out the issue.
It is not the DB. Compiling the exact same code on its own main method runs fine. It is something goofy with the maven build - it has to be. There’s no other explanation.
I am on the smallest piece. The slowdown happens after N insertions. Doing the jvm pstack command (i forget what its called) les me to believe most of the time is being spent in GC. Which is why i figured it was the heap approaching its limits set by the JVM. If the JVM senses its running out of heap space it aggressively tries to GC. My program is supposed to use a lot of memory, this is expected.