4 ms·
> To achieve good performance, it was therefore necessary to explicitly control how often and when the garbage collector would run to avoid needless interruptio
by twic 7y ago
> To achieve good performance, it was therefore necessary to explicitly control how often and when the garbage collector would run to avoid needless interruptions of the main program, especially during parallel phases.
Why? This is a batch program, interruptions don't matter, only the end-to-end time does.
> The goal of elPrep is to simultaneously keep both the runtime and the memory use low.
Why? Keeping runtime low lets you get more work done. Keeping memory use low means what? They are using a machine with 384 GB RAM, make use of it.
Worth noting also that they used GCC 7.2.1, Go 1.9.5, and Java 10. That's a pretty old GCC.
They don't seem to explicitly select a GC with Java, so they'll be using G1. G1 is still not entirely mature. It got much faster between 9 and 10, and somewhat faster from 10 to 11. For a batch process like this, though, the parallel collector is still probably a better choice. Using a newer JDK and a different collector should give better performance - but admittedly, probably won't reduce heap usage.
- WatchDog 7y agoYeah there are lots more knobs they could have played with in regards to Java garbage collection. I wonder if they capped the java max heap size to what the go implementation used, how much it would have affected the runtime.