4 ms·
Julia's non-incremental stop-the-world garbage collector is quite bad. I ran into problems with this when trying to implement an algorithm that creates and des
by fdej 12y ago
Julia's non-incremental stop-the-world garbage collector is quite bad.
I ran into problems with this when trying to implement an algorithm that creates and destroys lots of temporary objects, where each object is a wrapped C struct that can point to a large amount of extra memory (the algorithm in question is doing polynomial division, where each coefficient is a polynomial represented by a C object).
The Julia version of this program runs unexpectedly slowly and uses gigabytes of memory, even though a few megabytes should be enough. The same code runs much faster if you do it in C and free every temporary object when it's no longer used, or even you do it with a Python C extension. Since Python uses reference counting, the objects get freed as soon as they fall out of scope.
But Julia allocates thousands of objects at a time before running the GC. Inserting manual GC calls is even worse -- the memory usage drops, but the program slows down by an order of magnitude (every GC takes a long time, even if you've just allocated a couple of objects since the last time).
Part of the problem is that the C code in question uses an object pool to speed up frequent allocations and deallocations. But even if you disable the pool, Julia performs worse than you would hope. A good generational/incremental GC would solve the problem.
- ViralBShah 12y agoIncremental GC is in the works. Do see: https://github.com/JuliaLang/julia/pull/8699 https://github.com/JuliaLang/julia/pull/8699 https://github.com/JuliaLang/julia/pull/5227 https://github.com/JuliaLang/julia/pull/5227
- fdej 12y agoThat's good to know!