4 ms·
Python uses its own allocator for small sizes (below 512 bytes if I remember correctly). I've had a similar problem to yours. See https://glandium.org/blog/?p=
by glandium 8y ago
Python uses its own allocator for small sizes (below 512 bytes if I remember correctly). I've had a similar problem to yours. See https://glandium.org/blog/?p=3698 https://glandium.org/blog/?p=3698 and the followup
https://glandium.org/blog/?p=3723 https://glandium.org/blog/?p=3723 . I have a glibc bug open about the issue https://sourceware.org/bugzilla/show_bug.cgi?id=23416 https://sourceware.org/bugzilla/show_bug.cgi?id=23416 .
- MrRadar 8y agoWow, I've experienced an issue that's very similar to that one if not the same. Using malloc/free directly worked just fine on Windows and older versions of glibc but lead to runaway RSS on at least some newer versions. I ended up writing a small custom allocator on top of mmap to work around it. It's incredibly frustrating when the memory leak is in the system allocator and not your code.
- X-Istence 8y agoWe at Crunch.io run Python using jemalloc because we have found some really bad behaviour in the default glibc malloc when using Python with many threads. Using jemalloc we were able to reduce our memory pressure over time dramatically.