3 ms·
There's... a lot of hand waving in this article and no numbers. I'd be pretty surprised if `malloc_trim` had a significant effect on cpython memory usage as mo
by ris 6y ago
There's... a lot of hand waving in this article and no numbers.
I'd be pretty surprised if `malloc_trim` had a significant effect on cpython memory usage as most python memory gets allocated in 256KiB "arenas", which, what with fragmentation, are unlikely to _ever_ be reclaimed.
On the other hand, the article dismisses threads with some vagueries around the GIL, suggesting people need to reach straight for processes if they're serious. Really, unless your code has almost no I/O or C-accelerated, GIL-less sections, if you're not using both threads and processes, you're just burning memory unnecessarily.
(edit: oh and then there's async but I'm a bit old-fashioned for that)
- yxhuvud 6y agoOne thing that has been known for a while is that there are allocation patterns where LibC malloc doesn't give back a sufficient amount of pages even when the pages are clean. See for example https://www.joyfulbikeshedding.com/blog/2019-03-14-what-causes-ruby-memory-bloat.html#a-magic-trick-trimming https://www.joyfulbikeshedding.com/blog/2019-03-14-what-caus... . In the end I think the consensus was that jemalloc was just better than invocing malloc_trim, but invoking malloc_trim now and then can certainly be a lot better than using neither malloc_trim or jemalloc.