5 ms·
Scalable memory allocation using jemalloc
- ericHosick 16y ago40 points and no comments? Just wanted to say it is great submissions like this that make me come to Hacker News... Thank you for taking the time to write this up and post it here...
- liuliu 16y agoIt is a long article and requires some time to digest. No comments is a good thing that implies people do take time to read. I am still in the progress of reading ...
- JonnieCache 16y agoIt also shows people here are unwilling to comment from a position of ignorance; a smaller percentage of readers will be authoritative on such an advanced topic.
- krakensden 16y agoI'm impressed that there is still room to eke performance improvements out by fiddling with the base data structure, the original author has been working on refining it for quite a while* * http://t-t-travails.blogspot.com/2008/07/treaps-versus-red-black-trees.html http://t-t-travails.blogspot.com/2008/07/treaps-versus-red-b..., http://t-t-travails.blogspot.com/2010/04/red-black-trees-revisited.html http://t-t-travails.blogspot.com/2010/04/red-black-trees-rev...
- amadvance 16y agoIt would be interesting to see a performance comparison with nedmalloc. It uses a trie instead a of redblack-tree for a faster information access. http://www.nedprod.com/programs/portable/nedmalloc/ http://www.nedprod.com/programs/portable/nedmalloc/ Some time ago I evaluated both of them, and they seemed the best available choices. But I never found a comparison study.
- agazso 16y agoIt is not a performance evaluation, but there is a comment from Niall (the author of nedmalloc) in Jason Evans' blog that might give you a hint. http://t-t-travails.blogspot.com/2010/04/red-black-trees-revisited.html http://t-t-travails.blogspot.com/2010/04/red-black-trees-rev...
- jasone 16y agoI tried to include nedmalloc 1.0.5 in the benchmark, but it quite simply isn't polished enough for actual use. To start with I had to remove an invalid assertion in malloc() and implement posix_memalign(), only to find that it crashes within moments of application startup.
- dochtman 16y agoWhy is this not the default malloc shipped with libc's around the world yet?
- JonnieCache 16y agoBecause if there is one library that needs to be conservatively packaged, its libc. A critical flaw in libc pushed out around the world could cause nightmares too awful to contemplate. Our entire modern world literally depends upon it.
- Erwin 16y agoYMMV but for my Python web app executing mostly OO Python 2.5.x code, I got a 10% performance increase by using jemalloc compared to the malloc in RHEL 5. It's as simply as LD_PRELOAD=/path/to/libjemalloc.so -- memory usage is also better (millions of objects allocated than eventually released in a long running process ended up with smaller amount of memory used when using jemalloc).
- ot 16y agoThis is kind of surprising given that CPython uses its own ad-hoc allocator (see http://docs.python.org/release/2.3.5/whatsnew/section-pymalloc.html http://docs.python.org/release/2.3.5/whatsnew/section-pymall...) which resorts to malloc() only for large chunks of memory and buffers... I wonder what's happening there.