3 ms·
yup and the characterization of each allocator is so fuzzy, with zero methodology provided. allocators are so simple to just swap into your program. if you can
by skavi 17d ago
yup and the characterization of each allocator is so fuzzy, with zero methodology provided.
allocators are so simple to just swap into your program. if you can put together a few representative workloads, you should just try out a few allocators and profile whatever metrics you care about.
- imp0cat 17d agoAnd finally end up with either jemalloc or possibly mimalloc. ;)
- egberts1 17d agoInvariably so but I'm mulling over malloc()s on embedded topic now.
- skavi 17d agowe actually ended up with (new) tcmalloc. for us, tc was among the fastest in runtime while being very space efficient [0]. large rust application using far too many threads. we’ve since also had great success with tc’s built in profiling tools. [0]: https://news.ycombinator.com/item?id=47403847 https://news.ycombinator.com/item?id=47403847
- imp0cat 16d agoInteresting, is there an accompanying article? We've tried tcmalloc, too. I don't remember the exact details, but we basically ended using jemalloc because it was using way less memory. Same story with mimalloc - it usually provided a tiny bit more speed, but required more cpu and memory.
- skavi 16d agono article, sorry, grabbed the numbers from an old PR. to confirm, you were using tcmalloc from https://github.com/google/tcmalloc https://github.com/google/tcmalloc and not from https://github.com/gperftools/gperftools https://github.com/gperftools/gperftools, right? the latter is a lot worse iiuc.
- imp0cat 16d agoGood point. It was from the https://packages.debian.org/trixie/google-perftools https://packages.debian.org/trixie/google-perftools Debian package, which points to the gperftools project - so it was the worse one I guess.
- skavi 16d agoyeah that’s a common mistake when evaluating tcmalloc. gperftools tcmalloc diverged quite a while ago. doesn’t have a lot of the fancier features of modern tcmalloc [0]. [0]: https://github.com/google/tcmalloc/blob/master/docs/gperftools.md#differences https://github.com/google/tcmalloc/blob/master/docs/gperftoo...
- zX41ZdbW 17d agoClickHouse has been tested with jemalloc, mimalloc, tcmalloc (both variants), rpmalloc, lfalloc, hualloc, and ended up using jemalloc after a few patches and bug fixes.