2 ms·

We evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily writt

• hedora 7mo ago
  I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name). tcmalloc (thread caching malloc) assumes memory allocations have good thread locality. This is often a double win (less false sharing of cache lines, and most allocations hit thread-local data structures in the allocator). Multithreaded async systems destroy that locality, so it constantly has to run through the exception case: A allocated a buffer, went async, the request wakes up on thread B, which frees the buffer, and has to synchronize with A to give it back. Are you using async rust, or sync rust?
  • skavi 7mo ago
    modern tcmalloc uses per CPU caches via rseq [0]. We use async rust with multithreaded tokio executors (sometimes multiple in the same application). so relatively high thread counts. [0]: https://github.com/google/tcmalloc/blob/master/docs/design.md https://github.com/google/tcmalloc/blob/master/docs/design.m...
    • usrnm 7mo ago
      How do you control which CPU your task resumes on? If you don't then it's still the same problem described above, no?
      • skavi 7mo ago
        on the OS scheduler side, i'd imagine there's some stickiness that keeps tasks from jumping wildly between cores. like i'd expect migration to be modelled as a non zero cost. complete speculation though. tokio scheduler side, the executor is thread per core and work stealing of in progress tasks shouldn't be happening too much. for all thread pool threads or threads unaffiliated with the executor, see earlier speculation on OS scheduler behavior.
        • packetlost 7mo ago
          Correct. The Linux scheduler has been NUMA aware + sticky for awhile (which is more or less what this reduces to in common scenarios).
  • jhalstead 7mo ago
    > I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name). Indeed, it's not the old gperftools version. Blog: https://abseil.io/blog/20200212-tcmalloc https://abseil.io/blog/20200212-tcmalloc History / Diffs: https://google.github.io/tcmalloc/gperftools.html https://google.github.io/tcmalloc/gperftools.html
  • skavi 7mo ago
    also: 1. tcmalloc is actually the only allocator I tested which was not using thread local caches. even glibc malloc has tcache. 2. async executors typically shouldn’t have tasks jumping willy nilly between threads. i see the issue u describe more often with the use of thread pools (like rayon or tokio’s spawn_blocking). i’d argue that the use of thread pools isn’t necessarily an inherent feature of async executors. certainly tokio relies on its threadpool for fs operations, but io-uring (for example) makes that mostly unnecessary.
• ComputerGuru 7mo ago
  That’s a considerable regression for mimalloc between 2.1 and 2.2 – did you track it down or report it upstream? Edit: I see mimalloc v3 is out – I missed that! That probably moots this discussion altogether.
  • skavi 7mo ago
    nope.
• codexon 7mo ago
  This is similar to what I experienced when I tested mimalloc many years ago. If it was faster, it wasn't faster by much, and had pretty bad worst cases.