3 ms·

This article would've been a bit cooler if the conclusion wasn't "switch from default allocator to jemalloc" but instead "use jemalloc to prove something is wro

This article would've been a bit cooler if the conclusion wasn't "switch from default allocator to jemalloc" but instead "use jemalloc to prove something is wrong in the default allocator and track down + find a fix for what's wrong in the default allocator" Unless I misunderstood that the default Rust allocator, with high request bodies and concurrency, is always going to suffer unfixable heap fragmentation like displayed in the article?

Hooray_Darakian 4y ago
I'm not sure it's the case that there's something "wrong" with the default allocator, but rather that there are different tradeoffs at play. There's an old but good discussion of the issue here https://github.com/rust-lang/rfcs/blob/master/text/1183-swap-out-jemalloc.md https://github.com/rust-lang/rfcs/blob/master/text/1183-swap...
- MuffinFlavored 4y ago
  A sharp increase that never comes back down over time seems more "wrong" than right to me, no?
  - Hooray_Darakian 4y ago
    Without more context I think it's unclear. I know linux tends to avoid freeing memory until/unless the system is near capacity, so this test may be running on a system with low memory pressure.
  - chc 4y ago
    It's not good, but only looking at the bad half of a tradeoff will always look bad. Without diving into the glibc source code to understand what it's doing, it is hard to say if this is actually wrong.
    - moomin 4y ago
      I’ve recently been doing a lot of work that involves algorithmic design with non-technical people and this is the fun thing. It’s very easy to identify expected scenarios, harder to identify how to achieve them and sometimes actually impossible on e you consider things like “the algorithm doesn’t know what happens next”.
ht85 4y ago
All allocators make different trade-offs in terms of memory usage, min / max / mean time to allocate and optimizations for certain allocation patterns (small, large, mixed, ...). There might not be anything wrong with the default allocator, it just isn't the best suited for that particular use.
twic 4y ago
The default allocator is whatever is provided by the platform libc. In this case, i would guess that's the glibc allocator. It's possible that the glibc allocator contains some simple bug. It's more likely that it doesn't contain any simple bugs, but makes different tradeoffs to jemalloc, which make it less suitable to this particular slice of applications.
mcronce 4y ago
I agree that there could have been a more satisfying conclusion, but it is worth noting that jemalloc isn't a panacea. I've seen issues similar to Svix in both Rust and C++ applications that were heavy on ephemeral allocations, and have fixed it by doing all of the following, depending on the specific process: * Switching from libc malloc to jemalloc * Switching from libc malloc to tcmalloc (dating myself a little bit) * Switching from libc malloc to mimalloc * Switching from jemalloc to mimalloc * Switching from jemalloc to libc malloc * Switching from mimalloc to jemalloc Possibly others; I only want to list cases I'm 100% certain of. Heap fragmentation is just a reality of some allocation patterns without a GC runtime. One certainly can (and, in some cases, should) make their application more allocator-friendly, but - aside from some often-low-hanging fruit - this is a time-intensive process involving a bit of, for lack of a better word, arcane knowledge (I should inline all my fields and allocate on the stack as much as possible, right? Yes, well, except ...) If you already have a halfway decent benchmark suite or workload generator, which you'll want for other purposes anyway, it's often a lot quicker to just try a few other allocators and select the one that handles your workload best.
- scottlamb 4y ago
  > * Switching from libc malloc to tcmalloc (dating myself a little bit) If you think of tcmalloc as an old crusty allocator, you've probably only seen the gperftools version of it. This is the version Google now uses internally: https://github.com/google/tcmalloc https://github.com/google/tcmalloc It's worth a fresh look. In particular, it supports per-CPU caches as an alternative to per-thread caches. Those are fantastic if you have a lot more threads than CPUs. I haven't checked if it's been adapted for the latest upstream kernel API, but there's also the idea of "vcpu"-based caches: basically rather than a physical cpu id, it's an (optionally per-numa-node-based) dense id assigned to active threads, so that it still works well if you have a small cpu allocation for this process on a many-core machine.
  - jeffbee 4y ago
    I think we can safely say that jemalloc has much better marketing than tcmalloc. If you search on github there is only 1 real project with a bazel WORKSPACE that mentions tcmalloc ... the rest are either forks of tcmalloc itself or trivial toy programs that I personally put on github to share with someone. There are zillions of users of jemalloc and everyone has heard of it.
glandium 4y ago
Reading the article reminded me of https://sourceware.org/bugzilla/show_bug.cgi?id=23416 https://sourceware.org/bugzilla/show_bug.cgi?id=23416 and I wouldn't be surprised if it's the same underlying issue.

Hooray_Darakian 4y ago

I'm not sure it's the case that there's something "wrong" with the default allocator, but rather that there are different tradeoffs at play. There's an old but good discussion of the issue here https://github.com/rust-lang/rfcs/blob/master/text/1183-swap-out-jemalloc.md https://github.com/rust-lang/rfcs/blob/master/text/1183-swap...

MuffinFlavored 4y ago
A sharp increase that never comes back down over time seems more "wrong" than right to me, no?
- Hooray_Darakian 4y ago
  Without more context I think it's unclear. I know linux tends to avoid freeing memory until/unless the system is near capacity, so this test may be running on a system with low memory pressure.
- chc 4y ago
  It's not good, but only looking at the bad half of a tradeoff will always look bad. Without diving into the glibc source code to understand what it's doing, it is hard to say if this is actually wrong.
  - moomin 4y ago
    I’ve recently been doing a lot of work that involves algorithmic design with non-technical people and this is the fun thing. It’s very easy to identify expected scenarios, harder to identify how to achieve them and sometimes actually impossible on e you consider things like “the algorithm doesn’t know what happens next”.

ht85 4y ago

All allocators make different trade-offs in terms of memory usage, min / max / mean time to allocate and optimizations for certain allocation patterns (small, large, mixed, ...). There might not be anything wrong with the default allocator, it just isn't the best suited for that particular use.

twic 4y ago

The default allocator is whatever is provided by the platform libc. In this case, i would guess that's the glibc allocator. It's possible that the glibc allocator contains some simple bug. It's more likely that it doesn't contain any simple bugs, but makes different tradeoffs to jemalloc, which make it less suitable to this particular slice of applications.

mcronce 4y ago

I agree that there could have been a more satisfying conclusion, but it is worth noting that jemalloc isn't a panacea. I've seen issues similar to Svix in both Rust and C++ applications that were heavy on ephemeral allocations, and have fixed it by doing all of the following, depending on the specific process: * Switching from libc malloc to jemalloc * Switching from libc malloc to tcmalloc (dating myself a little bit) * Switching from libc malloc to mimalloc * Switching from jemalloc to mimalloc * Switching from jemalloc to libc malloc * Switching from mimalloc to jemalloc Possibly others; I only want to list cases I'm 100% certain of. Heap fragmentation is just a reality of some allocation patterns without a GC runtime. One certainly can (and, in some cases, should) make their application more allocator-friendly, but - aside from some often-low-hanging fruit - this is a time-intensive process involving a bit of, for lack of a better word, arcane knowledge (I should inline all my fields and allocate on the stack as much as possible, right? Yes, well, except ...) If you already have a halfway decent benchmark suite or workload generator, which you'll want for other purposes anyway, it's often a lot quicker to just try a few other allocators and select the one that handles your workload best.

scottlamb 4y ago
> * Switching from libc malloc to tcmalloc (dating myself a little bit) If you think of tcmalloc as an old crusty allocator, you've probably only seen the gperftools version of it. This is the version Google now uses internally: https://github.com/google/tcmalloc https://github.com/google/tcmalloc It's worth a fresh look. In particular, it supports per-CPU caches as an alternative to per-thread caches. Those are fantastic if you have a lot more threads than CPUs. I haven't checked if it's been adapted for the latest upstream kernel API, but there's also the idea of "vcpu"-based caches: basically rather than a physical cpu id, it's an (optionally per-numa-node-based) dense id assigned to active threads, so that it still works well if you have a small cpu allocation for this process on a many-core machine.
- jeffbee 4y ago
  I think we can safely say that jemalloc has much better marketing than tcmalloc. If you search on github there is only 1 real project with a bazel WORKSPACE that mentions tcmalloc ... the rest are either forks of tcmalloc itself or trivial toy programs that I personally put on github to share with someone. There are zillions of users of jemalloc and everyone has heard of it.

glandium 4y ago

Reading the article reminded me of https://sourceware.org/bugzilla/show_bug.cgi?id=23416 https://sourceware.org/bugzilla/show_bug.cgi?id=23416 and I wouldn't be surprised if it's the same underlying issue.