8 ms·

> Rust developers might consider switching to jemallocator for improved performance I am curious if this is something that everyone can do to get free performa

by a1o 3y ago

> Rust developers might consider switching to jemallocator for improved performance I am curious if this is something that everyone can do to get free performance or if there are caveats. Can C codebases benefit from this too? Is this performance that is simply left on table currently?

nicoburns 3y ago
I think it's pretty much free performance that's being left on the table. There's slight cost to binary size. And it may not perform better in absolutely all circumstances (but it will in almost all). Rust used to use jemalloc by default but switched as people found this surprising as the default.
Pop_- 3y ago
Switching to non-default allocator does not always brings performance boost. It really depend on your workload, which requires profiling and benchmarking. But C/C++/Rust and other lower level languages should all at least be able to choose from these allocators. One caveat is binary size. Custom allocator does add more bytes to executable.
- vlovich123 3y ago
  I don’t know why people still look to jemalloc. Mimalloc outperforms the standard allocator on nearly every single benchmark. Glibc’s allocator & jemalloc both are long in the tooth & don’t actually perform as well as state of the art allocators. I wish Rust would switch to mimalloc or the latest tcmalloc (not the one in gperftools).
  - masklinn 3y ago
    > I wish Rust would switch to mimalloc or the latest tcmalloc (not the one in gperftools). That's nonsensical. Rust uses the system allocators for reliability, compatibility, binary bloat, maintenance burden, ..., not because they're good (they were not when Rust switched away from jemalloc, and they aren't now). If you want to use mimalloc in your rust programs, you can just set it as global allocator same as jemalloc, that takes all of three lines: https://github.com/purpleprotocol/mimalloc_rust#usage https://github.com/purpleprotocol/mimalloc_rust#usage If you want the rust compiler to link against mimilloc rather than jemalloc, feel free to test it out and open an issue, but maybe take a gander at the previous attempt: https://github.com/rust-lang/rust/pull/103944 https://github.com/rust-lang/rust/pull/103944 which died for the exact same reason the the one before that (https://github.com/rust-lang/rust/pull/92249 https://github.com/rust-lang/rust/pull/92249) did: unacceptable regression of max-rss.
    - vlovich123 3y ago
      I know it’s easy to change but the arguments for using glibc’s allocator are less clear to me: 1. Reliability - how is an alternate allocator less reliable? Seems like a FUD-based argument. Unless by reliability you mean performance in which case yes - jemalloc isn’t reliably faster than standard allocators, but mimalloc is. 2. Compatibility - again sounds like a FUD argument. How is compatibility reduced by swapping out the allocator? You don’t even have to do it on all systems if you want. Glibc is just unequivocally bad. 3. Binary bloat - This one is maybe an OK argument although I don’t know what size difference we’re talking about for mimalloc. Also, most people aren’t writing hello world applications so the default should probably be for a good allocator. I’d also note that having a dependency of the std runtime on glibc in the first place likely bloats your binary more than the specific allocator selected. 4. Maintenance burden - I don’t really buy this argument. In both cases you’re relying on a 3rd party to maintain the code.
      - masklinn 3y ago
        > I know it’s easy to change but the arguments for using glibc’s allocator are less clear to me: You can find them at the original motivation for removing jemalloc, 7 years ago: https://github.com/rust-lang/rust/issues/36963 https://github.com/rust-lang/rust/issues/36963 Also it's not "glibc's allocator", it's the system allocator. If you're unhappy with glibc's, get that replaced. > 1. Reliability - how is an alternate allocator less reliable? Jemalloc had to be disabled on various platforms and architectures, there is no reason to think mimalloc or tcmalloc are any different. The system allocator, while shit, is always there and functional, the project does not have to curate its availability across platforms. > 2. Compatibility - again sounds like a FUD argument. How is compatibility reduced by swapping out the allocator? It makes interactions with anything which does use the system allocator worse, and almost certainly fails to interact correctly with some of the more specialised system facilities (e.g. malloc.conf) or tooling (in rust, jemalloc as shipped did not work with valgrind). > Also, most people aren’t writing hello world applications Most people aren't writing applications bound on allocation throughput either > so the default should probably be for a good allocator. Probably not, no. > I’d also note that having a dependency of the std runtime on glibc in the first place likely bloats your binary more than the specific allocator selected. That makes no sense whatsoever. The libc is the system's and dynamically linked. And changing allocator does not magically unlink it. > 4. Maintenance burden - I don’t really buy this argument. It doesn't matter that you don't buy it. Having to ship, resync, debug, and curate (cf (1)) an allocator is a maintenance burden. With a system allocator, all the project does is ensure it calls the system allocators correctly, the rest is out of its purview.
- charcircuit 3y ago
  I've never not gotten increased performance by swapping outc the allocator.
nh2 3y ago
Be aware `jemalloc` will make you suffer the observability issues of `MADV_FREE`. `htop` will no longer show the truth about how much memory is in use. * https://github.com/jemalloc/jemalloc/issues/387#issuecomment-443480621 https://github.com/jemalloc/jemalloc/issues/387#issuecomment... * https://gitlab.haskell.org/ghc/ghc/-/issues/17411 https://gitlab.haskell.org/ghc/ghc/-/issues/17411 Apparently now `jemalloc` will call `MADV_DONTNEED` 10 seconds after `MADV_FREE`: https://github.com/JuliaLang/julia/issues/51086#issuecomment-1696078203 https://github.com/JuliaLang/julia/issues/51086#issuecomment... So while this "fixes" the issue, it'll introduce a confusing time delay between you freeing the memory and you observing that in `htop`. But according to https://jemalloc.net/jemalloc.3.html https://jemalloc.net/jemalloc.3.html you can set `opt.muzzy_decay_ms = 0` to remove the delay. Still, the musl author has some reservations against making `jemalloc` the default: https://www.openwall.com/lists/musl/2018/04/23/2 https://www.openwall.com/lists/musl/2018/04/23/2 > It's got serious bloat problems, problems with undermining ASLR, and is optimized pretty much only for being as fast as possible without caring how much memory you use. With the above-mentioned tunables, this should be mitigated to some extent, but the general "theme" (focusing on e.g. performance vs memory usage) will likely still mean "it's a tradeoff" or "it's no tradeoff, but only if you set tunables to what you need".
- a1o 3y ago
  Thank you! That was very thorough! I will be reading the links. :)
- singron 3y ago
  Note that glibc has a similar problem in multithreaded contexts. It strands unused memory in thread-local pools, which grows your memory usage over time like a memory leak. We got lower memory usage that didn't grow over time by switching to jemalloc. Example of this: https://github.com/prestodb/presto/issues/8993 https://github.com/prestodb/presto/issues/8993
- masklinn 3y ago
  The musl remark is funny, because jemalloc's use of pretty fine-grained arenas sometimes leads to better memory utilisation through reduced fragmentation. For instance Aerospike couldn't fit in available memory under (admittedly old) glibc, and jemalloc fixed the issue: http://highscalability.com/blog/2015/3/17/in-memory-computing-at-aerospike-scale-when-to-choose-and-ho.html http://highscalability.com/blog/2015/3/17/in-memory-computin... And this is not a one-off: https://hackernoon.com/reducing-rails-memory-use-on-amazon-linux-with-jemalloc https://hackernoon.com/reducing-rails-memory-use-on-amazon-l... https://engineering.linkedin.com/blog/2021/taming-memory-fragmentation-in-venice-with-jemalloc https://engineering.linkedin.com/blog/2021/taming-memory-fra... jemalloc also has extensive observability / debugging capabilities, which can provide a useful global view of the system, it's been used to debug memleaks in JNI-bridge code: https://www.evanjones.ca/java-native-leak-bug.html https://www.evanjones.ca/java-native-leak-bug.html https://technology.blog.gov.uk/2015/12/11/using-jemalloc-to-get-to-the-bottom-of-a-memory-leak/ https://technology.blog.gov.uk/2015/12/11/using-jemalloc-to-...
TillE 3y ago
jemalloc and mimalloc are very popular in C and C++ software, yes. There are few drawbacks, and it's really easy to benchmark different allocators against eachother in your particular use case.
kragen 3y ago
basically that's why jason wrote it in the first place, but other allocators have caught up since then to some extent. so jemalloc might make your c either slower or faster, you'll have to test to know. it's pretty reliable at being close to the best choice does tend to use more ram tho
secondcoming 3y ago
You can override the allocator for any app via LD_PRELOAD
exxos 3y ago
[dead]
saagarjha 3y ago
Performance is not a one-dimensional scale where programs go from “slow” to “fast”, because there are always other factors at play. jemalloc can be the right fit for some applications but for others another choice might be faster, but it also might be that the choice is slower but better matches their goals (less dirty memory, better observability, certain security guarantees, …)
kelnos 3y ago
Rust used to use jemalloc as the default, but went back to using the system malloc back in 2018-ish[0]. Since Rust now has the GlobalAlloc trait (and the #[global_allocator] attribute), apps can use jemalloc as their allocator if they want. Not sure if there's a way for users to override via LD_PRELOAD or something, though. It turns out jemalloc isn't always best for every workload and use case. While the system allocator is often far from perfect, it at least has been widely tested as a general-purpose allocator. [0] https://github.com/rust-lang/rust/issues/36963 https://github.com/rust-lang/rust/issues/36963

by a1o 3y ago

nicoburns 3y ago

I think it's pretty much free performance that's being left on the table. There's slight cost to binary size. And it may not perform better in absolutely all circumstances (but it will in almost all). Rust used to use jemalloc by default but switched as people found this surprising as the default.

Pop_- 3y ago

Switching to non-default allocator does not always brings performance boost. It really depend on your workload, which requires profiling and benchmarking. But C/C++/Rust and other lower level languages should all at least be able to choose from these allocators. One caveat is binary size. Custom allocator does add more bytes to executable.

vlovich123 3y ago
I don’t know why people still look to jemalloc. Mimalloc outperforms the standard allocator on nearly every single benchmark. Glibc’s allocator & jemalloc both are long in the tooth & don’t actually perform as well as state of the art allocators. I wish Rust would switch to mimalloc or the latest tcmalloc (not the one in gperftools).
- masklinn 3y ago
  > I wish Rust would switch to mimalloc or the latest tcmalloc (not the one in gperftools). That's nonsensical. Rust uses the system allocators for reliability, compatibility, binary bloat, maintenance burden, ..., not because they're good (they were not when Rust switched away from jemalloc, and they aren't now). If you want to use mimalloc in your rust programs, you can just set it as global allocator same as jemalloc, that takes all of three lines: https://github.com/purpleprotocol/mimalloc_rust#usage https://github.com/purpleprotocol/mimalloc_rust#usage If you want the rust compiler to link against mimilloc rather than jemalloc, feel free to test it out and open an issue, but maybe take a gander at the previous attempt: https://github.com/rust-lang/rust/pull/103944 https://github.com/rust-lang/rust/pull/103944 which died for the exact same reason the the one before that (https://github.com/rust-lang/rust/pull/92249 https://github.com/rust-lang/rust/pull/92249) did: unacceptable regression of max-rss.
  - vlovich123 3y ago
    I know it’s easy to change but the arguments for using glibc’s allocator are less clear to me: 1. Reliability - how is an alternate allocator less reliable? Seems like a FUD-based argument. Unless by reliability you mean performance in which case yes - jemalloc isn’t reliably faster than standard allocators, but mimalloc is. 2. Compatibility - again sounds like a FUD argument. How is compatibility reduced by swapping out the allocator? You don’t even have to do it on all systems if you want. Glibc is just unequivocally bad. 3. Binary bloat - This one is maybe an OK argument although I don’t know what size difference we’re talking about for mimalloc. Also, most people aren’t writing hello world applications so the default should probably be for a good allocator. I’d also note that having a dependency of the std runtime on glibc in the first place likely bloats your binary more than the specific allocator selected. 4. Maintenance burden - I don’t really buy this argument. In both cases you’re relying on a 3rd party to maintain the code.
    - masklinn 3y ago
      > I know it’s easy to change but the arguments for using glibc’s allocator are less clear to me: You can find them at the original motivation for removing jemalloc, 7 years ago: https://github.com/rust-lang/rust/issues/36963 https://github.com/rust-lang/rust/issues/36963 Also it's not "glibc's allocator", it's the system allocator. If you're unhappy with glibc's, get that replaced. > 1. Reliability - how is an alternate allocator less reliable? Jemalloc had to be disabled on various platforms and architectures, there is no reason to think mimalloc or tcmalloc are any different. The system allocator, while shit, is always there and functional, the project does not have to curate its availability across platforms. > 2. Compatibility - again sounds like a FUD argument. How is compatibility reduced by swapping out the allocator? It makes interactions with anything which does use the system allocator worse, and almost certainly fails to interact correctly with some of the more specialised system facilities (e.g. malloc.conf) or tooling (in rust, jemalloc as shipped did not work with valgrind). > Also, most people aren’t writing hello world applications Most people aren't writing applications bound on allocation throughput either > so the default should probably be for a good allocator. Probably not, no. > I’d also note that having a dependency of the std runtime on glibc in the first place likely bloats your binary more than the specific allocator selected. That makes no sense whatsoever. The libc is the system's and dynamically linked. And changing allocator does not magically unlink it. > 4. Maintenance burden - I don’t really buy this argument. It doesn't matter that you don't buy it. Having to ship, resync, debug, and curate (cf (1)) an allocator is a maintenance burden. With a system allocator, all the project does is ensure it calls the system allocators correctly, the rest is out of its purview.
charcircuit 3y ago
I've never not gotten increased performance by swapping outc the allocator.

nh2 3y ago

Be aware `jemalloc` will make you suffer the observability issues of `MADV_FREE`. `htop` will no longer show the truth about how much memory is in use. * https://github.com/jemalloc/jemalloc/issues/387#issuecomment-443480621 https://github.com/jemalloc/jemalloc/issues/387#issuecomment... * https://gitlab.haskell.org/ghc/ghc/-/issues/17411 https://gitlab.haskell.org/ghc/ghc/-/issues/17411 Apparently now `jemalloc` will call `MADV_DONTNEED` 10 seconds after `MADV_FREE`: https://github.com/JuliaLang/julia/issues/51086#issuecomment-1696078203 https://github.com/JuliaLang/julia/issues/51086#issuecomment... So while this "fixes" the issue, it'll introduce a confusing time delay between you freeing the memory and you observing that in `htop`. But according to https://jemalloc.net/jemalloc.3.html https://jemalloc.net/jemalloc.3.html you can set `opt.muzzy_decay_ms = 0` to remove the delay. Still, the musl author has some reservations against making `jemalloc` the default: https://www.openwall.com/lists/musl/2018/04/23/2 https://www.openwall.com/lists/musl/2018/04/23/2 > It's got serious bloat problems, problems with undermining ASLR, and is optimized pretty much only for being as fast as possible without caring how much memory you use. With the above-mentioned tunables, this should be mitigated to some extent, but the general "theme" (focusing on e.g. performance vs memory usage) will likely still mean "it's a tradeoff" or "it's no tradeoff, but only if you set tunables to what you need".

a1o 3y ago
Thank you! That was very thorough! I will be reading the links. :)
singron 3y ago
Note that glibc has a similar problem in multithreaded contexts. It strands unused memory in thread-local pools, which grows your memory usage over time like a memory leak. We got lower memory usage that didn't grow over time by switching to jemalloc. Example of this: https://github.com/prestodb/presto/issues/8993 https://github.com/prestodb/presto/issues/8993
masklinn 3y ago
The musl remark is funny, because jemalloc's use of pretty fine-grained arenas sometimes leads to better memory utilisation through reduced fragmentation. For instance Aerospike couldn't fit in available memory under (admittedly old) glibc, and jemalloc fixed the issue: http://highscalability.com/blog/2015/3/17/in-memory-computing-at-aerospike-scale-when-to-choose-and-ho.html http://highscalability.com/blog/2015/3/17/in-memory-computin... And this is not a one-off: https://hackernoon.com/reducing-rails-memory-use-on-amazon-linux-with-jemalloc https://hackernoon.com/reducing-rails-memory-use-on-amazon-l... https://engineering.linkedin.com/blog/2021/taming-memory-fragmentation-in-venice-with-jemalloc https://engineering.linkedin.com/blog/2021/taming-memory-fra... jemalloc also has extensive observability / debugging capabilities, which can provide a useful global view of the system, it's been used to debug memleaks in JNI-bridge code: https://www.evanjones.ca/java-native-leak-bug.html https://www.evanjones.ca/java-native-leak-bug.html https://technology.blog.gov.uk/2015/12/11/using-jemalloc-to-get-to-the-bottom-of-a-memory-leak/ https://technology.blog.gov.uk/2015/12/11/using-jemalloc-to-...

TillE 3y ago

jemalloc and mimalloc are very popular in C and C++ software, yes. There are few drawbacks, and it's really easy to benchmark different allocators against eachother in your particular use case.

kragen 3y ago

basically that's why jason wrote it in the first place, but other allocators have caught up since then to some extent. so jemalloc might make your c either slower or faster, you'll have to test to know. it's pretty reliable at being close to the best choice does tend to use more ram tho

secondcoming 3y ago

You can override the allocator for any app via LD_PRELOAD

exxos 3y ago

[dead]

saagarjha 3y ago

Performance is not a one-dimensional scale where programs go from “slow” to “fast”, because there are always other factors at play. jemalloc can be the right fit for some applications but for others another choice might be faster, but it also might be that the choice is slower but better matches their goals (less dirty memory, better observability, certain security guarantees, …)

kelnos 3y ago

Rust used to use jemalloc as the default, but went back to using the system malloc back in 2018-ish[0]. Since Rust now has the GlobalAlloc trait (and the #[global_allocator] attribute), apps can use jemalloc as their allocator if they want. Not sure if there's a way for users to override via LD_PRELOAD or something, though. It turns out jemalloc isn't always best for every workload and use case. While the system allocator is often far from perfect, it at least has been widely tested as a general-purpose allocator. [0] https://github.com/rust-lang/rust/issues/36963 https://github.com/rust-lang/rust/issues/36963