6 ms·

TLDR glibc’s malloc doesn’t work well for server workloads. Nice deep dive into the available options to trim it, but the FUD around switching to something like

TLDR glibc’s malloc doesn’t work well for server workloads. Nice deep dive into the available options to trim it, but the FUD around switching to something like mimalloc or the most recent tcmalloc is unwarranted considering both have seen large scale deployments and have same defaults out of the gate that don’t need tuning (and any tuning you do for glibc would need similar validation requirements).

hinkley 3y ago
In early days Java’s allocator was quite a bit faster than C++ in part because memory allocation was not fully concurrent, making allocation part of the sequential element of Amdahl’s law. Where we are now is better, but has its own problems.
- masklinn 3y ago
  > Where we are now is better Not really. Lots of rust users come from high level languages so when people come in and complain about rust being unexpectedly slow allocation issues are in the top 5 at least. It’s incredible how shit the standard allocators are in almost all systems.
  - athanagor2 3y ago
    Isn’t this inevitable if the user allocates each time a struct/array is created, rather than allocating upfront?
    - masklinn 3y ago
      Meh. Users not used to tracking allocations keep to the same regime (lots of allocations), but because the platform allocators are utterly terrible the allocation overhead is orders of magnitude higher than in even a relatively basic runtime, thus the program is dog slow. This is not inevitable, platforms could provide allocators which are less awful. Obviously they can’t be as fast as specialised runtime facilities but when you see the gains many applications get by just swapping in jemalloc or some such…
      - PH95VuimJjqBqy 3y ago
        I would simplify it and say it's an issue with developers who don't understand manual memory management (allocating memory pools up front is just 1 such strategy for manual memory management).
        masklinn 3y ago
        That’s not a simplification, that’s a misattribution. It’s not the beginner’s fault that most system allocators suck so hard you must do your utmost to limit their use.
        hinkley 3y ago
        Otherwise known as Victim Blaming. C developers are that old guy who white knuckles everything and calls people weak for wanting help. Many of the features in modern programs didn’t exist in the 90’s not because they hadn’t been thought of but because people spent all their energy getting the first fifty features to work reliably. Every app had a couple cool features. Most of them are de rigeur now because we can.
        PH95VuimJjqBqy 3y ago
        This is akin to the idea that a compiler will eventually solve performance problems. We're still waiting on that to happen. There's a difference between a developer who understands manual memory management choosing to use a tech that does it for you and a developer choosing to use a tech that does it for you so they don't have to learn manual memory management. manual memory management is table stakes for any halfway competent developer.
      - wongarsu 3y ago
        Rust used to use jemalloc, which was arguably the better default. For some reason they switched it to the system allocator when they added support for specifying the global allocator
        steveklabnik 3y ago
  - saagarjha 3y ago
    They’re not. They are supposed to be general-purpose allocators with requirements that may not match your workload.
    - masklinn 3y ago
      > They’re not. Are too > They are supposed to be general-purpose allocators with requirements that may not match your workload. I don’t think they match any workload which exists anywhere anymore. Possibly their workload existed back in the 90s. Jemalloc is also a general-purpose allocator, it‘s the one freebsd uses.
jeffbee 3y ago
glibc's malloc doesn't work well for any workload. If your workload is anything other than trivial you should be using another one. > FUD around switching to something like mimalloc or the most recent tcmalloc Certainly, and the specific FUD about libc++ is unwarranted for tcmalloc, considering the developer of tcmalloc also uses libc++.
- spockz 3y ago
  So which Malloc should we be using? How about setting that as a default? If apparently the trade off is so cookie cutter clear.
  - jeffbee 3y ago
    If you have a non-trivial workload (i.e. one that costs money) you should shop around and use the one that is the most efficient in your application.
  - kelnos 3y ago
    I assume what the GP actually meant was that glibc's allocator isn't tuned for any specific workloads. It tries to be a jack-of-all-trades, and in doing so, just ends up being okay-ish. As for what allocator you should be using, it depends on your workload. You'd need to test out some different options to see what works best. And "best" might also be something you have to define. Maybe you're ok with higher baseline memory usage if allocations are faster. Or maybe you want the lower memory usage.
    - vlovich123 3y ago
      Actually, mimalloc and tcmalloc both show that across almost any workload they outperform glibc. It’s difficult to outperform everywhere of course, but in terms of “works best without knowing the workload a priori”, tcmalloc and mimalloc should handily trounce glibc. Know if you know your workload and start tuning glibc, maybe it can outperform? Not sure but I’d also be skeptical. Tcmalloc and mimalloc both apply really advanced techniques that also require newer kernel support to implement user side RCU iirc. Glibc’s allocator by comparison is much longer in the tooth and just can’t compete on that. Most workloads don’t care of course, but glibc’s penchant for hanging onto RAM when not needed is one very user visible consequence of this.

hinkley 3y ago

In early days Java’s allocator was quite a bit faster than C++ in part because memory allocation was not fully concurrent, making allocation part of the sequential element of Amdahl’s law. Where we are now is better, but has its own problems.

masklinn 3y ago
> Where we are now is better Not really. Lots of rust users come from high level languages so when people come in and complain about rust being unexpectedly slow allocation issues are in the top 5 at least. It’s incredible how shit the standard allocators are in almost all systems.
- athanagor2 3y ago
  Isn’t this inevitable if the user allocates each time a struct/array is created, rather than allocating upfront?
  - masklinn 3y ago
    Meh. Users not used to tracking allocations keep to the same regime (lots of allocations), but because the platform allocators are utterly terrible the allocation overhead is orders of magnitude higher than in even a relatively basic runtime, thus the program is dog slow. This is not inevitable, platforms could provide allocators which are less awful. Obviously they can’t be as fast as specialised runtime facilities but when you see the gains many applications get by just swapping in jemalloc or some such…
    - PH95VuimJjqBqy 3y ago
      I would simplify it and say it's an issue with developers who don't understand manual memory management (allocating memory pools up front is just 1 such strategy for manual memory management).
      - masklinn 3y ago
        That’s not a simplification, that’s a misattribution. It’s not the beginner’s fault that most system allocators suck so hard you must do your utmost to limit their use.
        hinkley 3y ago
        Otherwise known as Victim Blaming. C developers are that old guy who white knuckles everything and calls people weak for wanting help. Many of the features in modern programs didn’t exist in the 90’s not because they hadn’t been thought of but because people spent all their energy getting the first fifty features to work reliably. Every app had a couple cool features. Most of them are de rigeur now because we can.
        PH95VuimJjqBqy 3y ago
        This is akin to the idea that a compiler will eventually solve performance problems. We're still waiting on that to happen. There's a difference between a developer who understands manual memory management choosing to use a tech that does it for you and a developer choosing to use a tech that does it for you so they don't have to learn manual memory management. manual memory management is table stakes for any halfway competent developer.
    - wongarsu 3y ago
      Rust used to use jemalloc, which was arguably the better default. For some reason they switched it to the system allocator when they added support for specifying the global allocator
      - steveklabnik 3y ago
- saagarjha 3y ago
  They’re not. They are supposed to be general-purpose allocators with requirements that may not match your workload.
  - masklinn 3y ago
    > They’re not. Are too > They are supposed to be general-purpose allocators with requirements that may not match your workload. I don’t think they match any workload which exists anywhere anymore. Possibly their workload existed back in the 90s. Jemalloc is also a general-purpose allocator, it‘s the one freebsd uses.

jeffbee 3y ago

glibc's malloc doesn't work well for any workload. If your workload is anything other than trivial you should be using another one. > FUD around switching to something like mimalloc or the most recent tcmalloc Certainly, and the specific FUD about libc++ is unwarranted for tcmalloc, considering the developer of tcmalloc also uses libc++.

spockz 3y ago
So which Malloc should we be using? How about setting that as a default? If apparently the trade off is so cookie cutter clear.
- jeffbee 3y ago
  If you have a non-trivial workload (i.e. one that costs money) you should shop around and use the one that is the most efficient in your application.
- kelnos 3y ago
  I assume what the GP actually meant was that glibc's allocator isn't tuned for any specific workloads. It tries to be a jack-of-all-trades, and in doing so, just ends up being okay-ish. As for what allocator you should be using, it depends on your workload. You'd need to test out some different options to see what works best. And "best" might also be something you have to define. Maybe you're ok with higher baseline memory usage if allocations are faster. Or maybe you want the lower memory usage.
  - vlovich123 3y ago
    Actually, mimalloc and tcmalloc both show that across almost any workload they outperform glibc. It’s difficult to outperform everywhere of course, but in terms of “works best without knowing the workload a priori”, tcmalloc and mimalloc should handily trounce glibc. Know if you know your workload and start tuning glibc, maybe it can outperform? Not sure but I’d also be skeptical. Tcmalloc and mimalloc both apply really advanced techniques that also require newer kernel support to implement user side RCU iirc. Glibc’s allocator by comparison is much longer in the tooth and just can’t compete on that. Most workloads don’t care of course, but glibc’s penchant for hanging onto RAM when not needed is one very user visible consequence of this.