4 ms·

In the Before Times, the vast majority of software was written in garbage collected language where a working knowledge of the relative merits of C memory alloca

In the Before Times, the vast majority of software was written in garbage collected language where a working knowledge of the relative merits of C memory allocators is not useful or particularly relevant. Why would the scads of people writing JavaScript, Java, python, go, rails, etc need to be aware of jemalloc?

iam-da-author 16d ago
TL;DR: Because the runtime of most GC:d languages uses malloc for its internal data structures. I work for the runtime team of JPG @ Oracle. We use malloc in Hotspot, quite a lot actually! Providing your JVM with a good malloc can improve the performance of the runtime, both in terms of CPU and memory, by quite a bit. I don't think you need the details, but it's good to be aware that some mallocs are better than others, and there are multiple of them. Being aware of jemalloc is a good way of being aware of the facts I just mentioned :-).
- stackghost 16d ago
  The vast majority of professional developers are not in a position where they can just swap out allocators willy-nilly. They take what they get, and write the code they're assigned to write on the platform the CTO or their product lead or whoever has decided upon.
  - iam-da-author 16d ago
    Okay, well, I guess all I can say is that if you strive to be one of the developers who do get the chance to care about this stuff, then you should know this stuff :-).
xxs 16d ago
>...the vast majority of software was written in garbage collected language and even then recently it costed (us) quite a few months to blame JVM and later the default glibc memory allocator for running out native (not java heap memory) - had to exclude all possible native libs (zlib, zstd via jna), direct buffers, sockets, thread stacks and so on. Changing the malloc to jemalloc solved the issue, even though initially it was done for its debugging capabilities. It's just a great memory allocator.
- fc417fc802 16d ago
  I'm never sure if I should be upset or happy when I've been debugging a problem for long enough that I finally decide to switch something out in order to improve visibility and that immediately solves the problem for entirely unexpected reasons. Particularly all the times when I couldn't readily discern why.
- javier2 15d ago
  We changed to jemalloc first on a jvm service which we could never get to run in its kube memory limit. with jemalloc its been dead stable for years, and we made it the default for all jvm services
nh2 16d ago
Memory allocation behaviour has visible impact also for users of managed languages, and the behaviour of software for end users. In our Python program, a bit of numpy processing of large pictures led to 100 GB not being returned to the OS by glibc's default allocator and the machine running out of memory shortly after. With jemalloc's reliable memory return settings, those problems disappear.
- majora2007 16d ago
  I'm in the same boat, I just switched to using it for Kavita, which only does some basic open Image -> Thumbnail to smaller size -> write to disk when importing new comics/books and on linux, memory could swell to 10GB and never get released. Switched to jemalloc and instantly memory stayed well below 1GB.
  - nh2 16d ago
    Generally yes, but the wording of "instantly" begs for the following pedantic remark: This is controlled by jemalloc settings `dirty_decay_ms`, `muzzy_decay_ms`, and their interaction with `background_thread`. `dirty_decay_ms` currently defaults to 10 seconds, so it's not that instant. That is important e.g. for single-threaded programs that start other programs, such as my Python example: If it starts a subprocess before the 10 seconds elapse after `free()`, Python (and jemalloc) do not run, and get no chance to return memory to the OS. In such cases, either enable `background_thread`, or set the `_decay_` values to `0` to ensure immediate return to the OS upon `free()`. (This costs some performance.) See e.g. https://github.com/jemalloc/jemalloc/issues/2688 https://github.com/jemalloc/jemalloc/issues/2688
- stackghost 16d ago
  Okay but do you think the hordes of JavaScript developers at FANG can just change the browser’s allocator? The number of programmers who are in positions to care about jemalloc vs other malloc is minuscule
yxhuvud 16d ago
Because unfortunately, the runtimes belonging to all or at least most of those languages perform a lot better with jemalloc than with the system default. I wish that wasn't the case, but it is.

iam-da-author 16d ago

TL;DR: Because the runtime of most GC:d languages uses malloc for its internal data structures. I work for the runtime team of JPG @ Oracle. We use malloc in Hotspot, quite a lot actually! Providing your JVM with a good malloc can improve the performance of the runtime, both in terms of CPU and memory, by quite a bit. I don't think you need the details, but it's good to be aware that some mallocs are better than others, and there are multiple of them. Being aware of jemalloc is a good way of being aware of the facts I just mentioned :-).

stackghost 16d ago
The vast majority of professional developers are not in a position where they can just swap out allocators willy-nilly. They take what they get, and write the code they're assigned to write on the platform the CTO or their product lead or whoever has decided upon.
- iam-da-author 16d ago
  Okay, well, I guess all I can say is that if you strive to be one of the developers who do get the chance to care about this stuff, then you should know this stuff :-).

xxs 16d ago

>...the vast majority of software was written in garbage collected language and even then recently it costed (us) quite a few months to blame JVM and later the default glibc memory allocator for running out native (not java heap memory) - had to exclude all possible native libs (zlib, zstd via jna), direct buffers, sockets, thread stacks and so on. Changing the malloc to jemalloc solved the issue, even though initially it was done for its debugging capabilities. It's just a great memory allocator.

fc417fc802 16d ago
I'm never sure if I should be upset or happy when I've been debugging a problem for long enough that I finally decide to switch something out in order to improve visibility and that immediately solves the problem for entirely unexpected reasons. Particularly all the times when I couldn't readily discern why.
javier2 15d ago
We changed to jemalloc first on a jvm service which we could never get to run in its kube memory limit. with jemalloc its been dead stable for years, and we made it the default for all jvm services

nh2 16d ago

Memory allocation behaviour has visible impact also for users of managed languages, and the behaviour of software for end users. In our Python program, a bit of numpy processing of large pictures led to 100 GB not being returned to the OS by glibc's default allocator and the machine running out of memory shortly after. With jemalloc's reliable memory return settings, those problems disappear.

majora2007 16d ago
I'm in the same boat, I just switched to using it for Kavita, which only does some basic open Image -> Thumbnail to smaller size -> write to disk when importing new comics/books and on linux, memory could swell to 10GB and never get released. Switched to jemalloc and instantly memory stayed well below 1GB.
- nh2 16d ago
  Generally yes, but the wording of "instantly" begs for the following pedantic remark: This is controlled by jemalloc settings `dirty_decay_ms`, `muzzy_decay_ms`, and their interaction with `background_thread`. `dirty_decay_ms` currently defaults to 10 seconds, so it's not that instant. That is important e.g. for single-threaded programs that start other programs, such as my Python example: If it starts a subprocess before the 10 seconds elapse after `free()`, Python (and jemalloc) do not run, and get no chance to return memory to the OS. In such cases, either enable `background_thread`, or set the `_decay_` values to `0` to ensure immediate return to the OS upon `free()`. (This costs some performance.) See e.g. https://github.com/jemalloc/jemalloc/issues/2688 https://github.com/jemalloc/jemalloc/issues/2688
stackghost 16d ago
Okay but do you think the hordes of JavaScript developers at FANG can just change the browser’s allocator? The number of programmers who are in positions to care about jemalloc vs other malloc is minuscule

yxhuvud 16d ago

Because unfortunately, the runtimes belonging to all or at least most of those languages perform a lot better with jemalloc than with the system default. I wish that wasn't the case, but it is.