6 ms·

I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are signi

I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be had over what you get from a default GNU libc.

sylware 7mo ago
If there is so much performance difference among generic allocators, it means you need semantic optimized allocators (unless performance is actually not that much important in the end).
- Cloudef 7mo ago
  You are not wrong and this is indeed what zig is trying to push by making all std functions that allocate take a allocator parameter.
- codexon 7mo ago
  Agreed mostly. Going from standard library to something like jemalloc or tcmalloc will give you around 5-10% wins which can be significant, but the difference between those generic allocators seem small. I just made a slab allocator recently for a custom data type and got speedups of 100% over malloc.
  - sylware 7mo ago
    Here you go.
adgjlsfhk1 7mo ago
One of the best parts about GC languages is they tend to have much more efficient allocation/freeing because the cost is much more lumped together so it shows up better in a profile.
- pjmlp 7mo ago
  Agreed, however there is also a reason why the best ones also pack multiple GC algorithms, like in Java and .NET, because one approach doesn't fit all workloads.
  - nevdka 7mo ago
    Then there’s perl, which doesn’t free at all.
    - cermicelli 7mo ago
      Freedom is overrated... :P
    - NooneAtAll3 7mo ago
      doesn't java also? I heard that was a common complaint for minecraft
      - xxs 7mo ago
        What do you mean - if Java returns memory to the OS? Which one - Java heap of the malloc/free by the JVM?
        cogman10 7mo ago
        Java is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS. To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak.
        xxs 7mo ago
        Java has a quite strict max heap setting, it's very uncommon to let it allocate up to 25% of the system memory (the default). It won't grow past that point, though. Baring bugs/native leaks - Java has a very predictable memory allocation.
pjmlp 7mo ago
If you go into Dr Dobbs, The C/C++ User's Journal and BYTE digital archives, there will be ads of companies whose product was basically special cased memory allocator. Even toolchains like Turbo Pascal for MS-DOS, had an API to customise the memory allocator. The one size fits all was never a solution.
codexon 7mo ago
I've been using jemalloc for over 10 years and don't really see a need for it to be updated. It always holds up in benchmarks against any new flavor of the month malloc that comes out. Last time I checked mimalloc which was admittedly a while ago, probably 5 years, it was noticebly worse and I saw a lot of people on their github issues agreeing with me so I just never looked at it again.
- hrmtst93837 7mo ago
  [flagged]
  - codexon 7mo ago
    It still beat mimalloc when I checked 4-5 years ago.
    - imp0cat 7mo ago
      You really need to benchmark your workloads, ideally with the "big 3" (jemalloc, tcmalloc, mimalloc). They all have their strengths and weaknesses. Jemalloc can usually keep the smallest memory footprint, followed by tcmalloc. Mimalloc can really speed things up sometimes. As usually, YMMV.
      - codexon 7mo ago
        I've benchmarked them every few years, they never seem to differ by more than a few percent, and jemalloc seems to fragment and leak the least for processes running for months. Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing, so I am not inclined to trust it now.
        ComputerGuru 7mo ago
        > Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing That’s… ahistorical, at least so far as I remember. It wasn’t marketed as either of those; it was marketed as small/simple/consistent with an opt-in high-severity mode, and then its performance bore out as a result of the first set of target features/design goals. It was mainly pushed as easy to adopt, easy to use, easy to statically link, etc.
jeffbee 7mo ago
Just out of curiosity are you getting 1GB huge pages on Xeon or some other platform? I always thought this class of page is the hardest to exploit, considering that the machine only has, if I recall correctly, one TLB slot for those.
- bmenrigh 7mo ago
  Modern x86_64 has supported multiple page sizes for a long time. I'm on commodity Zen 5 hardware (9900X) with 128 GiB of RAM. Linux will still use a base page size of 4kb but also supports both 2 MiB and 1 GiB huge pages. You can pass something like `default_hugepagesz=2M hugepagesz=1G hugepages=16` to your kernel on boot to use 2 MiB pages but reserve 16 1 GiB pages for later use. The nice thing about mimalloc is that there are a ton of configurable knobs available via env vars. I'm able to hand those 16 1 GiB pages to the program at launch via `MIMALLOC_RESERVE_HUGE_OS_PAGES=16`. EDIT: after re-reading your comment a few times, I apologize if you already knew this (which it sounds like you did).
  - jeffbee 7mo ago
    Right but on Intel the 1G page size has historically been the odd one. For example Skylake-X has 1536 L2 shared TLB entries for either 4K or 2M pages, but it only has 16 entries that can be used for 1G pages. It wasn't unified until Cascade Lake. But Skylake-like Xeon is still incredibly common in the cloud so it's hard to target the later ones.
    - Dylan16807 7mo ago
      So for any process that's using less than 16GB, it's a significant performance boost. And most processes using more RAM, but not splitting accesses across more than 16 zones in rapid succession, will also see a performance boost. My old Intel CPU only has 4 slots for 1GB pages, and that was enough to get me about a 20% performance boost on Factorio. (I think a couple percent might have been allocator change but the boost from forcing huge pages was very significant)
      - 7mo ago
IshKebab 7mo ago
I feel like the real thing that needs to change is we need a more expressive allocation interface than just malloc/realloc. I'm sure that memory allocators could do a significantly better job if they had more information about what the program was intending to do.
- liuliu 7mo ago
  There are, look no further than jemalloc API surface itself: https://jemalloc.net/jemalloc.3.html https://jemalloc.net/jemalloc.3.html One thing to call out: sdallocx integrates well with C++'s sized delete semantics: https://isocpp.org/files/papers/n3778.html https://isocpp.org/files/papers/n3778.html
  - hedora 7mo ago
    You can also play tricks with inlining and constant propagation in C (especially on the malloc path, where the ground-truth allocation size is usually statically known).
anthk 7mo ago
I used mimalloc to run zenlisp under OpenBSD as it would clash with the paranoid malloc of base.
pocksuppet 7mo ago
In many cases you can also do better than using malloc e.g. if you know you need a huge page, map a huge page directly with mmap Yes, if you want to use huge pages with arbitrary alloc/free, then use a third-party malloc. If your alloc/free patterns are not arbitrary, you can do even better. We treat malloc as a magic black box but it's actually not very good.
skavi 7mo ago
We evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily written in Rust and the allocators were linked in statically (except for glibc). Unfortunately I didn't capture much context on the allocation patterns. I think in general the apps allocate and deallocate at a higher rate than most Rust apps (or more than I'd like at least). Our results from July 2025: rows are <allocator>: <RSS>, <time spent for allocator operations> app1: glibc: 215,580 KB, 133 ms mimalloc 2.1.7: 144,092 KB, 91 ms mimalloc 2.2.4: 173,240 KB, 280 ms tcmalloc: 138,496 KB, 96 ms jemalloc: 147,408 KB, 92 ms app2, bench1 glibc: 1,165,000 KB, 1.4 s mimalloc 2.1.7: 1,072,000 KB, 5.1 s mimalloc 2.2.4: tcmalloc: 1,023,000 KB, 530 ms app2, bench2 glibc: 1,190,224 KB, 1.5 s mimalloc 2.1.7: 1,128,328 KB, 5.3 s mimalloc 2.2.4: 1,657,600 KB, 3.7 s tcmalloc: 1,045,968 KB, 640 ms jemalloc: 1,210,000 KB, 1.1 s app3 glibc: 284,616 KB, 440 ms mimalloc 2.1.7: 246,216 KB, 250 ms mimalloc 2.2.4: 325,184 KB, 290 ms tcmalloc: 178,688 KB, 200 ms jemalloc: 264,688 KB, 230 ms tcmalloc was from github.com/google/tcmalloc/tree/24b3f29. i don't recall which jemalloc was tested.
- hedora 7mo ago
  I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name). tcmalloc (thread caching malloc) assumes memory allocations have good thread locality. This is often a double win (less false sharing of cache lines, and most allocations hit thread-local data structures in the allocator). Multithreaded async systems destroy that locality, so it constantly has to run through the exception case: A allocated a buffer, went async, the request wakes up on thread B, which frees the buffer, and has to synchronize with A to give it back. Are you using async rust, or sync rust?
  - skavi 7mo ago
    modern tcmalloc uses per CPU caches via rseq [0]. We use async rust with multithreaded tokio executors (sometimes multiple in the same application). so relatively high thread counts. [0]: https://github.com/google/tcmalloc/blob/master/docs/design.md https://github.com/google/tcmalloc/blob/master/docs/design.m...
m463 7mo ago
I remember in the early days of web services, using the apache portable runtime, specifically memory pools. If you got a web request, you could allocate a memory pool for it, then you would do all your memory allocations from that pool. And when your web request ended - either cleanly or with a hundred different kinds of errors, you could just free the entire pool. it was nice and made an impression on me. I think the lowly malloc probably has lots of interesting ways of growing and changing.
- jra_samba 7mo ago
  Look into talloc, used inside Samba (and other FLOSS projects like sssd). Exactly this.
- Sesse__ 7mo ago
  This is called “an arena” more generally, and it is in wide use across many forms of servers, compilers, and others.
Dylan16807 7mo ago
I think some operating system improvements could get people motivated to use huge pages a lot better. In particular make them less fragile on linux and make them not need admin rights on windows. The biggest factor causing problems there is that neither OS can swap a 2MB page. So someone needs to care enough to fix that.
vesselapi 7mo ago
[dead]

sylware 7mo ago

If there is so much performance difference among generic allocators, it means you need semantic optimized allocators (unless performance is actually not that much important in the end).

Cloudef 7mo ago
You are not wrong and this is indeed what zig is trying to push by making all std functions that allocate take a allocator parameter.
codexon 7mo ago
Agreed mostly. Going from standard library to something like jemalloc or tcmalloc will give you around 5-10% wins which can be significant, but the difference between those generic allocators seem small. I just made a slab allocator recently for a custom data type and got speedups of 100% over malloc.
- sylware 7mo ago
  Here you go.

adgjlsfhk1 7mo ago

One of the best parts about GC languages is they tend to have much more efficient allocation/freeing because the cost is much more lumped together so it shows up better in a profile.

pjmlp 7mo ago
Agreed, however there is also a reason why the best ones also pack multiple GC algorithms, like in Java and .NET, because one approach doesn't fit all workloads.
- nevdka 7mo ago
  Then there’s perl, which doesn’t free at all.
  - cermicelli 7mo ago
    Freedom is overrated... :P
  - NooneAtAll3 7mo ago
    doesn't java also? I heard that was a common complaint for minecraft
    - xxs 7mo ago
      What do you mean - if Java returns memory to the OS? Which one - Java heap of the malloc/free by the JVM?
      - cogman10 7mo ago
        Java is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS. To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak.
        xxs 7mo ago
        Java has a quite strict max heap setting, it's very uncommon to let it allocate up to 25% of the system memory (the default). It won't grow past that point, though. Baring bugs/native leaks - Java has a very predictable memory allocation.

pjmlp 7mo ago

If you go into Dr Dobbs, The C/C++ User's Journal and BYTE digital archives, there will be ads of companies whose product was basically special cased memory allocator. Even toolchains like Turbo Pascal for MS-DOS, had an API to customise the memory allocator. The one size fits all was never a solution.

codexon 7mo ago

I've been using jemalloc for over 10 years and don't really see a need for it to be updated. It always holds up in benchmarks against any new flavor of the month malloc that comes out. Last time I checked mimalloc which was admittedly a while ago, probably 5 years, it was noticebly worse and I saw a lot of people on their github issues agreeing with me so I just never looked at it again.

hrmtst93837 7mo ago
[flagged]
- codexon 7mo ago
  It still beat mimalloc when I checked 4-5 years ago.
  - imp0cat 7mo ago
    You really need to benchmark your workloads, ideally with the "big 3" (jemalloc, tcmalloc, mimalloc). They all have their strengths and weaknesses. Jemalloc can usually keep the smallest memory footprint, followed by tcmalloc. Mimalloc can really speed things up sometimes. As usually, YMMV.
    - codexon 7mo ago
      I've benchmarked them every few years, they never seem to differ by more than a few percent, and jemalloc seems to fragment and leak the least for processes running for months. Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing, so I am not inclined to trust it now.
      - ComputerGuru 7mo ago
        > Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing That’s… ahistorical, at least so far as I remember. It wasn’t marketed as either of those; it was marketed as small/simple/consistent with an opt-in high-severity mode, and then its performance bore out as a result of the first set of target features/design goals. It was mainly pushed as easy to adopt, easy to use, easy to statically link, etc.

jeffbee 7mo ago

Just out of curiosity are you getting 1GB huge pages on Xeon or some other platform? I always thought this class of page is the hardest to exploit, considering that the machine only has, if I recall correctly, one TLB slot for those.

bmenrigh 7mo ago
Modern x86_64 has supported multiple page sizes for a long time. I'm on commodity Zen 5 hardware (9900X) with 128 GiB of RAM. Linux will still use a base page size of 4kb but also supports both 2 MiB and 1 GiB huge pages. You can pass something like `default_hugepagesz=2M hugepagesz=1G hugepages=16` to your kernel on boot to use 2 MiB pages but reserve 16 1 GiB pages for later use. The nice thing about mimalloc is that there are a ton of configurable knobs available via env vars. I'm able to hand those 16 1 GiB pages to the program at launch via `MIMALLOC_RESERVE_HUGE_OS_PAGES=16`. EDIT: after re-reading your comment a few times, I apologize if you already knew this (which it sounds like you did).
- jeffbee 7mo ago
  Right but on Intel the 1G page size has historically been the odd one. For example Skylake-X has 1536 L2 shared TLB entries for either 4K or 2M pages, but it only has 16 entries that can be used for 1G pages. It wasn't unified until Cascade Lake. But Skylake-like Xeon is still incredibly common in the cloud so it's hard to target the later ones.
  - Dylan16807 7mo ago
    So for any process that's using less than 16GB, it's a significant performance boost. And most processes using more RAM, but not splitting accesses across more than 16 zones in rapid succession, will also see a performance boost. My old Intel CPU only has 4 slots for 1GB pages, and that was enough to get me about a 20% performance boost on Factorio. (I think a couple percent might have been allocator change but the boost from forcing huge pages was very significant)
    - 7mo ago

IshKebab 7mo ago

I feel like the real thing that needs to change is we need a more expressive allocation interface than just malloc/realloc. I'm sure that memory allocators could do a significantly better job if they had more information about what the program was intending to do.

liuliu 7mo ago
There are, look no further than jemalloc API surface itself: https://jemalloc.net/jemalloc.3.html https://jemalloc.net/jemalloc.3.html One thing to call out: sdallocx integrates well with C++'s sized delete semantics: https://isocpp.org/files/papers/n3778.html https://isocpp.org/files/papers/n3778.html
- hedora 7mo ago
  You can also play tricks with inlining and constant propagation in C (especially on the malloc path, where the ground-truth allocation size is usually statically known).

anthk 7mo ago

I used mimalloc to run zenlisp under OpenBSD as it would clash with the paranoid malloc of base.

pocksuppet 7mo ago

In many cases you can also do better than using malloc e.g. if you know you need a huge page, map a huge page directly with mmap Yes, if you want to use huge pages with arbitrary alloc/free, then use a third-party malloc. If your alloc/free patterns are not arbitrary, you can do even better. We treat malloc as a magic black box but it's actually not very good.

skavi 7mo ago

We evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily written in Rust and the allocators were linked in statically (except for glibc). Unfortunately I didn't capture much context on the allocation patterns. I think in general the apps allocate and deallocate at a higher rate than most Rust apps (or more than I'd like at least). Our results from July 2025: rows are <allocator>: <RSS>, <time spent for allocator operations> app1: glibc: 215,580 KB, 133 ms mimalloc 2.1.7: 144,092 KB, 91 ms mimalloc 2.2.4: 173,240 KB, 280 ms tcmalloc: 138,496 KB, 96 ms jemalloc: 147,408 KB, 92 ms app2, bench1 glibc: 1,165,000 KB, 1.4 s mimalloc 2.1.7: 1,072,000 KB, 5.1 s mimalloc 2.2.4: tcmalloc: 1,023,000 KB, 530 ms app2, bench2 glibc: 1,190,224 KB, 1.5 s mimalloc 2.1.7: 1,128,328 KB, 5.3 s mimalloc 2.2.4: 1,657,600 KB, 3.7 s tcmalloc: 1,045,968 KB, 640 ms jemalloc: 1,210,000 KB, 1.1 s app3 glibc: 284,616 KB, 440 ms mimalloc 2.1.7: 246,216 KB, 250 ms mimalloc 2.2.4: 325,184 KB, 290 ms tcmalloc: 178,688 KB, 200 ms jemalloc: 264,688 KB, 230 ms tcmalloc was from github.com/google/tcmalloc/tree/24b3f29. i don't recall which jemalloc was tested.

hedora 7mo ago
I’m surprised (unless they replaced the core tcmalloc algorithm but kept the name). tcmalloc (thread caching malloc) assumes memory allocations have good thread locality. This is often a double win (less false sharing of cache lines, and most allocations hit thread-local data structures in the allocator). Multithreaded async systems destroy that locality, so it constantly has to run through the exception case: A allocated a buffer, went async, the request wakes up on thread B, which frees the buffer, and has to synchronize with A to give it back. Are you using async rust, or sync rust?
- skavi 7mo ago
  modern tcmalloc uses per CPU caches via rseq [0]. We use async rust with multithreaded tokio executors (sometimes multiple in the same application). so relatively high thread counts. [0]: https://github.com/google/tcmalloc/blob/master/docs/design.md https://github.com/google/tcmalloc/blob/master/docs/design.m...

m463 7mo ago

I remember in the early days of web services, using the apache portable runtime, specifically memory pools. If you got a web request, you could allocate a memory pool for it, then you would do all your memory allocations from that pool. And when your web request ended - either cleanly or with a hundred different kinds of errors, you could just free the entire pool. it was nice and made an impression on me. I think the lowly malloc probably has lots of interesting ways of growing and changing.

jra_samba 7mo ago
Look into talloc, used inside Samba (and other FLOSS projects like sssd). Exactly this.
Sesse__ 7mo ago
This is called “an arena” more generally, and it is in wide use across many forms of servers, compilers, and others.

Dylan16807 7mo ago

I think some operating system improvements could get people motivated to use huge pages a lot better. In particular make them less fragile on linux and make them not need admin rights on windows. The biggest factor causing problems there is that neither OS can swap a 2MB page. So someone needs to care enough to fix that.

vesselapi 7mo ago

[dead]