10 ms·

I understand the decision to archive the upstream repo; as of when I left Meta, we (i.e. the Jemalloc team) weren’t really in a great place to respond to all th

I understand the decision to archive the upstream repo; as of when I left Meta, we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file (my favorite was the time someone filed an issue because our test suite didn’t pass on Itanium lol). Still, it makes me sad to see. Jemalloc is still IMO the best-performing general-purpose malloc implementation that’s easily usable; TCMalloc is great, but is an absolute nightmare to use if you’re not using bazel (this has become slightly less true now that bazel 7.4.0 added cc_static_library so at least you can somewhat easily export a static library, but broadly speaking the point still stands). I’ve been meaning to ask Qi if he’d be open to cutting a final 6.0 release on the repo before re-archiving. At the same time it’d be nice to modernize the default settings for the final release. Disabling the (somewhat confusingly backwardly-named) “cache oblivious” setting by default so that the 16 KiB size-class isn’t bloated to 20 KiB would be a major improvement. This isn’t to disparage your (i.e. Jason’s) original choice here; IIRC when I last talked to Qi and David about this they made the point that at the time you chose this default, typical TLB associativity was much lower than it is now. On a similar note, increasing the default “page size” from 4 KiB to something larger (probably 16 KiB), which would correspondingly increase the large size-class cutoff (i.e. the point at which the allocator switches from placing multiple allocations onto a slab, to backing individual allocations with their own extent directly) from 16 KiB up to 64 KiB would be pretty impactful. One of the last things I looked at before leaving Meta was making this change internally for major services, as it was worth a several percent CPU improvement (at the cost of a minor increase in RAM usage due to increased fragmentation). There’s a few other things I’d tweak (e.g. switching the default setting of metadata_thp from “disabled” to “auto”, changing the extent-sizing for slabs from using the nearest exact multiple of the page size that fits the size-class to instead allowing ~1% guaranteed wasted space in exchange for reducing fragmentation), but the aforementioned settings are the biggest ones.

kstrauser 1y ago
Stuff like this is what keeps me coming back here. Thanks for posting this! What's hard about using TCMalloc if you're not using bazel? (Not asking to imply that it's not, but because I'm genuinely curious.)
- Svetlitski 1y ago
  It’s just a huge pain to build and link against. Before the bazel 7.4.0 change your options were basically: 1. Use it as a dynamically linked library. This is not great because you’re taking at a minimum the performance hit of going through the PLT for every call. The forfeited performance is even larger if you compare against statically linking with LTO (i.e. so that you can inline calls to malloc, get the benefit of FDO , etc.). Not to mention all the deployment headaches associated with shared libraries. 2. Painfully manually create a static library. I’ve done this, it’s awful; especially if you want to go the extra mile to capture as much performance as possible and at least get partial LTO (i.e. of TCMalloc independent of your application code, compiling all of TCMalloc’s compilation units together to create a single object file). When I was at Meta I imported TCMalloc to benchmark against (to highlight areas where we could do better in Jemalloc) by pain-stakingly hand-translating its bazel BUILD files to buck2 because there was legitimately no better option. As a consequence of being so hard to use outside of Google, TCMalloc has many more unexpected (sometimes problematic) behaviors than Jemalloc when used as a general purpose allocator in other environments (e.g. it basically assumes that you are using a certain set of Linux configuration options [1] and behaves rather poorly if you’re not) [1] https://google.github.io/tcmalloc/tuning.html#system-level-optimizations https://google.github.io/tcmalloc/tuning.html#system-level-o...
  - kstrauser 1y ago
    Wow. That does sound quite unpleasant. Thanks again. This is far outside my regular work, but it fascinates me.
  - prpl 1y ago
    I’ve successfully used LLMs to migrate Makefiles to bazel, more or less. I’ve not tried the reverse but suspect (2) isn’t so bad these days. YMMV, of course, but food for thought
    - rfoo 1y ago
      Dunno why you got downvoted, but I've also tried to let Claude translate a bunch of BUILD files to equivalent CMakeLists.txt. It worked. The resulting CMakeLists.txt looks super terrible, but so is 95% of CMakeLists.txt in this world, so why bother, it's doomed anyway.
matoro 1y ago
That was me that filed the Itanium test suite failure. :)
- boulos 1y ago
  The Itanic was kind of great :). I'm convinced it helped sink SGI.
  - froh 1y ago
    Sunk by the Great Itanic ?
  - sitkack 1y ago
    Why was the sinking of SGI great?
    - boulos 1y ago
      Oh, that wasn't the intent. I meant two separate things. The Itanic itself was kind of fascinating, but mostly panned (hence the nickname). SGI's decision to built out Itanium systems may have helped precipitate their own downfall. That was sad.
      - cogman10 1y ago
        Still makes me sad. I partially think a major reason for the demise was that it was simply constructed too soon. Compiler tech wasn't nearly good enough to handle the ISA. Nowadays because of the efforts that have gone in to making SIMD effective, I'd think modern compilers would have an easier time taking advantage of that unique and strange uarch.
        sitkack 1y ago
        VLIW has a fatal flaw in how it was used in these systems. You cannot run general purpose dynamically scheduled workloads unless you combine the JIT engine and the scheduler. PRIOR ART. Which is the same exact problem of trying to run multiple compute kernels on a GPU at the same time. VLIW with an OS and runtime that uses a higher level language, Wasm or the JVM, could forseably support dynamic workloads where the main cpu was VLIW. Now if they had been designed as GPU like devices for processing data, then Fortune 1000 would have never needed or used Hadoop.
EnPissant 1y ago
Do you have any opinions on mimalloc?
gazpacho 1y ago
I would love to see these changes - or even some sort of blog post or extended documentation explaining rational. As is the docs are somewhat barren. I feel that there’s a lot of knowledge that folks like you have right now from all of the work that was done internally at Meta that would be best shared now before it is lost.
einpoklum 1y ago
> TCMalloc is great, but is an absolute nightmare to use if you’re not using bazel custom-malloc-newbie question: Why is the choice of build system (generator) significant when evaluating the usability of a library?
- fc417fc802 1y ago
  Because you need to build it to use it, and you likely already have significant build related infrastructure, and you are going to need to integrate any new dependencies into that. I'm increasingly convinced that the various build systems are elaborate and wildly successful ploys intended only to sap developer time and energy.
- CamouflagedKiwi 1y ago
  Because you have to build it. If they don't use the same build system as you, you either want to invoke their system, or import it into yours. The former is unappealing if it's 'heavy' or doesn't play well as a subprocess; the latter can take a lot of time if the build process you're replicating is complex. I've done both before, and seen libraries at various levels of complexity; there is definitely a point where you just want to give up and not use the thing when it's very complex.
  - username223 1y ago
    This. When step one is "install our weird build system," I'll immediately look for something else that meets my needs. All build systems suck, so everyone thinks they can write a better one, and too many people try. Pretty soon you end up having to learn a majority of this (https://en.wikipedia.org/wiki/List_of_build_automation_software https://en.wikipedia.org/wiki/List_of_build_automation_softw...) to get your code to compile.
    - einpoklum 1y ago
      If TCMalloc uses bazel, then you build it with Bazel. It just needs to install itself where you tell it to, and then either it has given you a pkg-config file, or otherwise, your own build system needs some library-finding logic for it ("find module" in CMake terms). Or - are you saying the problem is that you need to install Bazel?
klabb3 1y ago
> we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file Why not? I mean this is complete drive-by comment, so please correct me, but there was a fully staffed team at Meta that maintained it, but was not in the best place to manage the issues?
- xcrjm 1y ago
  They said the team was not in a great place to do it, eg. they probably had competing priorities that overshadowed triaging issues.
- anonymoushn 1y ago
  Well, to be blunt, the company does not care about this, so it does not get done.
Thaxll 1y ago
It's kind of wild that great software is hindered by a complicated build and integration process.
michaelcampbell 1y ago
> filed an issue because our test suite didn’t pass on Itanium lol For the non low-level programmers in the bowels of memory allocators among us, why is this a "lol"?
- Svetlitski 1y ago
  The Itanium ISA was an infamous failure, never seeing widespread usage, hence people often referring to it as “The Itanic” (i.e. the much-touted ship that immediately sunk). The fact that anyone would be using it today at all is sort of hilariously niche, and is illustrative of how wide-ranging and obscure the issues filed to the GitHub repo could be. On a similar token I recall seeing an issue (or maybe it was a PR?) to fix our build on GNU Herd.

kstrauser 1y ago

Stuff like this is what keeps me coming back here. Thanks for posting this! What's hard about using TCMalloc if you're not using bazel? (Not asking to imply that it's not, but because I'm genuinely curious.)

Svetlitski 1y ago
It’s just a huge pain to build and link against. Before the bazel 7.4.0 change your options were basically: 1. Use it as a dynamically linked library. This is not great because you’re taking at a minimum the performance hit of going through the PLT for every call. The forfeited performance is even larger if you compare against statically linking with LTO (i.e. so that you can inline calls to malloc, get the benefit of FDO , etc.). Not to mention all the deployment headaches associated with shared libraries. 2. Painfully manually create a static library. I’ve done this, it’s awful; especially if you want to go the extra mile to capture as much performance as possible and at least get partial LTO (i.e. of TCMalloc independent of your application code, compiling all of TCMalloc’s compilation units together to create a single object file). When I was at Meta I imported TCMalloc to benchmark against (to highlight areas where we could do better in Jemalloc) by pain-stakingly hand-translating its bazel BUILD files to buck2 because there was legitimately no better option. As a consequence of being so hard to use outside of Google, TCMalloc has many more unexpected (sometimes problematic) behaviors than Jemalloc when used as a general purpose allocator in other environments (e.g. it basically assumes that you are using a certain set of Linux configuration options [1] and behaves rather poorly if you’re not) [1] https://google.github.io/tcmalloc/tuning.html#system-level-optimizations https://google.github.io/tcmalloc/tuning.html#system-level-o...
- kstrauser 1y ago
  Wow. That does sound quite unpleasant. Thanks again. This is far outside my regular work, but it fascinates me.
- prpl 1y ago
  I’ve successfully used LLMs to migrate Makefiles to bazel, more or less. I’ve not tried the reverse but suspect (2) isn’t so bad these days. YMMV, of course, but food for thought
  - rfoo 1y ago
    Dunno why you got downvoted, but I've also tried to let Claude translate a bunch of BUILD files to equivalent CMakeLists.txt. It worked. The resulting CMakeLists.txt looks super terrible, but so is 95% of CMakeLists.txt in this world, so why bother, it's doomed anyway.

matoro 1y ago

That was me that filed the Itanium test suite failure. :)

boulos 1y ago
The Itanic was kind of great :). I'm convinced it helped sink SGI.
- froh 1y ago
  Sunk by the Great Itanic ?
- sitkack 1y ago
  Why was the sinking of SGI great?
  - boulos 1y ago
    Oh, that wasn't the intent. I meant two separate things. The Itanic itself was kind of fascinating, but mostly panned (hence the nickname). SGI's decision to built out Itanium systems may have helped precipitate their own downfall. That was sad.
    - cogman10 1y ago
      Still makes me sad. I partially think a major reason for the demise was that it was simply constructed too soon. Compiler tech wasn't nearly good enough to handle the ISA. Nowadays because of the efforts that have gone in to making SIMD effective, I'd think modern compilers would have an easier time taking advantage of that unique and strange uarch.
      - sitkack 1y ago
        VLIW has a fatal flaw in how it was used in these systems. You cannot run general purpose dynamically scheduled workloads unless you combine the JIT engine and the scheduler. PRIOR ART. Which is the same exact problem of trying to run multiple compute kernels on a GPU at the same time. VLIW with an OS and runtime that uses a higher level language, Wasm or the JVM, could forseably support dynamic workloads where the main cpu was VLIW. Now if they had been designed as GPU like devices for processing data, then Fortune 1000 would have never needed or used Hadoop.

EnPissant 1y ago

Do you have any opinions on mimalloc?

gazpacho 1y ago

I would love to see these changes - or even some sort of blog post or extended documentation explaining rational. As is the docs are somewhat barren. I feel that there’s a lot of knowledge that folks like you have right now from all of the work that was done internally at Meta that would be best shared now before it is lost.

einpoklum 1y ago

> TCMalloc is great, but is an absolute nightmare to use if you’re not using bazel custom-malloc-newbie question: Why is the choice of build system (generator) significant when evaluating the usability of a library?

fc417fc802 1y ago
Because you need to build it to use it, and you likely already have significant build related infrastructure, and you are going to need to integrate any new dependencies into that. I'm increasingly convinced that the various build systems are elaborate and wildly successful ploys intended only to sap developer time and energy.
CamouflagedKiwi 1y ago
Because you have to build it. If they don't use the same build system as you, you either want to invoke their system, or import it into yours. The former is unappealing if it's 'heavy' or doesn't play well as a subprocess; the latter can take a lot of time if the build process you're replicating is complex. I've done both before, and seen libraries at various levels of complexity; there is definitely a point where you just want to give up and not use the thing when it's very complex.
- username223 1y ago
  This. When step one is "install our weird build system," I'll immediately look for something else that meets my needs. All build systems suck, so everyone thinks they can write a better one, and too many people try. Pretty soon you end up having to learn a majority of this (https://en.wikipedia.org/wiki/List_of_build_automation_software https://en.wikipedia.org/wiki/List_of_build_automation_softw...) to get your code to compile.
  - einpoklum 1y ago
    If TCMalloc uses bazel, then you build it with Bazel. It just needs to install itself where you tell it to, and then either it has given you a pkg-config file, or otherwise, your own build system needs some library-finding logic for it ("find module" in CMake terms). Or - are you saying the problem is that you need to install Bazel?

klabb3 1y ago

> we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file Why not? I mean this is complete drive-by comment, so please correct me, but there was a fully staffed team at Meta that maintained it, but was not in the best place to manage the issues?

xcrjm 1y ago
They said the team was not in a great place to do it, eg. they probably had competing priorities that overshadowed triaging issues.
anonymoushn 1y ago
Well, to be blunt, the company does not care about this, so it does not get done.

Thaxll 1y ago

It's kind of wild that great software is hindered by a complicated build and integration process.

michaelcampbell 1y ago

> filed an issue because our test suite didn’t pass on Itanium lol For the non low-level programmers in the bowels of memory allocators among us, why is this a "lol"?

Svetlitski 1y ago
The Itanium ISA was an infamous failure, never seeing widespread usage, hence people often referring to it as “The Itanic” (i.e. the much-touted ship that immediately sunk). The fact that anyone would be using it today at all is sort of hilariously niche, and is illustrative of how wide-ranging and obscure the issues filed to the GitHub repo could be. On a similar token I recall seeing an issue (or maybe it was a PR?) to fix our build on GNU Herd.