14 ms·
Jemalloc Postmortem
- gdiamos 1y agoCongrats on the great run and the future. Jemalloc was an inspirational to many memory allocators.
- kstrauser 1y agoI was using FreeBSD back when jemalloc came along, and it blew my mind to imagine swapping out just that one (major) part of its libc. Honestly, it hadn't occured to me, and made me wonder what else we could wholesale replace.
- Twirrim 1y agoOh that's interesting. jemalloc is the memory allocator used by redis, among other projects. Wonder what the performance impact will be if they have to change allocators.
- dpe82 1y agoWhy would they have to change? Sometimes software development is largely "done" and there isn't much more you need to do to a library.
- jeffbee 1y agoFor an example of why an allocator is a maintenance treadmill, consider that C++ recently (relatively) added sized delete, and Linux recently gained transparent huge pages.
- Twirrim 1y agoIt's been 14 years since THP got added to the kernel[1], surely we're past calling that "recent" :) https://www.kernelconfig.io/config_transparent_hugepage https://www.kernelconfig.io/config_transparent_hugepage
- jeffbee 1y agoBut if they'd declared the allocators "done" 15 years ago, then you wouldn't have it.
- foldr 1y ago> In particular, the seeds for principled huge page allocation (HPA) were sown way back in 2016! HPA work continued apace for several years, slowed, then stagnated as tweaks piled on top of each other without the requisite refactoring that keeps a codebase healthy. This feature trajectory recently cratered.
- senderista 1y agoAnother example is rseq (which was originally implemented for tcmalloc).
- deleted 1y ago[deleted]
- dymk 1y agoTechnology marches on, and in some number of years other allocators will exist that outperform/outfeature jemalloc.
- jcelerier 1y agoThis number of years depending on your allocation profile could be something like -10 years easily. New allocators constantly crop up
- edflsafoiewq 1y agoPresumably then the performance impact of any switch will be positive.
- Analemma_ 1y agoMemory allocators are something I expect to rapidly degrade in the absence of continuous updates as the world changes underneath you. Changing page sizes, new ucode latencies, new security features etc. all introduce either outright breakage or at least changing the optimum allocation strategy and making your old profiling obsolete. Not to mention the article already pointed out one instance where a software stack (KDE, in that case) used allocation profiles that broke an earlier version completely. Even though that's fixed now, any language runtime update or new feature could introduce a new allocation style that grinds you down. As much as it's nice to think software can be done, I think something so closely tied to the kernel and hardware and the application layer, which all change constantly, never can be.
- binary132 1y ago“Software is just done sometimes” is a common refrain I see repeated among communities where irreplaceable software projects are often abandoned. The community consensus has a tendency to become “it is reliable and good enough, it must be done”.
- poorman 1y agoJemalloc is used as an easy performance boost probably by every major Ruby on Rails server.
- Twirrim 1y agoWhile I certainly wish that more software would reach a "done" stage, I don't think jemalloc is necessarily there yet. Unfortunately I'm aware of there being bugs in the current version of jemalloc, when run in certain environment configurations, including memory leaks. I know the folks that found it were looking to report it, but I guess that won't happen now. Even from a quick look at the open issues, I can see https://github.com/jemalloc/jemalloc/issues/2838 https://github.com/jemalloc/jemalloc/issues/2838, and https://github.com/jemalloc/jemalloc/issues/2815 https://github.com/jemalloc/jemalloc/issues/2815 as two examples, but there's a fair number of issues still open against the repository. So that'll leave projects like redis & valkey with some decisions to make. 1) Keep jemalloc and accept things like memory leak bugs 2) Fork and maintain their own version of jemalloc. 3) Spend time replacing it entirely. 4) Hope someone else picks it up?
- senderista 1y agojemalloc is used enough at Amazon that it would make sense for them to maintain it, but that's not really their style.
- burnt-resistor 1y agoSome people believe everything must always be constantly tweaked, redone, broken and fixed, and churned for no reason. The only things that need to be fixed in mature, working software are bugs and security issues. It doesn't magically stop working or get "stale" unless dependencies, the OS, or build tools break.
- almostgotcaught 1y ago> Sometimes software development is largely "done" Lol absolutely not
- spookie 1y agoFirefox as well.
- perbu 1y agoBack in 2008-2009 I remember the Varnish project struggled with what looked very much like a memory leak. Because of the somewhat complex way memory was used, replacing the Glibc malloc with jemalloc was an immediate improvement and removed the leak-like behavior.
- technion 1y agoI know through years of looking at Ruby on Rails performance a commonly cited quick win was to run with jemalloc.
- swinglock 1y agoLast I checked Redis used their own fork of jemalloc. It may not even be updated to the latest release.
- jeffbee 1y agoThe article mentioned the influence of large-scale profiling on both jemalloc and tcmalloc, but doesn't mention mimalloc. I consider mimalloc to be on par with these others, and now I am wondering whether Microsoft also used large scale profiling to develop theirs, or if they just did it by dead reckoning.
- bch 1y agoHow does mimalloc telemetry compare to jemalloc?
- deleted 1y ago[deleted]
- poorman 1y agoHow cool would it be to see Doug Lea pick up the torch and create a modern day multi-threaded dlmalloc2!?
- ecshafer 1y agodl is just an observer on the open jdk governance board now, so he might have enough time.
- nevon 1y agoI very recently used jemalloc to resolve a memory fragmentation issue that caused a service to OOM every few days. While jemalloc as it is will continue to work, same as it does today, I wonder what allocator I should reach for in the future. Does anyone have any experiences to share regarding tcmalloc or other allocators that aim to perform better than stock glibc?
- kev009 1y agosnmalloc
- sanxiyn 1y agomimalloc is a good choice. CPython recently switched to mimalloc.
- beyonddream 1y agoTry mimalloc. I have prototyped a feature on top of mimalloc and while effort was a dead end, the code (this was around 2020) was nicely written and well maintained and it was fun to hack on it. When I swapped jemalloc in our system with mimalloc, it was on par if not better when it comes to fragmentation growth control and heap usage perspective.
- deleted 1y ago[deleted]
- meisel 1y agoI believe there’s no other allocator besides jemalloc that can seamlessly override macOS malloc/free like people do with LD_PRELOAD on Linux (at least as of ~2020). jemalloc has a very nice zone-based way of making itself the default, and manages to accommodate Apple’s odd requirements for an allocator that have tripped other third-party allocators up when trying to override malloc/free.
- glandium 1y agoNote this requires hackery that relies on Apple not changing things in its system allocator, which has happened at least twice IIRC.
- adgjlsfhk1 1y agoI believe mimalloc works here (but might be wrong).
- KingLancelot 1y ago[dead]
- chubot 1y agoNice post -- so does Facebook no longer use jemalloc at all? Or is it maintenance mode? Or I wonder if they could simply use tcmalloc or another allocator these days? Facebook infrastructure engineering reduced investment in core technology, instead emphasizing return on investment.
- Svetlitski 1y agoAs of when I left Meta nearly two years ago (although I would be absolutely shocked if this isn’t still the case) Jemalloc is the allocator, and is statically linked into every single binary running at the company. > Or I wonder if they could simply use tcmalloc or another allocator these days? Jemalloc is very deeply integrated there, so this is a lot harder than it sounds. From the telemetry being plumbed through in Strobelight, to applications using every highly Jemalloc-specific extension under the sun (e.g. manually created arenas with custom extent hooks), to the convergent evolution of applications being written in ways such that they perform optimally with respect to Jemalloc’s exact behavior.
- charcircuit 1y agoMeta has a fork that they still are working on, where development is continuing. https://github.com/facebook/jemalloc https://github.com/facebook/jemalloc
- burnt-resistor 1y agoThey take everything FLOSS and ruin it with bureaucracy, churn, breakage, and inconsideration to external use. They may claim FOSS broadly but it's mostly FOSS-washed, unusable garbage except for a few popular things.
- School-Cotton 1y agoReact, PyTorch, and RocksDB are all extremely significant. Not to mention them being one of the biggest contributors to the Linux kernel.
- nh2 1y ago
- kstrauser 1y agoI’ve wondered about this before but never when around people who might know. From my outsider view, jemalloc looked like a strict improvement over glibc’s malloc, according to all the benchmarks I’d seen when the subject came up. So, why isn’t it the default allocator?
- sanxiyn 1y agoAs far as I know there is no technical reason why jemalloc shouldn't be the default allocator. In fact, as pointed out in the article, it IS the default allocator on FreeBSD. My understanding is it is largely political.
- kstrauser 1y agoNow that I think about it, I could easily imagine it being left out of glibc because it doesn't build on Hurd or something.
- lloeki 1y ago> I could easily imagine it being left out of glibc because [...] ... its license is BSD-2-Clause ;) hence "political"
- vkazanov 1y agoHuh? Bsd-style licenses are fully compatible with gpl. The problem is exactly this: Facebook becomes the upstream of a key part of your system. And Facebook can just walk away from the project. Like it did just now.
- lloeki 1y agoThey are compatible but that's not the point. If it were included it would instantly become a LGPL hard-fork because of any subsequently added line of code, if not by "virality" of the glibc license, at least because any glibc author code addition would be LGPL, per GNU project policy/ideology. Also also this would he a hard bar to pass: https://sourceware.org/glibc/wiki/CopyrightFSForDisclaim https://sourceware.org/glibc/wiki/CopyrightFSForDisclaim As I recall this is what prevented Apple from contributing C blocks† back to upstream GCC. † https://github.com/lloeki/cblocks-clobj https://github.com/lloeki/cblocks-clobj
- skeptrune 1y agoKind of nuts that he worked on Jemalloc for over a decade while having personal preference for garbage collection. I'm surprised he doesn't have more regret.
- deleted 1y ago[deleted]
- kstrauser 1y agoWhy are those two mutually exclusive? I'd think that a high performance allocator would be especially crucial in the implementation of a fast garbage collected language. For example, in Python you can't alloc(n * sizeof(obj)) to reserve that much contiguous space for n objects. Instead, you use the builtins which isolate you from that low-level bookkeeping. Those builtins have to be pretty fast or performance would be terrible.
- procaryote 1y agoPython performance is terrible though...
- fermentation 1y agoA job is a job
- dikei 1y agoI still remember the day when I used jemalloc debug features to triage and resolve some nasty memory bloat issues in our code that use RockDB. Good times.
- userbinator 1y agoA bad choice of title, as "postmortem" made me think there was some severe outage caused by jemalloc.
- deleted 1y ago[deleted]
- chrisweekly 1y agoWell, that's not the only meaning of "postmortem". The fine article does open with, "The jemalloc memory allocator was first conceived in early 2004, and has been in public use for about 20 years now. Thanks to the nature of open source software licensing, jemalloc will remain publicly available indefinitely. But active upstream development has come to an end. This post briefly describes jemalloc’s development phases, each with some success/failure highlights, followed by some retrospective commentary."
- stingraycharles 1y agoI think this implies your understanding of the term “post-mortem” is incorrect, rather than the title.
- drysine 1y agoOr maybe not
- runevault 1y agopostmortem is looking back after an event. That can be a security event/outage, it can also be the completion of a project (see: game studios often do postmortems once their game is out to look back on what went wrong and right between preproduction, production, and post launch).
- gilgoomesh 1y agoIt's weird that we use "postmortem" in those cases since the word literally means "after death"; kind of implying something bad happened. I get that most of these postmortems are done after major development ceases, so it kind of is "dead" but still. Surely a "retrospective" would be a better word for a look back. It even means "look back.
- Svetlitski 1y agoI understand the decision to archive the upstream repo; as of when I left Meta, we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file (my favorite was the time someone filed an issue because our test suite didn’t pass on Itanium lol). Still, it makes me sad to see. Jemalloc is still IMO the best-performing general-purpose malloc implementation that’s easily usable; TCMalloc is great, but is an absolute nightmare to use if you’re not using bazel (this has become slightly less true now that bazel 7.4.0 added cc_static_library so at least you can somewhat easily export a static library, but broadly speaking the point still stands). I’ve been meaning to ask Qi if he’d be open to cutting a final 6.0 release on the repo before re-archiving. At the same time it’d be nice to modernize the default settings for the final release. Disabling the (somewhat confusingly backwardly-named) “cache oblivious” setting by default so that the 16 KiB size-class isn’t bloated to 20 KiB would be a major improvement. This isn’t to disparage your (i.e. Jason’s) original choice here; IIRC when I last talked to Qi and David about this they made the point that at the time you chose this default, typical TLB associativity was much lower than it is now. On a similar note, increasing the default “page size” from 4 KiB to something larger (probably 16 KiB), which would correspondingly increase the large size-class cutoff (i.e. the point at which the allocator switches from placing multiple allocations onto a slab, to backing individual allocations with their own extent directly) from 16 KiB up to 64 KiB would be pretty impactful. One of the last things I looked at before leaving Meta was making this change internally for major services, as it was worth a several percent CPU improvement (at the cost of a minor increase in RAM usage due to increased fragmentation). There’s a few other things I’d tweak (e.g. switching the default setting of metadata_thp from “disabled” to “auto”, changing the extent-sizing for slabs from using the nearest exact multiple of the page size that fits the size-class to instead allowing ~1% guaranteed wasted space in exchange for reducing fragmentation), but the aforementioned settings are the biggest ones.
- kstrauser 1y agoStuff like this is what keeps me coming back here. Thanks for posting this! What's hard about using TCMalloc if you're not using bazel? (Not asking to imply that it's not, but because I'm genuinely curious.)
- mavis 1y agoSwitching to jemalloc instantly fixed an irksome memory leak in an embedded Linux appliance I inherited many moons ago. Thank you je, we salute you!
- vlovich123 1y agoThat’s because sane allocators that aren’t glibc will return unused memory periodically to the OS while glibc prefers to permanently retain said memory.
- masklinn 1y agoglibc will return memory to the OS just fine, the problem is that its arena design is extremely prone to fragmentation, so you end up with a bunch of arenas which are almost but not quite empty and can't be released, but can’t really be used either. In fact, Jason himself (the author of jemalloc and TFA) posted an article on glibc malloc fragmentation 15 years ago: https://web.archive.org/web/20160417080412/http://www.canonware.com/~ttt/2009/05/mr-malloc-gets-schooled.html https://web.archive.org/web/20160417080412/http://www.canonw... And it's an issue to this day: https://blog.arkey.fr/drafts/2021/01/22/native-memory-fragmentation-with-glibc/ https://blog.arkey.fr/drafts/2021/01/22/native-memory-fragme...
- nh2 1y agoglibc does NOT return memory to the OS just fine. In my experience it delays it way too much, causing memory overuse and OOMs. I have a Python program that allocates 100 GB for some work, free()s it, and then calls a subprocess that takes 100 GB as well. Because the memory use is serial, it should fit in 128 GB just fine. But it gets OOM-killed, because glibc does not turn the free() into an munmap() before the subprocess is launched, so it needs 200 GB total, with 100 GB sitting around pointlessly unused in the Python process. This means if you use glibc, you have no idea how much memory your system will use and whether they will OOM-crash, even if your applications are carefully designed to avoid it. Similar experience: https://news.ycombinator.com/item?id=24242571 https://news.ycombinator.com/item?id=24242571 I commented there 4 years ago the glibc settings MALLOC_MMAP_THRESHOLD_ and MALLOC_TRIM_THRESHOLD_ should fix that, but I was wrong: MALLOC_TRIM_THRESHOLD_ is apparently bugged and has no effect in some situations. A bug I think might be involved: "free() doesn't honor M_TRIM_THRESHOLD" https://sourceware.org/bugzilla/show_bug.cgi?id=14827 https://sourceware.org/bugzilla/show_bug.cgi?id=14827 Open since 13 years ago. This stuff doesn't seem to get fixed. The fix in general is to use jemalloc with MALLOC_CONF="retain:false,muzzy_decay_ms:0,dirty_decay_ms:0" which tells it to immediately munmap() at free(). So in jemalloc, the settings to control this behaviour seem to actually work, in contrast to glibc malloc. (I'm happy to be proven wrong here, but so far no combination of settings seem to actually make glibc return memory as written in their docs.) From this perspective, it is frightening to see the jemalloc repo being archived, because that was my way to make sure stuff doesn't OOM in production all the time.
- p0w3n3d 1y agoThank you. Jemalloc was recently recommended to me on some presentation about Java optimization. I wonder if you did get everything you should from the companies that use it. I mean sometimes I feel that big tech firms only use free software, never giving anything to it, so I hope you were the exception here.
- masklinn 1y ago> jemalloc was probably booted from Rust binaries sooner than the natural course of development might have otherwise dictated. FWIW while it was a factor it was just one of a number: https://github.com/rust-lang/rust/issues/36963#issuecomment-252029017 https://github.com/rust-lang/rust/issues/36963#issuecomment-... And jemalloc was only removed two years after that issue was opened: https://github.com/rust-lang/rust/pull/55238 https://github.com/rust-lang/rust/pull/55238
- Aissen 1y agoInteresting that one of the factor listed in there, the hardcoded page-size on arm64, is still is an unsolved issue upstream, and that forces app developers to either ship multiple arm64 linux binaries, or drop support for some platforms. I wonder if some kind of dynamic page-size (with dynamic ftrace-style binary patching for performance?) would have been that much slower.
- pkhuong 1y agoYou can run jemalloc configured with 16KB pages on a 4KB page system.
- schrep 1y agoYour work was so impactful over a long period from Firefox to Facebook. Honored to have been a small part of it.
- lbrandy 1y agoSuppose this is as good a place to pile-on as any. Though this was not the post I was expecting to show up today, it was super awesome for me to get to have played my tiny part in this big journey. Thanks for everything @je (and qi + david -- and all the contributors before and after my time!).
- liuliu 1y agoYour leadership on continuing investing in core technologies in Facebook were as fruitful as it could ever being. GraphQL, PyTorch, React to name a few cannot happen without.
- deleted 1y ago[deleted]
- dao- 1y agoHmm, if I had to choose between not having Facebook and having React, I'd pick the former in a heartbeat. Not that this was a real choice, but it was nonetheless bitter to see colleagues join the behemoth that was Facebook.
- Omarbev 1y agoThis is a good thing
- adityapatadia 1y agoJason, here is a story about how much your work impacts us. We run a decently sized company that processes hundreds of millions of images/videos per day. When we first started about 5 years ago, we spent countless hours debugging issues related to memory fragmentation. One fine day, we discovered Jemalloc and put it in our service, which was causing a lot of memory fragmentation. We did not think that those 2 lines of changes in Dockerfile were going to fix all of our woes, but we were pleasantly surprised. Every single issue went away. Today, our multi-million dollar revenue company is using your memory allocator on every single service and on every single Dockerfile. Thank you! From the bottom of our hearts!
- laszlojamf 1y agoI really don't mean to be snarky, but honest question: Did you donate? Nothing says thank you like some $$$...
- onli 1y agoIt was a meta project and development ceased. For a regular project that expectation is fine, but here it does not apply IMHO.
- adityapatadia 1y agoWe regularly donate to project via open collective. We frankly did not see here due to FB involvement I think.
- thewisenerd 1y agoindeed! most image processing golang services suggest/use jemalloc the top 3 from https://github.com/topics/resize-images https://github.com/topics/resize-images (as of 2025-06-13) imaginary: https://github.com/h2non/imaginary/blob/1d4e251cfcd58ea66f8361f8721d7b8cc85002a3/Dockerfile#L90 https://github.com/h2non/imaginary/blob/1d4e251cfcd58ea66f83... imgproxy: https://web.archive.org/web/20210412004544/https://docs.imgproxy.net/#/memory_usage_tweaks?id=using-jemalloc https://web.archive.org/web/20210412004544/https://docs.imgp... (linked from a discussion in the imaginary repo) imagor: https://github.com/cshum/imagor/blob/f6673fa6656ee8ef17728f2eb3af466d541043dd/Dockerfile#L78 https://github.com/cshum/imagor/blob/f6673fa6656ee8ef17728f2...
- b0a04gl 1y ago[dead]
- brcmthrowaway 1y agoWhat allocator does Apple use?
- half-kh-hacker 1y agoyou probably want to look at their 'libmalloc'
- forty 1y agoProbably iMalloc ;)
- wiz21c 1y agoFTA: > And people find themselves in impossible situations where the main choices are 1) make poor decisions under extreme pressure, 2) comply under extreme pressure, or 3) get routed around. It doesn't sound like a work place :-(
- bravetraveler 1y agoSounds like every workplace I've 'enjoyed' since ~2008
- throwaway314155 1y agonice username - fsociety
- mrweasel 1y agoNow I'm not one for victim blaming, but if that's more than three places of employment, maybe you need to rethink the positions you apply for.
- acdha 1y agoThere’s something to that but it is victim blaming if you’re not acknowledging the larger trends. There are a lot of places whose MBAs are attending the same conferences, getting the same recommendations from consultants, and hearing the same demands from investors. The push against remote work, for example, was all driven by ideology against most of the available data but it affected a huge number of jobs.
- throw0101d 1y ago> The push against remote work, for example, was all driven by ideology against most of the available data but it affected a huge number of jobs. And before that, open office plans. You're saving on rent: great. But what is it doing to productivity? * https://business.adobe.com/blog/perspectives/what-science-says-about-open-offices https://business.adobe.com/blog/perspectives/what-science-sa... Of course productivity doesn't show up on a spreadsheet, but rent does, so it's what about "the numbers" say.
- the_mitsuhiko 1y agoAll the allocators have the same issue. They largely work against a shared set of allocation APIs. Many of their users mostly engage via malloc and free. So the flow is like this: user has an allocation looking issue. Picks up $allocator. If they have an $allocator type problem then they keep using it, otherwise they use something else. There are tons of users if these allocators but many rarely engage with the developers. Many wouldn’t even notice improvements or regressions on upgrades because after the initial choice they stop looking. I’m not sure how to fix that, but this is not healthy for such projects.
- Cloudef 1y agomalloc is bad api in general, if you want to go fast you don't rely on general purpose allocator
- const_cast 1y agoThis is true, but the unfortunate thing with how C and C++ were developed is that pretty much everything just assumes the existence of malloc/free. So if you’re using third-party libraries then it’s out of your control mostly. Linking a new allocator is a very easy and pretty much free way to improve performance.
- zozbot234 1y ago[flagged]
- hyperpape 1y agoThe maintainers are probably all making personally reasonable choices that we should support. But it’s still sad that there’s probably no world where someone will still focus on jemalloc with professional support from their employer. It means that an important piece of technology will not continue improving. Forking is possible, but it doesn’t look like the kind of project that many people could fork and improve, it requires a lot of focus by people with specific domain knowledge.
- 7bit 1y agoI don't understand why you don't understand that you can be sad about this. Parent stated that he's sad the project is no longer maintained. That's a perfectly reasonable and human response. Parent does not have to defend having an emotion, even less provide objective truth for why he feels sad. If you don't agree, fine. But I don't see why one would write a paragraph long statement, I validating someone's emotional response.
- zozbot234 1y ago> That's a perfectly reasonable and human response. It's hardly a reasonable response. By casually saying that it's "sad" when a maintainer gives up on a project, GP is also including a very real and heavy implication that this is somehow wrong on their part and that they should continue to shoulder that burden, as a demand from the community. This is a really unhealthy attitude and we should all do away with it. Instead, just acknowledge that the project is now there for the taking by anyone who may be interested. There's nothing "sad!" about this whatsoever, and we should not pretend that there is.
- lukeasrodgers 1y agoIf my kid is sad that she cannot have an extra granola bar, that does not imply that I am wrong to deny her, or even that she thinks I am wrong, it just means she wishes she could have one.
- dazzawazza 1y agoI've used jemalloc in every game engine I've written for years. It's just the thing to do. WAY faster on win32 than the default allocator. It's also nice to have the same allocator across all platforms. I learned of it from it's integration in FreeBSD and never looked back. jemalloc has help entertained a lot of people :)
- Iwan-Zotow 1y ago+1 windows def allocator is pos. Jemalloc rules
- ahartmetz 1y ago>windows def allocator is pos Wow, still? I remember allocator benchmarks from 10-15 years ago where there were some notable differences between allocators... and then Windows with like 20% the performance of everything else!
- ComputerGuru 1y agoIt’s improved considerably since.
- int_19h 1y ago> windows def allocator Which one of them? These days it could mean HeapAlloc, or it could mean malloc from uCRT.
- carey 1y agomalloc in uCRT just calls HeapAlloc, though? You can see the code in ucrt\heap\malloc_base.cpp if you have the Windows SDK installed. Programs can opt in to the _segment_ heap in their manifest, but it’s not necessarily any faster.
- mrweasel 1y agoLooking at all the comments and lightly browsing the source code, I'm amazed. Both at how much impact a memory allocator can make, but also how much code is involved. I'm not really sure what I expected, but somehow I expect a memory allocator to be ... smaller, simpler perhaps?
- ratorx 1y agoMemory allocators can be simple. In fact it was an assignment for a course in the 2nd year of my CS degree to make an (almost) complete allocator. However it is typically always more complex to make production quality software, especially in a performance sensitive domain.
- burnt-resistor 1y agoNaive allocators are very easy: just subdivide RAM and defragment only when absolutely necessary (if virtual memory is unavailable). Performant allocators are hard. I think we lost a great deal of potential when ORCA was too tied to Pony and not extracted to a framework, tool, and/or library useful outside of it such as integrated or working with LLVM.
- const_cast 1y agoIt’s the same way with garbage collectors. You can write a naive mark-and-sweep in an afternoon. You can write a reference counter in even less time. And for some runtimes this is fine. But writing a generational, concurrent, moving GC takes a lot of time. But if you can achieve it, you can get amazing performance gains. Just look at recent versions of Java.
- swinglock 1y agomimalloc is cleaner but lacks the very useful profiling features. To be fair it also has not gone through decades of changes as described in the postmortem either.
- senderista 1y agoYou can write a simple size-class allocator (even lock-free) in just a couple dozen lines of code. (I've done it both for interviews and for a work presentation.) But an allocator that is fast, scalable, and performs well over diverse workloads--that is HARD.
- burnt-resistor 1y agoLesson: Don't let one megacorp dominate or take over your FOSS project. Push back somewhat and say "no" to too much help from one source.
- igrunert 1y agoI think the author was happy to be employed by a megacorp, along with a team to push jemalloc forward. He and the other previous contributors are free to find new employers to continue such an arrangement, if any are willing to make that investment. Alternatively they could cobble together funding from a variety of smaller vendors. I think the author is happy to move on to other projects, after spending a long time in this problem space. I don’t think that “don’t let one megacorp hire a team of contributors for your FOSS project” is the lesson here. I’d say it’s a lesson in working upstream - the contributions made during their Facebook / Meta investment are available for the community to build upon. They could’ve just as easily been made in a closed source fork inside Facebook, without violating the terms of the license. Also Mozilla were unable to switch from their fork to the upstream version, and didn’t easily benefit from the Facebook / Meta investment as a result.
- ecshafer 1y agoHe worked for like a decade at Facebook it looks like. I would guess at least at a Staff level. How many millions of dollars do you think he got from that? It doesnt sound like the worse trade in the world.
- KingLancelot 1y ago[dead]
- curtisszmania 1y ago[dead]
- didip 1y agoThanks for everything, JE! jemalloc is always the first thing I installed whenever I had to provision bare servers. If jemalloc is somehow the default allocator in Linux, I think it will not have a hard time retaining contributors.
- soulbadguy 1y agoMaybe add a link to the post on the github repo. I feel like this is important context for people visiting the repo in the future