7 ms·
Disclaimer: I'm not an allocator engineer, this is just an anecdote. A while back, I had a conversation with an engineer who maintained an OS allocator, and th
by favorited 1y ago
Disclaimer: I'm not an allocator engineer, this is just an anecdote.
A while back, I had a conversation with an engineer who maintained an OS allocator, and their claim was that custom allocators tend to make one process's memory allocation faster at the expense of the rest of the system. System allocators are less able to make allocation fair holistically, because one process isn't following the same patterns as the rest.
Which is why you see it recommended so frequently with services, where there is generally one process that you want to get preferential treatment over everything else.
- jeffbee 1y agoI don't think that's really a position that can be defended. Both jemalloc and tcmalloc evolved and were refined in antagonistic multitenant environments without one overwhelming application. They are optimal for that exact thing.
- favorited 1y agoIt's possible that they were referring to something specific about their platform and its system allocator, but like I said it was an anecdote about one engineer's statement. I just remember thinking it sounded fair at the time.
- vlovich123 1y agoThe “system” allocator is managing memory within a process boundary. The kernel is responsible for managing it across processes. Claiming that a user space allocator is greedily inefficient is voodoo reasoning that suggests the person making the claim has a poor grasp of architecture.
- jdsully 1y agoThe "greedy" part is likely not releasing pages back to the OS in a timely manner.
- nicoburns 1y agoThat seems odd though, seeing as this is one of the main criticisms of glibc's allocator.
- jeffbee 1y agoIn the containerized environments where these allocators were mainly developed, it is all but totally pointless to return memory to the kernel. You might as well keep everything your container is entitled to use, because it's not like the other containers can use it. Someone or some automatic system has written down how much memory the container is going to use.
- toast0 1y agoReturning no longer used anonymous memory is not without benefits. Returning pages allows them to be used for disk cache. They can be zeroed in the background by the kernel which may save time when they're needed again, or zeroing can be avoided if the kernel uses them as the destination of a full page DMA write. Also, returning no longer used pages helps get closer to a useful memory used measurement. Measuring memory usage is pretty difficult of course, but making the numbers a little more accurate helps.
- vlovich123 1y agoZeroed pages also compress more efficiently because the compressor doesn’t actually need to process them.
- atombender 1y agoI know Google has good engineering, but I find this a bit implausible? For most applications, especially request/response type apps like web servers, "right sizing" truly correctly while accounting for spikes takes a lot of engineering effort to fully account for how much allocation a single request will need, then ensuring the maximum concurrent requests never go beyond that so you never risk OOMs. I can see this being fine-tuned for extremely high-scale, core services like load balancers, SDNs, file systems etc., where you probably want to allocate all your data structures at startup time and never actually allocate anything after that, and you probably have whole teams of engineers devoted to just single services. But not most apps? Surely it's better for containers to share system memory, and rely on limits and resource-driven autoscaling to make the system resilient?
- jeffbee 1y agoThere are shared resources involved though, for example one process can cause a lot of traffic in khugepaged. However I would point out that is an endemic risk of Linux's overall architecture. Any process can cause chaos by dirtying pages, or otherwise triggering reclaim.
- vlovich123 1y agoThat’s generally true of any allocator and assuming glibc’s behavior would help mitigate this is critically not something kernel engineers design around nor something glibc allocator is trying to achieve as a design goal.
- favorited 1y agoFor context, the "allocator engineer" I was talking to was a kernel engineer - they have an extremely solid grasp of their platform's architecture. The whole advantage of being the platform's system allocator is that you can have a tighter relationship between the library function and the kernel implementation.
- vlovich123 1y agoI’m not generally aware of any system allocator that’s written hand in glove with the kernel’s allocator or somehow interops better for overall system efficiency at the cost of behavior in-app. Care to provide an example?
- lmm 1y ago> Both jemalloc and tcmalloc evolved and were refined in antagonistic multitenant environments without one overwhelming application. They are optimal for that exact thing. They were mostly optimised on Facebook/Google server-side systems, which were likely one application per VM, no? (Unlike desktop usage where users want several applications to run cooperatively). Firefox is a different case but apparently mainline jemalloc never matched Firefox jemalloc, and even then it's entirely plausible that Firefox benefitted from a "selfish" allocator.
- mort96 1y agoThe only way I can see that this would be true is if a custom allocator is worse about unmapping unused memory than the system allocator. After all, processes aren't sharing one heap, it's not like fragmentation in one process's address space is visible outside of that process... The only aspects of one process's memory allocation that's visible to other processes is, "that process uses N pages worth of resident memory so there's less available for me". But one of the common criticisms against glibc is that it's often really bad at unmapping its pages, so I'd think that most custom allocators are nicer to the system? It would be interested in hearing their thoughts directly, I'm also not an allocator engineer and someone who maintains an OS allocator probably knows wayyy more about this stuff than me. I'm sure there's some missing nuance or context or which would've made it make sense.