I believe that Linux has all the tools to solve this in practice: 1. When you allocate memory, but do not write into it yet, those pages not mapped in RAM and

I believe that Linux has all the tools to solve this in practice: 1. When you allocate memory, but do not write into it yet, those pages not mapped in RAM and thus don't occupy actual space. 2. When you're done with a specific page of memory, you can madvise(MADV_FREE) it, which means that the kernel can discard these pages from RAM and use it for caches and buffers. But you still hold on to the virtual memory allocation, so you can just start writing into the page again when you need more memory and the kernel will map it again. If I understand all that correctly, you can have your allocator work in such a way that it keeps a large reserve of preallocated memory pages, but the corresponding amount of RAM can be used for caches and buffers when it doesn't need everything. An interesting question would be how that scenario appears in ps(1) and top(1), i.e. whether those MADV_FREE'd pages would count towards RSS.