Strongly agree with all of this. I think PHK is right that you don't want user-space and the kernel to fight, but the solution isn't "leave it all to the kerne

Strongly agree with all of this. I think PHK is right that you don't want user-space and the kernel to fight, but the solution isn't "leave it all to the kernel," it's "leave it out of the kernel." The fact that you can outperform the kernel's abstractions (as you mentioned) clearly demonstrates that the kernel's algorithms aren't "one size fits all." So why are they in the kernel at all? Thank goodness that facilities like O_DIRECT let you bypass them (even though Linus thinks you should never do this: https://lkml.org/lkml/2007/1/10/233 https://lkml.org/lkml/2007/1/10/233). I am not arguing that everyone should roll their own memory management. We have a lot of sophisticated ways of sharing code in user-space, like shared libraries. The beautiful thing about user-space libraries is that you voluntarily choose them based on the value they provide to you. If caching library A gets one-up'd by caching library B, competitive forces will allow the upstart competitor to displace the incumbent. But OS kernels are in a privileged position; they have exclusive access to the hardware. It's not easy to just drop in a competing OS scheduler, even though other approaches may be better than what Linux does. That's why kernels should contain as little functionality as possible. Another reason is accounting and isolation. The resources used by kernel-space are shared between processes, which can lower overall resource use but also means that no user-space process can be "charged" for them or guaranteed an isolated share of them. For this reason, some systems I've been in contact with lately actually do all disk caching in user-space, so that (for example) a huge read from one process can't evict all the pages from someone else's process.