"In the current state of affair we are way better off leaving this to the hardware." That is something I'm not so sure about. That might have been true before,

"In the current state of affair we are way better off leaving this to the hardware." That is something I'm not so sure about. That might have been true before, when the hardware served well the then simple computational model of a single high frequency processing core. Things got more complicated since the times of first CPU cache offerings and hardware general use-case implementation can only give you so much. The performance boost that once could be relied on without much care now dissipated. This cache thing now become just a leaky abstraction. We can not abstract it away completely if we care for performance, nor we can control it in any meaningful way. It is not a good model anymore. Just recognize it as a failed experiment. "Unless you want programmers to explicitely manage this fast memory..." Yes, that's exactly what I have in mind (and implied before). Most applications do not need maximal performance and can disregard the fast memory completely. That will only mean that those processes either will not consume their share and leave it more for the processes that need it, or that the fast memory will (occasionally) be used indirectly, through the optimized bits in the layers that the given process relies on. This memory, however, won't be unproductively overwritten or invalidated on every cold cache or whatnot.