Personally, trying to automagically cache controlled at the hardware level has... issues. It far too often is/becomes a leaky abstraction. The problem is that

Personally, trying to automagically cache controlled at the hardware level has... issues. It far too often is/becomes a leaky abstraction. The problem is that hardware is too inflexible. And trying to explicitly manage things in software, that is from the operating system, has other issues. It's expensive, both in terms of time taken for the context switch / etc - and it tends to wipe out a large chunk of cache too. Modern processors are mainly cache, in terms of die area. There's plenty of room for adding new bits and pieces if it helps speed things up. So, what's between the hardware and the OS? Well, what about an FPGA? So you have the hardware read cache if present, but have the FPGA make the decisions about how to manage the cache. What to keep, what to toss, that sort of thing. This FPGA could also be used for basic system calls / scheduling, and perhaps even for specialized disk access (read: something like "this process can read in this range without consulting the OS") (As OS time can be most of the latency when reading from an SSD.) - with a system to fallback to a full syscall if necessary. You could perhaps even implement the branch prediction unit in the FPGA.