I agree that hardware/software codesign are critical to solving things like this, but features like prefetching, speculation, and prediction are absolutely crit

I agree that hardware/software codesign are critical to solving things like this, but features like prefetching, speculation, and prediction are absolutely critical to modern pipelines and broadly speaking are what enable what we think of as "modern computer performance." This has been true for over 20 years now. In terms of "overhead" it's not in the same ballpark -- or even the same sport, frankly -- as something like bounds checking or even garbage collection. Hell, if the difference was within even one order magnitude, they'd have done it already.