> A total cache miss to main mem is far too many cycles, OOO can't hide that. That view is only a very partial understanding of the bigger story, and is missin

> A total cache miss to main mem is far too many cycles, OOO can't hide that. That view is only a very partial understanding of the bigger story, and is missing the key benefit of OOO. The key benefit of OOO and speculative execution is that they allow you to generate multiple outstanding L2/L3 misses in flight. Memory latency is hundreds of cycles, and if you can get one more memory read in flight while there's already one miss, you cut the effective stall time for memory significantly. OOO and speculative execution help not only because they allow the pipeline to run more instructions while being stalled, they help because that extra execution can create multiple outstanding cache misses, effectively reducing the stall cycles due to cache misses.