Unlike some other optimizations, the effects of locality optimization are usually not local. The reduction in cache and bandwidth pressure makes everything run

Unlike some other optimizations, the effects of locality optimization are usually not local. The reduction in cache and bandwidth pressure makes everything run better. These are common optimizations in e.g. databases, even shaving a few bytes off of key data structures can result in system-wide gains of 5-10% performance just from the cache pressure reduction resulting from a seemingly innocuous reduction in data size. You don’t need to measure it to know it will have impact, the relationship between reducing cache pressure and performance is pretty direct and repeatable. Spatial locality is straightforward, temporal locality is a bigger engineering commitment. A canonical example of locality optimization is thread-per-core architectures versus classic multi-threading architectures. By random luck, I was around when thread-per-core architectures were first developed to improve efficiency on many-core processors (on supercomputers). Instant 10x performance improvement on the same hardware. By comparison, aggressive vectorization with AVX-512 might buy you 3x improvement. Locality optimizations are the highest leverage optimization we can do today. It does require disciplined software architecture. There are limits to locality optimization in real systems. If you are doing a good job of it, you frequently run out of memory bandwidth at some point. That’s a pretty good outcome.