> Cache is seen as an optimization for small data: if it fits in L2, it’s going to be processed faster Nobody worth their salt believes just this and nothing e

> Cache is seen as an optimization for small data: if it fits in L2, it’s going to be processed faster Nobody worth their salt believes just this and nothing else. Yes, if the data fits entirely into a given cache, that's a nice case that's easy to reason about. No matter what access pattern is applied to the data, it doesn't matter because it's in the cache. Hopefully everyone working with caches understands that they provide a potential speedup when not everything fits into the cache, and that this depends on the pattern of access (mainly, does it exhibit "locality"). Moreover, this case is extremely important. The article gives an example of exactly that: improving the locality of access. If you don't know this, you don't know one of the first facts about caching. There is something else to know about: you can't tell by size alone whether a given data set will fit into a cache. The problem is that caches are not always fully associative. In a set associative cache, a given block of data cannot be stored in any cache line: it is assigned to a small set of possible cache lines. Then within a set, the cache lines are dynamically allocated and tagged. A given bunch of working which appears to be just smaller than the cache might be arranged in such a poor way in memory that it doesn't map to all of the cache's sets. And so, it actually does not fit into the cache.