3 ms·
"Without repacking, the microkernel would have to read rows of A separated by potentially large stride. If this stride happens to be a multiple of a large power
by DoctorOetker 8y ago
"Without repacking, the microkernel would have to read rows of A separated by potentially large stride. If this stride happens to be a multiple of a large power of 2, elements from different rows of A in the panel may fall into the same cache set. If the number of colliding rows exceeds cache associativity, they evict each other and performance falls off a cliff. Fortunately, this situation cannot happen when the panel fits into L1, as with the models for which QNNPACK is optimized."
I only very roughly understand the concepts of caching and eviction algorithms, but I wish processors and OS'es (which presumably can configure cache behaviour) would expose the current cache configuration so that compilers and library designers can take this into account in a more automated/uniform way. Alternatively how accurately do emulators model cache behaviour? It would be nice to see where a bottleneck is, or how long ago a needed value was evicted for profiling purpouses