Consider this scenario: Say one execution thread's working memory footprint just barely fits in cache. Now with 2 execution threads sharing that same slice of c

Consider this scenario: Say one execution thread's working memory footprint just barely fits in cache. Now with 2 execution threads sharing that same slice of cache, you have more cache misses and trashing. And cache misses can have disastrous effect on performance. It doesn't matter that there's CPU available for hyperthreading to use during the cache misses. Visually, say we have 4 slots of cache available, we'll mark cache misses with capital letters, and we have an algorithm that goes like time goes this way--> ABCDabcdabcdabcdabcd Now put two threads there AB C D A EF G H E ^--cache now contains DAHE ^--cache now contains CDGH ^--cache now contains BCFG Related, I've seen situations where algorithm 1 wins every microbenchmark, while algorithm 2 is faster in the real app. Why? Because alg1 needed more icache space than the simpler alg2, and in the real app there was more icache pressure than in the microbenchmark.