4 ms·

If it increases performance by 5% then 5% of execution time was spent missing branches and instruction cache misses... Which sounds utterly implausible. Convent

If it increases performance by 5% then 5% of execution time was spent missing branches and instruction cache misses... Which sounds utterly implausible. Conventional wisdom has it that instruction caching is not a problem because whatever the size of the binary it is dwarfed by the size of the data. And hot loops are generally no more than a few KBs at most anyway. I'm skeptical.

lawlessone 2y ago
>5% of execution time was spent missing branches and instruction cache misses... Which sounds utterly implausible. You could be right, but this reminds of the "efficient markets" joke. Wherein two economists refuse to believe the $100 the see on the ground is real because "someone would have picked it up" :-)
maxpan 2y ago
The performance loss due to cache misses in data-center applications far exceeds 5%. Combined data and instruction cache misses are contributing to more than half of stalled cycles. The following publication by Google from 2015 goes into details: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44271.pdf https://static.googleusercontent.com/media/research.google.c... "Our results demonstrate a significant and growing problem with instruction-cache bottlenecks. Front-end core stalls account for 15-30% of all pipeline slots, with many workloads showing 5-10% of cycles completely starved on instructions (Section 6)."
- bjourne 2y ago
  You're conflating data misses with instruction misses. Besides, the article is about hundred-megabytes-binaries (forgot to strip debug data?), which the Linux kernel isn't.
  - maxpan 2y ago
    The quote above talks exclusively of instruction cache misses. In case you are really interested, the two kinds are related as L2 and L3 caches are shared my instructions and data. In terms of the execution profile, the kernel is very close to a typical WSC application. I.e. very flat without hotspots. The size of L1 I$ is 32KB, hence your application doesn't have to have 100s of megabytes of code to benefit from layout optimizations.
tardy_one 2y ago
5% doesn't sound like what you would get addressing the lowest hanging fruit.. Which makes sense because the Linux kernel's primary bloat is mostly in the form of drivers that never load on any specific system. So sure, a pretty good result for not a very relevant optimization but I'd be disappointed if I built this tool specifically for the kernel instead of applied it because I had it..
fooker 2y ago
You can measure it yourself, this is not a philosophical thing you have to be skeptical about. It's not that 5% of the time was spent in icache misses, it’s more about cascading benefits of hitting the cache or predicting a branch correctly. You get more instruction level parallelism, less stalls, and in general more straightline execution without hitting memory.

lawlessone 2y ago

>5% of execution time was spent missing branches and instruction cache misses... Which sounds utterly implausible. You could be right, but this reminds of the "efficient markets" joke. Wherein two economists refuse to believe the $100 the see on the ground is real because "someone would have picked it up" :-)

maxpan 2y ago

The performance loss due to cache misses in data-center applications far exceeds 5%. Combined data and instruction cache misses are contributing to more than half of stalled cycles. The following publication by Google from 2015 goes into details: https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44271.pdf https://static.googleusercontent.com/media/research.google.c... "Our results demonstrate a significant and growing problem with instruction-cache bottlenecks. Front-end core stalls account for 15-30% of all pipeline slots, with many workloads showing 5-10% of cycles completely starved on instructions (Section 6)."

bjourne 2y ago
You're conflating data misses with instruction misses. Besides, the article is about hundred-megabytes-binaries (forgot to strip debug data?), which the Linux kernel isn't.
- maxpan 2y ago
  The quote above talks exclusively of instruction cache misses. In case you are really interested, the two kinds are related as L2 and L3 caches are shared my instructions and data. In terms of the execution profile, the kernel is very close to a typical WSC application. I.e. very flat without hotspots. The size of L1 I$ is 32KB, hence your application doesn't have to have 100s of megabytes of code to benefit from layout optimizations.

tardy_one 2y ago

5% doesn't sound like what you would get addressing the lowest hanging fruit.. Which makes sense because the Linux kernel's primary bloat is mostly in the form of drivers that never load on any specific system. So sure, a pretty good result for not a very relevant optimization but I'd be disappointed if I built this tool specifically for the kernel instead of applied it because I had it..

fooker 2y ago

You can measure it yourself, this is not a philosophical thing you have to be skeptical about. It's not that 5% of the time was spent in icache misses, it’s more about cascading benefits of hitting the cache or predicting a branch correctly. You get more instruction level parallelism, less stalls, and in general more straightline execution without hitting memory.