4 ms·
Working on the most recently retired instruction would be odd, since it'd lead to things like the cost of cache misses being assigned to instructions that have
by jsnell 9y ago
Working on the most recently retired instruction would be odd, since it'd lead to things like the cost of cache misses being assigned to instructions that have nothing to do with memory access.
But other than that, I've never been able to build a mental model for exactly how the accounting works. For example take an inner loop with no memory accesses, like this:
18: add $0x1,%rax
cmp %rax,%rdx
jne 18
On a Skylake this will execute once per cycle (IPC of ~3), so we're clearly spending equal amounts of time on each instruction. But perf will assign basically all of the cost to the jump.
0.28 │18:┌─→add $0x1,%rax
0.27 │ │ cmp %rax,%rdx
99.45 │ └──jne 18
So it could be consistent with some kind of "first in-flight" or "last in-flight" model. But change the loop to be 5 instructions long (and with an IPC of ~5, so still executing the whole loop once per cycle), and suddenly the PC gets smeared across two instructions:
2.12 │20:┌─→add %rsi,%rax
0.94 │ │ add $0x1,%rcx
45.36 │ │ add %rax,%rdx
0.10 │ │ cmp %rcx,%rsi
51.48 │ └──jne 20
Take an almost equivalent loop, just swapping two instructions. It's still 5 instructions and an IPC of ~5. And...
80.79 │20: add $0x1,%rcx
1.29 │ add %rax,%rdx
0.89 │ add %rsi,%rax
0.00 │ cmp %rcx,%rsi
17.02 │ ↑ jne 20
Multiple runs of an individual test case will be pretty consistent on a specific machine; so it's not that the assignment is just random. It's deterministic, but the exact mechanism is very opaque.
- tedunangst 9y agoOh, nice examples. (And yeah, had I thought another minute, the PC will never be actually retired, but something after.)
- ot 9y ago> it'd lead to things like the cost of cache misses being assigned to instructions that have nothing to do with memory access. But this happens all the time with perf. I often see cache misses attributed to a jump dependent on a data load.
- jsnell 9y agoSure, but that's still an in-flight instruction that is stalled waiting on a memory access (just indirectly). You don't see the miss being attributed to the instructions that preceded the load.
- brucedawson 9y agoTake a look at the "cmp ebx,eax" on line 24 of the spreadsheet linked from the article. ebx is only ever modified by pure register manipulations. eax was mostly recently modified by having a register added to it. And yet the cmp instruction is quite expensive - much more so than those that proceed it and which are executed equally frequently. So, somehow the cost of the r8d four instructions early must end up being propagated through the add in the previous instruction and then gets charged to the "innocent bystander" of the compare. Crazy. I would have expected the "add eax,r8d" on line 23 to pay the price.
- qb45 9y agoThis looks funny, it seems that RDI is never modified by the loop (?) so lines 18-24 always load the same data, presumably from L1. It's mysterious why of all those instructions, 20 and 24 get most interrupt hits.
- brucedawson 9y agoGood observation about rdi and its implications for the load of rcx (line 18) and therefore the load through rcx (line 20). I added a note to that effect and also enabled adding comments to the spreadsheet.