4 ms·

I still don't have an answer, but here are some more readings that might give someone a hint. Fast is with .p2align 3, Slow is with .p2align4 as above. Measur

by nkurz 9y ago

I still don't have an answer, but here are some more readings that might give someone a hint. Fast is with .p2align 3, Slow is with .p2align4 as above. Measurements are on Skylake with Likwid (https://github.com/RRZE-HPC/likwid/wiki/likwid-perfctr https://github.com/RRZE-HPC/likwid/wiki/likwid-perfctr). Fast Slow Runtime (RDTSC) [s] | 4.912542e-01 | 6.476124e-01 RESOURCE_STALLS_ANY | 135,743,115 |1,298,835,857 IDQ_DSB_UOPS |1,441,709,567 | 97,635 LSD_UOPS | 197,902,567 |1,999,996,841 IDQ_MITE_UOPS | 360,547,507 | 82,354 INSTR_RETIRED_ANY | 1600095725 | 1600095769 CPU_CLK_UNHALTED_CORE | 1662620966 | 2192914751 Again, these are the same instructions just with different alignment. I assume that Fast vs Slow is essentially arbitrary, although there may be cache line effects such that only the smaller alignment has the chance of being fast. First, RESOURCE_STALLS_ANY is picking up some important difference. Slow has lots, Fast has fewer. Breaking these down to see what subtype it is might give an exact answer: https://download.01.org/perfmon/index/skylake.html https://download.01.org/perfmon/index/skylake.html. It's not RESOURCE_STALLS_SB that changes, but the others don't seem to be predefined. LSD_UOPS is the number of µops delivered by the "Loop Stream Detector", which is the smallest and fastest cache for instructions. It's very small, and only works for tiny loops. Usually this is where you want the µops to stream from, but in this case using it results in a slower final result. IDQ_DSB_UOPS is the number of µops delivered from the "Decode Stream Buffer". This is usually slightly slower than the "Loop Stream Buffer", but in this case using it produces a faster result. This is large enough to hold a reasonably sized function. IDQ_MITE_UOPS is the "legacy" instruction decoder. This is the normal path by which instructions are decoded into µops. It's always used the first time instructions are encountered, but for hot loops and hot functions the results can be cached. Usually you'd like to see less of this, but here using the legacy decoding path gives a faster result. What does this mean? It probably means that something about the alignment combined with the jump defeats what would normally be the fastest µop cache. This isn't that strange in itself, but the oddity is that for some reason, this causes an overall faster result. Why? I'd thought it was something to do with "store forwarding", and it still might be, but I haven't found any counters that directly point to that as being the problem. Instead, I think it might have to do with the processor getting too far ahead on the reads and stores, and then having to redo them once it realizes that the data is out of date. Somehow having fewer instructions available in the reorder buffer (because they are coming in from a slightly slower decoder) improves overall performance. But I don't yet understand the specifics. I'm going to stop for the night, but I'd love to wake up to someone revealing the "true" answer!

pbsd 9y ago
We are observing different effects. I suggest you apply the latest microcode update, seeing that you have LSD_UOPS being dispatched; the latest update disabled the LSD altogether, due to the SKL150 CPU errata. The LSD was also permanently disabled in the newer Skylake-X, probably for the same reason. In any case, I have to admit being wrong; port 6 has nothing to do with it, though the general principle of stalling the frontend to prevent the backend being overloaded still holds. For example, here's how to get the speedup without any change in control flow: mov rcx, 400000000-1 align 32 nop up: align 32 add rcx, [counter] align 32 dec rcx align 32 jnz up ret The align directives here must resolve to an appropriate series of multi-byte nops. What we are doing here is to purposefully waste the uops cache. Each useful instruction comes at the beginning of a new 32-byte uop cache line, and the CPU cannot load more than 1 cache line per cycle. Effectively, we're limiting the rate of decoding by adding nops, which will not result in actual uops after decoding. I suspect the CALL and JMP instructions in the other variants were doing the same thing, but in a more obscure way. It's possible that CALL disables the LSD, which would matter in Sandy Bridge, but not in (updated) Skylake. As to why this actually makes things faster, I don't know. There are many more stalls owing to the reservation station being full in the tight loop: 1,348,491,506 resource_stalls_any 1,694,613,980 uops_issued_stall_cycles 759,763,767 uops_executed_stall_cycles 994,974,421 uops_retired_stall_cycles vs 109,300,935 resource_stalls_any 120,303,318 uops_issued_stall_cycles 164,322,430 uops_executed_stall_cycles 15,834,456 uops_retired_stall_cycles But how does this translate to the observed performance penalty? One clue lies in the port breakdown: 756,686,062 uops_dispatched_port_port_0 782,579,771 uops_dispatched_port_port_1 279,387,822 uops_dispatched_port_port_2 314,214,218 uops_dispatched_port_port_3 1,994,313,938 uops_dispatched_port_port_4 807,514,909 uops_dispatched_port_port_5 1,050,884,205 uops_dispatched_port_port_6 209,316,457 uops_dispatched_port_port_7 vs 640,957,968 uops_dispatched_port_port_0 637,621,266 uops_dispatched_port_port_1 399,224,647 uops_dispatched_port_port_2 400,205,411 uops_dispatched_port_port_3 1,610,294,604 uops_dispatched_port_port_4 643,780,748 uops_dispatched_port_port_5 848,142,785 uops_dispatched_port_port_6 470,700 uops_dispatched_port_port_7 Focus on port 4, the only port available for memory stores. It does nothing else, so ideally we would expect ~400000000 uops to be dispatched to port 4. Instead, we have ~4x that for the fast loop, and ~5x that for the slow loop. These, presumably, are speculative stores that had to be invalidated. By dispatching fewer instructions that we can handle per cycle, there is less speculation and thus less wasted work?

by nkurz 9y ago

pbsd 9y ago

We are observing different effects. I suggest you apply the latest microcode update, seeing that you have LSD_UOPS being dispatched; the latest update disabled the LSD altogether, due to the SKL150 CPU errata. The LSD was also permanently disabled in the newer Skylake-X, probably for the same reason. In any case, I have to admit being wrong; port 6 has nothing to do with it, though the general principle of stalling the frontend to prevent the backend being overloaded still holds. For example, here's how to get the speedup without any change in control flow: mov rcx, 400000000-1 align 32 nop up: align 32 add rcx, [counter] align 32 dec rcx align 32 jnz up ret The align directives here must resolve to an appropriate series of multi-byte nops. What we are doing here is to purposefully waste the uops cache. Each useful instruction comes at the beginning of a new 32-byte uop cache line, and the CPU cannot load more than 1 cache line per cycle. Effectively, we're limiting the rate of decoding by adding nops, which will not result in actual uops after decoding. I suspect the CALL and JMP instructions in the other variants were doing the same thing, but in a more obscure way. It's possible that CALL disables the LSD, which would matter in Sandy Bridge, but not in (updated) Skylake. As to why this actually makes things faster, I don't know. There are many more stalls owing to the reservation station being full in the tight loop: 1,348,491,506 resource_stalls_any 1,694,613,980 uops_issued_stall_cycles 759,763,767 uops_executed_stall_cycles 994,974,421 uops_retired_stall_cycles vs 109,300,935 resource_stalls_any 120,303,318 uops_issued_stall_cycles 164,322,430 uops_executed_stall_cycles 15,834,456 uops_retired_stall_cycles But how does this translate to the observed performance penalty? One clue lies in the port breakdown: 756,686,062 uops_dispatched_port_port_0 782,579,771 uops_dispatched_port_port_1 279,387,822 uops_dispatched_port_port_2 314,214,218 uops_dispatched_port_port_3 1,994,313,938 uops_dispatched_port_port_4 807,514,909 uops_dispatched_port_port_5 1,050,884,205 uops_dispatched_port_port_6 209,316,457 uops_dispatched_port_port_7 vs 640,957,968 uops_dispatched_port_port_0 637,621,266 uops_dispatched_port_port_1 399,224,647 uops_dispatched_port_port_2 400,205,411 uops_dispatched_port_port_3 1,610,294,604 uops_dispatched_port_port_4 643,780,748 uops_dispatched_port_port_5 848,142,785 uops_dispatched_port_port_6 470,700 uops_dispatched_port_port_7 Focus on port 4, the only port available for memory stores. It does nothing else, so ideally we would expect ~400000000 uops to be dispatched to port 4. Instead, we have ~4x that for the fast loop, and ~5x that for the slow loop. These, presumably, are speculative stores that had to be invalidated. By dispatching fewer instructions that we can handle per cycle, there is less speculation and thus less wasted work?