3 ms·
This is definitely not the case, x86 chips can do a load+arith_op or a store+arith_op in the same uop. I don't think so? While these operations can be written
by nkurz 12y ago
This is definitely not the case, x86 chips can do a load+arith_op or a store+arith_op in the same uop.
I don't think so? While these operations can be written with a single x86 assembly instruction, internally they are split into different µops which execute on different ports. Here's Agner's chart[1] for ADD and SUB on Haswell (with some typographic editing for clarity):
inst operands fused unfused ports latency throughput
ADD SUB r <- r/i 1 1 p0156 1 0.25
ADD SUB r <- m 1 2 p0156 p23 0.5
ADD SUB m <- r/i 2 4 2p0156 2p237 p4 6 1
This is saying that adding register (r) to a register or an immediate (i) takes one µop which executes on any one of Ports 0, 1, 5, or 6, has a latency of 1 cycle, and that 4 of these can be executed per cycle.
For an add that comes from memory with the result put in the register, the instruction is decoded as a single µop (fused), but this is executed as two separate µops: one on Port 0, 1, 5, or 6; and one on Port 2 or 3. Latency is unspecified (depends on the source of the load), and you can execute a maximum of 2 per cycle.
For an adding a register with a memory location and storing the result to memory, the single x86 assembly instruction decodes to 2 fused µops, which then execute as 4 unfused µops: two on any combination of Ports 0, 1, 5, or 6; one on Port 2, 3, or 7; and one on Port 4. The latency is at least 6 cycles, and at best you can sustain one of these per cycle.
The uops/instruction number looks bogus. I remember those numbers being very close 1. The cycles per uop of 0.37 seems wildly optimistic. I remember numbers closer to 1.
It's easier now than then, but depending on workload, executing 3 µops per cycle (.33) is easily achievable for optimized code. The current limit is essentially 4 (.25), but even this is within reach. I recently had a pathological case where I sustained almost 5 µops executed per cycle (.21), which at first glance is impossible since only 4 µops can be issued or retired per cycle. I think what was happening was that due to failed memory disambiguation, some of the stores and loads were being executed twice before being retired.
[1] http://www.agner.org/optimize/microarchitecture.pdf http://www.agner.org/optimize/microarchitecture.pdf
- wsxcde 12y agoThe paper is concerned with the front end, so the fused uops are what matter there even if they get split up for the actual execution. Similarly for structures like ROB, you'll artificially inflate occupancy, and possibly create fake bottlenecks, if you split up uops too much. But yes, if you're pointing out a add reg, men instruction actually gets executed in two places, you're right. Also for the uops per cycle, I don't doubt you can get really nice numbers for tight highly optimized. But if you're running entire applications like Winzip, which do a whole lot of things including lots of I/O and system calls, I doubt that you'll still get those numbers.
- Marat_Dukhan 12y agouop fusion was added in Pentium-M, the Pentium III modeled in the paper didn't have it.
- brigade 12y agoThe decoders have a throughput of 4 fused uops or instructions per cycle, but are limited to 16 bytes per cycle (which tends to become the limit if you attempt to push >4 unfused uop perc cycle). But in Haswell, the uop cache has double that size limit, plus it counts late-fused uops as one, so it's not that impossible to sustain >4 unfused uop per cycle in certain loops... Agner puts the limit at 6 instructions, so at least 6 unfused uop per cycle (maybe 8) Renaming or retirement could bottleneck, but Agner hasn't found them to bottleneck in Haswell so that's more theoretical.
- nkurz 12y agoThe bright part is that tight loops are almost always going to be coming out of the ~1500 µop decoded instruction cache or the 28 µop Loop Stream detector. These have the same issue limit of 4 µops per cycle, but count both macro- and micro-fused µops as 1. I'm not certain how they are treated at retirement. Is it the same? A big surprise to me recently was learning that having the number of µops in a tight loop be a multiple of 4 can make a big difference. I put up what I think is a mind-bending example last week: https://gist.github.com/nkurz/1ecb46486746c4e078b3 https://gist.github.com/nkurz/1ecb46486746c4e078b3 Perversely, in this example making the loop slower actually has the effect of making the run time faster.
- pbsd 12y agoI don't think the number of uops in the loop being a multiple of 4 is the factor there. To see this, replace `ADD RAX, 1` with `INC RAX`, which does the exact same thing but does not alter the carry flag. The number of uops remains the same, but the runtime difference vanishes. I can't verify this at the moment, but my guess is that the true effect of SBB is to partially serialize the loop, therefore avoiding numerous store-forwarding and other pesky memory subsystem stalls down the line.
- nkurz 12y agoI'd love for you to dive into this one and see where you get. I think this example has a number of confusing layers. My conclusion after staring at it a long time was that the difference between SBB and SUB is that SUB/JNE allows macro fusion and leaves the loop with 4 µops, whereas SBB JNE does not fuse, leaving the loop with 5 µops. Having 5 µops slows the loop down by a cycle, which as you guessed, saves us from a problem caused by issuing the load too soon after the store, although from measuring events, I'm pretty sure this isn't related to store-forwarding. The relevant sections in the Intel Optimization manual are 3.4.2.5 for the µop multiples of 4, and 3.6.5.2 for the load too quickly after store. I'd love to understand better what's actually happening during that delay. Measurements seem to say that it causes the load to be re-executed multiple times.