4 ms·
Saturating the execution ports in a modern CPU is something compilers demonstrably have a difficult time doing automatically because the the code involved in th
by jandrewrogers 10y ago
Saturating the execution ports in a modern CPU is something compilers demonstrably have a difficult time doing automatically because the the code involved in this optimization has poor locality by definition. Compilers tend to have a bit of tunnel vision to a very local context so that operation relationships are easily analyzed.
Optimizing this requires detecting patterns in scattered parts of the code and reorganizing the code in a functionally neutral way. Not difficult to do but the human programmer has to do the work because it requires a higher level view of code sequencing relationships. It doesn't produce code that is any less obvious or readable in my experience. Even on older Sandybridge CPUs, I often saw upwards of 2x throughput by mildly reorganizing perfectly correct C++ for better operation concurrency. Writing code that is well-optimized in this regard becomes natural with awareness and experience.
Optimizing cache line hit rates has an even higher throughput improvement than execution port saturation but is a similar optimization.
- dom0 10y agoI don't think it's even possible any more. Haswell and later are 8-wide designs, but can only fetch+decode and retire 4 instructions per cycle (peak). So one would need ops that decode to 2+ execution-unit independent µops. Linear data-dependent algorithms are also such a huge pain: "Why does that only shovel 500 MB/s?" Well... five instructions long dependency chain, that's why! (Yes, the last bit may be a bit unrelated to the discussion at hand)
- kbwt 10y ago> 8-wide designs, but can only fetch+decode and retire 4 instructions per cycle That's where the Trace Cache helps in tight loops.
- dom0 10y agoThe µop cache (trace cache went out the window) doesn't help with fetching instructions AIUI, but accelerates decoding them. So the 4 insns/cycle fetch bandwidth is not changed by that.
- ant6n 10y ago> five instructions long dependency chain, what's why! Skylake apparently has up to 224 instructions in flight. So during those dependent 5 instructions, theoretically it could be doing some of the next hundred. Or the previous hundred. I simply find it hard to believe that Intel would design monsters with so many out-of-order instructions, but still every random bit of software will constantly get completely stalled by dependent instructions.
- dom0 10y ago> So during those 5 instructions, theoretically it could be doing some of the next hundred. Not if the next iteration of the loop not only branches on the previous result (okay, that one's easy on the predictors), but also depends on the previous iteration :) -- The basic idea behind these designs is that a) most software is a fat, ugly blob of shit which sometimes calls into b) specific functions implementing high-performance algorithms (be it compression, encryption, hashing, ..., image manipulation, ...) So for b) you want a capable design to have high throughput, and that also plays well with a) because it means that the core can spend most of it's time blocking on memory or executing an instruction or two every couple cycles. It also means that SMT works very, very well, since now you got two fat blobs smearing their shit around, but the core has more than enough resources to deal with that. That's why SMT gives 50++ % speed-ups for many applications. Because they're fat blobs.
- ant6n 10y agoThat doesn't strike me as true. It appears more likely that these cpu monsters with OOE and register renaming are like just-in-time trace optimizers that can use run-time values to improve the optimization. That is, take shitty code, and try to run it as fast as possible, and try to squeeze multiple instructions per cycle out of it, by optimizing over hundreds of instructions in real time. If the intention was just to speed up hand-tuned library functions, it would be much easier to ask the library-hand-tuners to figure out instruction dependencies ahead of time (instead of doing re-ordering) and to provide a lot of registers (instead of doing renaming).
- 10y ago