5 ms·
Micro-op fusion in x86
- monocasa 9y ago>> Micro-fusion improves instruction bandwidth delivered from decode to retirement and saves power. Coding an instruction sequence by using single-uop instructions will increases the code size, which can decrease fetch bandwidth from the legacy pipeline. > Out of all runs that I did unfused version was never faster then the fused one. I mean, yeah. These little microbenchmarks wouldn't cause the kind of I$ pressure that Intel's talking about.
- stmw 9y agoNice article. I wish it also discussed the potential value of future-proofing the code (for future Intel or - more important - Intel-compatible CPUs), in which case wouldn't it make sense to use 3 simpler instructions? Even at the risk of instruction cache pressure. It's not worse here, since the problem was alignment rather than instruction selection.
- userbinator 9y agoin which case wouldn't it make sense to use 3 simpler instructions? No. Any competitor's CPUs will also have similar constraints on fetch and decode bandwidth, so smaller code that isn't slower will always be the better choice (and sometimes, even slightly slower for one sequence is better if it being smaller results in less cache misses overall). Plus, the requirement to use an extra register for the simpler sequence instead of the implicitly allocated ones that memory-ops use will affect other code nearby.
- amluto 9y agoThis article appears to have nothing to do with micro-op fusion, which is an internal optimization in some CPUs. See, for example, section 8.4 in Anger Fog manual [1]. This article is instead about whether a particular operation is better written as a single-instruction memory RMW or as there separate instructions to read, modify, and write. [1] http://www.agner.org/optimize/microarchitecture.pdf http://www.agner.org/optimize/microarchitecture.pdf
- chrisseaton 9y agoIntel says that: > Micro-fusion fuses multiple micro-ops from the same instruction into a single complex micro-op. And that: > Macro-fusion merges two instructions to a single micro-op. The point is that the three instruction format and the single instruction format you would think would decode both to three micro-ops (or something like that), but instead the single instruction format can use a single fused micro op. That's why it's better to write as a single instruction. And your reference backs this up - it talks about single instructions in this section on micro-op fusion. You may be confusing it with macro-fusion, which is about fusing multiple logical instructions, which is discussed in the next section in that Agner document, but isn't discussed in this blog. > This article is instead about whether a particular operation is better written as a single-instruction Yes - it's better to write as a single-instruction, because then micro-op fusion applies. Macro-op fusion isn't as strong, as doesn't apply to the three logical instructions.
- deleted 9y ago[deleted]
- pbsd 9y ago> The point is that the three instruction format and the single instruction format you would think would decode both to three micro-ops (or something like that), but instead the single instruction format can use a single fused micro op. That's why it's better to write as a single instruction. This is not exactly what happens. A read-modify-write instruction in general results in 2 fused uops, one for the read+op, the other for the address generation+write. In the specific case of INC, you still have 3 fused uops for unclear reasons---possibly because INC only partially changes the flags. Replacing the expression by ADD [memory], 1 results on the expected 2 fused ops.
- jnordwick 9y agoDo the uop counters he's printing out count fused uops as only 1 uop or do they do something like count the uops separate for uop dispatch and as one for retire?
- userbinator 9y agoThis "it's all RISC-like uops anyway" is what lead many (incidentally mostly academic, open-source) compiler writers to not attempt to emit the more complex instructions, thinking that the equivalent series of simpler ones (which uses an explicit register) would be just as fast (because it is, in microbenchmarks)... Unfused instructions also add register pressure to the compiler, because it needs to find the free register to load the value from the memory. ...and then they complain about the lack of registers, when, to put it bluntly, "they're doing it wrong". x86 has fewer registers than most RISCs but makes up for that with memory-register and memory-immediate operations. Instead of wasting an architectural register, you're supposed to use those operations which automatically "allocate" and use a virtual register from the register renamer. Look at what icc (Intel's own compiler)C does; it definitely will choose the ALU m, r or ALU m, i operations when it makes sense to. As others here have already noted, the size savings definitely have an effect on macrobenchmarks.
- Y_Y 9y agoicc is awkward to work with, but crazy fast for the type of code that I write. It seems to optimise much more aggressively than clang or gcc. Pity it's not open source.
- userbinator 9y agoIt seems to optimise much more aggressively than clang or gcc. The other thing worthy of note is that icc seems to be far less aggressive at exploiting undefined behaviour than e.g. clang (which it is extremely notorious for), contradicting their notion/theory that exploiting UB is necessary for good optimisation. From what I've seen, icc's main strength is in instruction selection and scheduling.
- remcob 9y agoOptimizations based on language semantics (clang) and instruction selection (icc) should be quite complementary, so we could have a best-of-both compiler.
- 9y ago
- jnordwick 9y agoHis terminology appears to be weird and i don't think he's testing what he's trying to test. 1- using the "INC mem" form is not a fused instruction. Intel has macro op fusion where two macro ops are send to be decoder as a unit like cmp+jmp to be decoded into uops more efficiently. This isn't that. 2- micro op fusion is when after decoding from macro to micro op, two micro ops that are adjacent are retired together, just as "ADD reg mem" goes to the micro ops "LOAD reg2 mem" and "ADD reg reg2". Or in his example: INC mem -> LOAD reg mem INC reg STORE mem reg and the INC and STORE will be fused and retired together (not the LOAD and INC), IIRC. 3- he's hitting the loop stream detector so decoding is bypassed anyways. His 3 macro op version is hitting the uop cache, same as tbe 1 macro op version. 4- he's trying to say both the "good" and "bad" will execute the same, but i dont think this is impacted by the uop fusion. And nothing in his counters shows any difference. The single direct memory form of INC cuts down on register and L1 instruct cache pressure, neither of which this benchmark will pick up. I'm a software guy though, so corrections are definitely encouraged.
- BeeOnRope 9y ago> 1- using the "INC mem" form is not a fused instruction. Intel has macro op fusion inc mem is a micro-fused instruction. He's not talking about macro-fusion here, which as you point out involves two separate instructions (a flag-setting ALU op and a conditional jump) being fused together into one uop. > and the INC and STORE will be fused and retired together (not the LOAD and INC), IIRC. Actually it's the load and the inc that will be fused together. The store itself internally is 2 uops, which are also fused together. So there are a total of 4 (unufsed) uops implied by inc [mem] and those get fused in pairs for a total of 2 fused uops. Edit: what I said is true for "add [mem], 1" but apparently not for "inc [mem]" which apparently needs 3 fused domain uops, not 2. I'm not sure why it's worse and which "pair" isn't fusing (but if I had to guess it's the load-op pair that fails to fuse). It's kind of dumb since you could just use add [mem], 1 to get the same effect, except in the rare case you wanted the different flag setting behavior of inc.
- jnordwick 9y agoThanks. Is all this learnable from the Intel docs? Or is there a lot of IACA experimentation?
- dendibakh 9y agoHi, Thanks to everybody who commented the post. With your help I was able to close the gap in my knowledge and summarized it in my new post: https://dendibakh.github.io/blog/2018/02/15/MicroFusion-in-Intel-CPUs https://dendibakh.github.io/blog/2018/02/15/MicroFusion-in-I...