6 ms·

Being ARM has something to do with it. The x86 instruction decoder may be only about ~5% of the die, but it's 5% of the die that has to run all the time. Think

by api 6y ago

Being ARM has something to do with it. The x86 instruction decoder may be only about ~5% of the die, but it's 5% of the die that has to run all the time. Think about how warm your CPU gets when you run e.g. heavy FPU loads and then imagine that's happening all the time. You can see the power difference right there. It's also very hard to achieve more than 4X parallelism (though I think Ice Lake got 6X at some additional cost) in decode, making instruction level parallelism harder. X86's hack to get around this is SMT/hyperthreading to keep the core fed with 2X instruction streams, but that adds a lot more complexity and is a security minefield. Last but not least: ARM's looser default memory model allows for more read/write reordering and a simpler cache. ARM has a distinct simplicity and low-overhead advantage over X86/X64.

mhh__ 6y ago
This is why I said it's ARM's for the taking. I'm not familiar with how ARM's memory model effects the cache design - Source?
NortySpock 6y ago
> x86 instruction decoder may be only about ~5% of the die What percent of the die is an ARM instruction decoder?
- duskwuff 6y ago
  Much less. x86 instruction decoding is complicated by the fact that instructions are variable-width and are byte-aligned (i.e. any instruction can begin at any address). This makes decoding more than one instruction per clock cycle complicated -- I believe the silicon has to try decoding instructions at every possible offset within the decode buffer, then mask out the instructions which are actually inside another instruction. ARM A32/A64 instruction decoding is dramatically simpler -- all instructions are 32 bits wide and word-aligned, so decoding them in parallel is trivial. T32 ("Thumb") is a bit more complex, but still easier than x86.
  - monocasa 6y ago
    I totally agree with the core of your argument (aarch64 decoding is inherently simpler and more power efficient than x86), but I'll throw out there that it's not quite as bad as you say on x86 as there's some nonobvious efficiencies (I've been writing a parallel x86 decoder). What nearly everyone uses is a 16 byte buffer aligned to the program counter being fed into the first stage decode. This first stage, yes has to look at each byte offset as if it could be a new instruction, but doesn't have to do full decode. It only finds instruction length information. From there you feed this length information in and do full decode on the byte offsets that represent actual instruction boundaries. That's how you end up with x86 cores with '4 wide decode' despite needing to initially look at each byte. Now for the efficiencies. Each length decoder for each byte offset isn't symmetric. Only the length decoder at offset 0 in the buffer has to handle everything, and the other length decoders can simply flag "I can't handle this", and the buffer won't be shifted down past where they were on the next cycle and the byte 0 decoder can fix up any goofiness. Because of this, they can * be stripped out of instructions that aren't really used much anymore if that helps them * can be stripped of weird cases like handling crazy usages of prefix bytes * don't have to handle instructions bigger than their portion of the decode buffer. For instance a length decoder starting at byte 12 can't handle more than a 4 byte instruction anyway, so that can simplify it's logic considerably. That means that the simpler length decoders end up feeding into the higher stack up full decoder selection, so some of the overhead cancels out in a nice way. On top of that, I think that 5% includes pieces like the microcode ROMs. Modern ARM cores almost certainly have (albeit much smaller) microcode ROMs as well to handle the more complex state transitions. Once again, totally agreed with your main point, but it's closer than what the general public consensus says.
    - ant6n 6y ago
      I wonder whether a modern byte-sized instruction encoding would sort of look like Unicode, where every byte is self synchronizing... I guess it can be even weaker than that, probably only every second or fourth byte needs to synchronize.
      - monocasa 6y ago
pbsd 6y ago
The x86 decoder is not running all the time; the uops cache and the LSD exist precisely to avoid this. With instructions fed from the decoders you can only sustain 4 instructions per cycle, while to get to 5 or 6 your instructions need to be coming from either the uops cache or the LSD. In the case of the Zen 3, the cache can deliver 8 uops per cycle to the pipeline (but the overall thoughput is limited elsewhere at 6)! Furthermore, the high-performance ARM designs, starting with the Cortex-A77, started using the same trick---the 6-wide execution happens only when instructions are being fed from the decoded macro-op cache.
- mhh__ 6y ago
  The decoder might not be running strictly all the time, but I would wager that for some applications at least it doesn't make much of a difference. For HPC or DSP or whatever where you spend a lot of time in relatively dense loops the uop cache is probably big enough to ease the strain on the decoder, but for sparser code (Compilers come to mind, lots of function calls and memory bound work) I wouldn't be surprised if it didn't make as much difference. I have vTune installed so I guess I could investigate this if I dig out the right PMCs
  - pbsd 6y ago
    I agree; compiler-type code will miss the cache most of the time. A simple test with clang++ compiling some nontrivial piece of C++: 0 lsd_uops 1,092,318,746 idq_dsb_uops ( +- 0.49% ) 4,045,959,682 idq_mite_uops ( +- 0.06% ) The LSD is disabled in this chip (Skylake) due to errata, but we can see only 1/5th of the uops come from the uops cache. However, the more relevant experiment in terms of power is how many cycles is the cache active instead of the decoders: 0 lsd_cycles_active 378,993,057 idq_dsb_cycles ( +- 0.18% ) 1,616,999,501 idq_mite_cycles ( +- 0.07% ) The ratio is similar: the regular decoders are not active only around 1/5th of the time. In comparison, gzipping a 20M file looks a lot better: 0 lsd_cycles_active 2,900,847,992 idq_dsb_cycles ( +- 0.07% ) 407,705,985 idq_mite_cycles ( +- 0.33% )

by api 6y ago

mhh__ 6y ago

This is why I said it's ARM's for the taking. I'm not familiar with how ARM's memory model effects the cache design - Source?

NortySpock 6y ago

> x86 instruction decoder may be only about ~5% of the die What percent of the die is an ARM instruction decoder?

duskwuff 6y ago
Much less. x86 instruction decoding is complicated by the fact that instructions are variable-width and are byte-aligned (i.e. any instruction can begin at any address). This makes decoding more than one instruction per clock cycle complicated -- I believe the silicon has to try decoding instructions at every possible offset within the decode buffer, then mask out the instructions which are actually inside another instruction. ARM A32/A64 instruction decoding is dramatically simpler -- all instructions are 32 bits wide and word-aligned, so decoding them in parallel is trivial. T32 ("Thumb") is a bit more complex, but still easier than x86.
- monocasa 6y ago
  I totally agree with the core of your argument (aarch64 decoding is inherently simpler and more power efficient than x86), but I'll throw out there that it's not quite as bad as you say on x86 as there's some nonobvious efficiencies (I've been writing a parallel x86 decoder). What nearly everyone uses is a 16 byte buffer aligned to the program counter being fed into the first stage decode. This first stage, yes has to look at each byte offset as if it could be a new instruction, but doesn't have to do full decode. It only finds instruction length information. From there you feed this length information in and do full decode on the byte offsets that represent actual instruction boundaries. That's how you end up with x86 cores with '4 wide decode' despite needing to initially look at each byte. Now for the efficiencies. Each length decoder for each byte offset isn't symmetric. Only the length decoder at offset 0 in the buffer has to handle everything, and the other length decoders can simply flag "I can't handle this", and the buffer won't be shifted down past where they were on the next cycle and the byte 0 decoder can fix up any goofiness. Because of this, they can * be stripped out of instructions that aren't really used much anymore if that helps them * can be stripped of weird cases like handling crazy usages of prefix bytes * don't have to handle instructions bigger than their portion of the decode buffer. For instance a length decoder starting at byte 12 can't handle more than a 4 byte instruction anyway, so that can simplify it's logic considerably. That means that the simpler length decoders end up feeding into the higher stack up full decoder selection, so some of the overhead cancels out in a nice way. On top of that, I think that 5% includes pieces like the microcode ROMs. Modern ARM cores almost certainly have (albeit much smaller) microcode ROMs as well to handle the more complex state transitions. Once again, totally agreed with your main point, but it's closer than what the general public consensus says.
  - ant6n 6y ago
    I wonder whether a modern byte-sized instruction encoding would sort of look like Unicode, where every byte is self synchronizing... I guess it can be even weaker than that, probably only every second or fourth byte needs to synchronize.
    - monocasa 6y ago

pbsd 6y ago

The x86 decoder is not running all the time; the uops cache and the LSD exist precisely to avoid this. With instructions fed from the decoders you can only sustain 4 instructions per cycle, while to get to 5 or 6 your instructions need to be coming from either the uops cache or the LSD. In the case of the Zen 3, the cache can deliver 8 uops per cycle to the pipeline (but the overall thoughput is limited elsewhere at 6)! Furthermore, the high-performance ARM designs, starting with the Cortex-A77, started using the same trick---the 6-wide execution happens only when instructions are being fed from the decoded macro-op cache.

mhh__ 6y ago
The decoder might not be running strictly all the time, but I would wager that for some applications at least it doesn't make much of a difference. For HPC or DSP or whatever where you spend a lot of time in relatively dense loops the uop cache is probably big enough to ease the strain on the decoder, but for sparser code (Compilers come to mind, lots of function calls and memory bound work) I wouldn't be surprised if it didn't make as much difference. I have vTune installed so I guess I could investigate this if I dig out the right PMCs
- pbsd 6y ago
  I agree; compiler-type code will miss the cache most of the time. A simple test with clang++ compiling some nontrivial piece of C++: 0 lsd_uops 1,092,318,746 idq_dsb_uops ( +- 0.49% ) 4,045,959,682 idq_mite_uops ( +- 0.06% ) The LSD is disabled in this chip (Skylake) due to errata, but we can see only 1/5th of the uops come from the uops cache. However, the more relevant experiment in terms of power is how many cycles is the cache active instead of the decoders: 0 lsd_cycles_active 378,993,057 idq_dsb_cycles ( +- 0.18% ) 1,616,999,501 idq_mite_cycles ( +- 0.07% ) The ratio is similar: the regular decoders are not active only around 1/5th of the time. In comparison, gzipping a 20M file looks a lot better: 0 lsd_cycles_active 2,900,847,992 idq_dsb_cycles ( +- 0.07% ) 407,705,985 idq_mite_cycles ( +- 0.33% )