6 ms·

The cost of continuing support is increased decoder complexity. The simplicity/orthogonality of the ARM ISA allows simpler instruction decoders. Simpler, faster

The cost of continuing support is increased decoder complexity. The simplicity/orthogonality of the ARM ISA allows simpler instruction decoders. Simpler, faster, and easier to scale. Intel and AMD instruction decoders are impressive beasts that consume considerable manpower, chip space, and physical power.

klelatti 5y ago
I'd always assumed that decoding x86 and x64 had a lot in common so not much to be saved there? Happy to be told otherwise. Agreed that Arm (esp AArch64) must be a lot simpler.
- usefulcat 5y ago
  As I understand it, the fact that x86 has variable-length instructions makes a significant difference. On ARM if you want to look ahead to see what the next instructions are, you can just add a fixed offset to the address of the current instruction. If you want to do that on x86, you have to do at least enough work to figure out what each instruction is, so that you know how large it is, and only then will you know where the next instruction begins. Obviously this is not very amenable to any kind of concurrent processing.
  - Someone 5y ago
    You can do quite a bit concurrently, but at the expense of hardware. You ‘just’ speculatively assume an instruction starts at every byte offset and start decoding. Then, once you figure out that the instruction at offset 0 is N bytes, ignore the speculative decodings starting at offsets 1 through N-1, and tell the speculative decoding at offset N that it is good to go. That means that it in turn can inform its successors whether they are doing needless work or are good to go, etc. That effectively means you need more decoders, and, I guess, have to accept a (¿slightly?) longer delay in decoding instructions that are ‘later’ in this cascade.
    - colejohnson66 5y ago
      IIRC, Intel actually does do that. They do a 16-way decode, each a single byte of offset.
mastax 5y ago
I remember reading (in 2012 maybe) that ISA doesn't really matter and decoders don't really matter for performance. The decoder is such a small part of the power and area compared to the rest of the chip. I had a hypothesis that as we reach the tail end of microarchitectural performance the ISA will start to matter a lot more, since there are fewer and fewer places left to optimize. Well now in 2021 cores are getting wider and wider to have any throughput improvement and power is a bigger concern than ever. Apple M1 has an 8-wide decoder feeding the rest of the extremely wide core. For comparison, Zen 3 has a 4-wide decoder and Ice Lake has 5-wide. We'll see in 3 years if Intel and AMD were just being economical or unimaginative or if they really can't go wider due to the x86 decode complexity. I suppose we'll never really know if they cover for a power hungry decoder with secret sauce elsewhere in the design.
- dragontamer 5y ago
  The state-machine that can determine the start-of-instructions can be run in parallel. In fact, any RegEx / state machine / can be run in parallel on every byte with Kogge-stone / parallel prefix, because the state-machine itself is an associative (but not communitive) operation. As such, I believe that decoders (even complicated ones like x86) scale at O(n) total work (aka power used) and O(log(n)) for depth (aka: clock cycles of latency). ------- Obviously, a simpler ISA would allow for simpler decoding. But I don't think that decoders would scale poorly, even if you had to build a complicated parallel-execution engine to discover the start-of-instruction information.
  - klelatti 5y ago
    Maybe but O(n) on a complex decoder could be a material penalty vs a much simpler design.
    - dragontamer 5y ago
      Sure, but only within a constant factor. My point is that the ARM's ISA vs x86 ISA is not any kind of asymptotic difference in efficiency that grossly prevents scaling. Even the simplest decoder on a exactly 32-bit ISA (like POWER9) with no frills would need O(n) scaling. If you do 16-instructions per clock tick, you need 16x individual decoders on every 4-bytes. Sure, that delivers the results in O(1) instead of O(log(n)) like a Kogge-stone FSM would do, but ARM64 isn't exactly cake to decode either. There's microop / macro-op fusion going on in ARM64 (ex: AESE + AESMC are fused on ARM64 and executed as one uop).
gchadwick 5y ago
> The simplicity/orthogonality of the ARM ISA allows simpler instruction decoders. Simpler, faster, and easier to scale. Intel and AMD instruction decoders are impressive beasts that consume considerable manpower, chip space, and physical power These claims are often made and indeed make some sense but is there any actual evidence for them? To really know you'd need access to the RTL of both cutting edge x86 and arm designs to do the analysis to work out what the decoders are actually costing in terms of power and area and whether they tend to produce critical timing paths. You'd also need access to the companies project planning/timesheets to get an estimate of engineering effort for both (and chances are data isn't really tracked at that level of granularity, you'll also need a deep dive of their bug tracking to determine what is decoder related for instance and estimate how much time has been spent on dealing with decoder issues). I suspect Intel/AMD/arm have no interest in making the relevant information publicly available. You could attempt this analysis without access to RTL but isolating the power cost of the decoder with the silicon alone sounds hard and potentially infeasible.
- hajile 5y ago
  x86 die breakdowns put the area of the decoder as bigger than the integer ALUs. While unused ALUs can power gate, there's almost never a time when the decoders are not in use. Likewise, parsing is a well-studied field and parallel parsing has been a huge focus for decades now. If you look around, you can find papers and patents around decoding highly serialized instruction sets (aka x86). The speedups over a naive implementation are huge, but come at the cost of many transistors while still not being as efficient or scalable as parallel parsing of fixed-length instructions. The insistence that parsing compressed, serial streams can be done for free mystifies me. I believe you can still find where some AMD exec said that they weren't going wider than 4 decoders because the power/performance ratio became much too bad. If decoders weren't a significant cost to their designs (both in transistors and power), you'd expect widening to be a non-issue. EDIT: here's a link to a die breakdown from AMD https://forums.anandtech.com/threads/annotated-hi-res-core-die-shots-of-zen-1-and-2.2577382/ https://forums.anandtech.com/threads/annotated-hi-res-core-d...
  - klelatti 5y ago

klelatti 5y ago

I'd always assumed that decoding x86 and x64 had a lot in common so not much to be saved there? Happy to be told otherwise. Agreed that Arm (esp AArch64) must be a lot simpler.

usefulcat 5y ago
As I understand it, the fact that x86 has variable-length instructions makes a significant difference. On ARM if you want to look ahead to see what the next instructions are, you can just add a fixed offset to the address of the current instruction. If you want to do that on x86, you have to do at least enough work to figure out what each instruction is, so that you know how large it is, and only then will you know where the next instruction begins. Obviously this is not very amenable to any kind of concurrent processing.
- Someone 5y ago
  You can do quite a bit concurrently, but at the expense of hardware. You ‘just’ speculatively assume an instruction starts at every byte offset and start decoding. Then, once you figure out that the instruction at offset 0 is N bytes, ignore the speculative decodings starting at offsets 1 through N-1, and tell the speculative decoding at offset N that it is good to go. That means that it in turn can inform its successors whether they are doing needless work or are good to go, etc. That effectively means you need more decoders, and, I guess, have to accept a (¿slightly?) longer delay in decoding instructions that are ‘later’ in this cascade.
  - colejohnson66 5y ago
    IIRC, Intel actually does do that. They do a 16-way decode, each a single byte of offset.

mastax 5y ago

I remember reading (in 2012 maybe) that ISA doesn't really matter and decoders don't really matter for performance. The decoder is such a small part of the power and area compared to the rest of the chip. I had a hypothesis that as we reach the tail end of microarchitectural performance the ISA will start to matter a lot more, since there are fewer and fewer places left to optimize. Well now in 2021 cores are getting wider and wider to have any throughput improvement and power is a bigger concern than ever. Apple M1 has an 8-wide decoder feeding the rest of the extremely wide core. For comparison, Zen 3 has a 4-wide decoder and Ice Lake has 5-wide. We'll see in 3 years if Intel and AMD were just being economical or unimaginative or if they really can't go wider due to the x86 decode complexity. I suppose we'll never really know if they cover for a power hungry decoder with secret sauce elsewhere in the design.

dragontamer 5y ago
The state-machine that can determine the start-of-instructions can be run in parallel. In fact, any RegEx / state machine / can be run in parallel on every byte with Kogge-stone / parallel prefix, because the state-machine itself is an associative (but not communitive) operation. As such, I believe that decoders (even complicated ones like x86) scale at O(n) total work (aka power used) and O(log(n)) for depth (aka: clock cycles of latency). ------- Obviously, a simpler ISA would allow for simpler decoding. But I don't think that decoders would scale poorly, even if you had to build a complicated parallel-execution engine to discover the start-of-instruction information.
- klelatti 5y ago
  Maybe but O(n) on a complex decoder could be a material penalty vs a much simpler design.
  - dragontamer 5y ago
    Sure, but only within a constant factor. My point is that the ARM's ISA vs x86 ISA is not any kind of asymptotic difference in efficiency that grossly prevents scaling. Even the simplest decoder on a exactly 32-bit ISA (like POWER9) with no frills would need O(n) scaling. If you do 16-instructions per clock tick, you need 16x individual decoders on every 4-bytes. Sure, that delivers the results in O(1) instead of O(log(n)) like a Kogge-stone FSM would do, but ARM64 isn't exactly cake to decode either. There's microop / macro-op fusion going on in ARM64 (ex: AESE + AESMC are fused on ARM64 and executed as one uop).

gchadwick 5y ago

> The simplicity/orthogonality of the ARM ISA allows simpler instruction decoders. Simpler, faster, and easier to scale. Intel and AMD instruction decoders are impressive beasts that consume considerable manpower, chip space, and physical power These claims are often made and indeed make some sense but is there any actual evidence for them? To really know you'd need access to the RTL of both cutting edge x86 and arm designs to do the analysis to work out what the decoders are actually costing in terms of power and area and whether they tend to produce critical timing paths. You'd also need access to the companies project planning/timesheets to get an estimate of engineering effort for both (and chances are data isn't really tracked at that level of granularity, you'll also need a deep dive of their bug tracking to determine what is decoder related for instance and estimate how much time has been spent on dealing with decoder issues). I suspect Intel/AMD/arm have no interest in making the relevant information publicly available. You could attempt this analysis without access to RTL but isolating the power cost of the decoder with the silicon alone sounds hard and potentially infeasible.

hajile 5y ago
x86 die breakdowns put the area of the decoder as bigger than the integer ALUs. While unused ALUs can power gate, there's almost never a time when the decoders are not in use. Likewise, parsing is a well-studied field and parallel parsing has been a huge focus for decades now. If you look around, you can find papers and patents around decoding highly serialized instruction sets (aka x86). The speedups over a naive implementation are huge, but come at the cost of many transistors while still not being as efficient or scalable as parallel parsing of fixed-length instructions. The insistence that parsing compressed, serial streams can be done for free mystifies me. I believe you can still find where some AMD exec said that they weren't going wider than 4 decoders because the power/performance ratio became much too bad. If decoders weren't a significant cost to their designs (both in transistors and power), you'd expect widening to be a non-issue. EDIT: here's a link to a die breakdown from AMD https://forums.anandtech.com/threads/annotated-hi-res-core-die-shots-of-zen-1-and-2.2577382/ https://forums.anandtech.com/threads/annotated-hi-res-core-d...
- klelatti 5y ago