5 ms·

I remember reading this Jim Keller interview: https://web.archive.org/web/20210622080634/https://www.anandtech.com/show/16762/an-anandtech-interview-with-jim-k

I remember reading this Jim Keller interview: https://web.archive.org/web/20210622080634/https://www.anandtech.com/show/16762/an-anandtech-interview-with-jim-keller-laziest-person-at-tesla https://web.archive.org/web/20210622080634/https://www.anand... Basically the gist of it is that the difference between ARM/x86 mostly boils down to instruction decode, and: - Most instructions end up being simple load/store/conditional branch etc. on both architectures, where there's literally no difference in encoding efficiency - Variable length instruction has pretty much been figured out on x86 that it's no longer a bottleneck Also my personal addendum is that today's Intel efficiency cores are have more transistors and better perf than the big Intel cores of a decade ago

fanf2 1y ago
Apple’s ARM cores have wider decode than x86 M1 - 8 wide M4 - 10 wide Zen 4 - 4 wide Zen 5 - 8 wide
- wmf 1y ago
  Skymont - 9 wide
- adgjlsfhk1 1y ago
  pure decoder width isn't enough to tell you everything. X86 has some commonly used ridiculously compact instructions (e.g. lea) that would turn into 2-3 instructions on most other architectures.
  - kimixa 1y ago
    Also the op cache - if it hits that the decoder is completely skipped.
  - ajross 1y ago
    The whole ModRM addressing encoding (to which LEA is basically a front end) is actually really compact, and compilers have gotten frightently good at exploiting it. Just look at the disassembly for some non-trivial code sometime and see what it's doing.
  - monocasa 1y ago
    Additionally, stuff llike rmw instructions are really like at least three, maybe four or five risc instructions.
  - ack_complete 1y ago
    Yes, but so does ARM. ld1 {v0.16b,v1.16b,v2.16b,v3.16b},x0,#64 loads 4 x 128-bit vector registers and post-increments a pointer register.
- mort96 1y ago
  Wow, I had no idea we were up to 8 wide decoders in amd64 CPUs.
- ryuuchin 1y ago
  Is Zen 5 more like a 4x2 than a true 8 since it has dual decode clusters and one thread on a core can't use more than one? https://chipsandcheese.com/i/149874010/frontend https://chipsandcheese.com/i/149874010/frontend
AnotherGoodName 1y ago
For variable vs fixed width i have heard that fixed width is part of apple silicons performance. There’s literally gains to be had here for sure imho.
- astrange 1y ago
  It's easier but it's not that important. It's more important for security - you can reinterpret variable length instructions by jumping inside them.
ahartmetz 1y ago
Variable length decoding is more or less figured out, but it takes more design effort, transistors and energy. They cost, but not a lot, relatively, in a current state of the art super wide out-of-order CPU.
- rasz 1y ago
  Not a lot is not how I would describe it. Take a 64bit piece of fetched data. On ARM64 you will just push that into two decoder blocks and be done with it. On x86 you got what, 1 to 15 bytes range per instruction? I dont even want to think about possible permutations, its in the 10 ^ some two digit number order.
  - ahartmetz 1y ago
    It's a lot for a decoder, but not for a whole core. Citation needed, but I remember that the decoder is about 10% of a Ryzen core's power budget, and of course that is with a few techniques better than complete brute force.
  - mohinder 1y ago
    You don't need all the permutations. If there are 32 bytes in a cache line then each instruction can only start at one of 32 possible positions. Then if you want to decode N instructions per cycle you need N 32-to-1 muxes. You can reduce the number of inputs to the later muxes since instructions can't be zero size.
    - monocasa 1y ago
      It was even simpler until very recently where the decode stage would only look at a max 16 byte floating window.
  - saagarjha 1y ago
    Yes, but you're not describing it from the right position. Is instruction decode hard? Yes, if you think about it in isolation (also, fwiw, it's not a permutation problem as you suggest). But the core has a bunch of other stuff it needs to do that is far harder. Even your lowliest Pentium from 2000 can do instruction decode.
codedokode 1y ago
x86 decoding must be a pain - I vaguely remember that they have trace caches (a cache of decoded micro-operations) to skip decoding in some cases. You probably don't make such caches when decoding is easy. Also, more complicated decoding and extra caches means longer pipeline, which means more price to pay when a branch is mispredicted (binary search is a festival of branch misprediction for example, and I got 3x acceleration of linear search on small arrays when I switched to the branchless algorithm). Also I am not a CPU designer, but branch prediction with wide decoder also must be a pain - imagine that while you are loading 16 or 32 bytes from instruction cache, you need to predict the address of next loaded chunk in the same cycle, before you even see what you got from cache. As for encoding efficiency, I played with little algorithms (like binary search or slab allocator) on godbolt, and RISC-V with compressed instruction generates similar amount of code as x86 - in rare cases, even slightly smaller. So x86 has a complex decoding that doesn't give any noticeable advantages. x86 also has flags, which add implicit dependencies between instructions, and must make designer's life harder.
- phire 1y ago
  Intel’s E cores decode x86 without a trace cache (μop cache), and are very efficient. The latest (Skymont) can decode 9 x86 instructions per cycle, more than the P core (which can only decode 8) AMD isn’t saying that decoding x86 is easy. They are just saying that decoding x86 doesn’t have a notable power impact.
  - varispeed 1y ago
    Does that really say anything about efficiency? Why can't they decode 100 instructions per cycle?
    - ajross 1y ago
      > Why can't they decode 100 instructions per cycle? Well, obviously because there aren't 100 individual parallel execution units to which those instructions could be issued. And lower down the stack because a 3000 bit[1] wide cache would be extremely difficult to manage. An instruction fetch would be six (!) cache lines wide, causing clear latency and bottleneck problems (or conversely would demand your icache be 6x wider, causing locality/granularity problems as many leaf functions are smaller than that). But also because real world code just isn't that parallel. Even assuming perfect branch prediction the number of instructions between unpredictable things like function pointer calls or computed jumps is much less than 100 in most performance-sensitive algorithms. And even if you could, the circuit complexity of decoding variable length instructions is superlinear. In x86, every byte can be an instruction boundary, but most aren't, and your decoder needs to be able to handle that. [1] I have in my head somewhere that "the average x86_64 instruction is 3.75 bytes long", but that may be off by a bit. Somewhere around that range, anyway.
jorvi 1y ago
Nice followup to your link: https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter. Personally I do not entirely buy it. Intel and AMD have had plenty of years to catch up to Apple's M-architecture and they still aren't able to touch it in efficiency. The PC Snapdragon chips AFAIK also offer better performance-per-watt than AMD or Intel, with laptops offering them often having 10-30% longer battery life at similar performance. The same goes for GPUs, where Apple's M1 GPU completely smoked an RTX3090 in performance-per-watt, offering 320W of RTX 3090 performance in a 110W envelope: https://images.macrumors.com/t/xuN87vnxzdp_FJWcAwqFhl4IOXs=/1600x0/article-new/2022/03/m1-ultra-performance-chart.jpg https://images.macrumors.com/t/xuN87vnxzdp_FJWcAwqFhl4IOXs=/...
- ohdeargodno 1y ago
  [dead]
- diddid 1y ago
  I mean the M1 is nice but pretending that it can do in 110w what the 3090 does with 320w is Apple marketing nonsense. Like if your use case is playing games like cp2077, the 3090 will do 100fps in ultra ray tracing and an M4 Max will only do 30fps. Not to mention it’s trivial to undervolt nvidia cards and get 100% performance at 80% power. So 1/3 the power for 1/3 the performance? How is that smoking anything?
  - deleted 1y ago
    [deleted]
  - whatevaa 1y ago
    Apple fans drinking apple juice, nothing new with fans, sadly.
    - pjmlp 1y ago
      Indeed, like talking as if Apple mattered at all in server space, or digital workstations (studio is not a replacement for people willing to buy Mac Pros, which still keep being built with Intel Xeons).
topspin 1y ago
I've listened Keller's views on CPU design and the biggest takeaway I found is that performance is overwhelmingly dominated by predictors. Good predictors mitigate memory latency and keep pipelines full. Bad predictors stall everything while cores spin on useless cache lines. The rest, including ISA minutiae, rank well below predictors on the list of things that matter. At one time, ISA had a significant impact on predictors: variable length instructions complicated predictor design. The consensus is that this is no longer the case: decoders have grown to overcome this and now the difference is negligible.
IshKebab 1y ago
Yeah I'm not sure I buy it either. It doesn't matter if most instructions have simple encodings. You still need to design your front end to handle the crazy encodings. I doubt it makes a big difference, so until recently he would have been correct - why change your ISA when you can just wait a couple of months to get the same performance improvement. But Moore's law is dead now so small performance differences matter way more now.
- imtringued 1y ago
  The core argument in RISC vs CISC has never been that you can't add RISC style instructions to a CISC. If anything, the opposite is true, because CISC architectures just keep adding more and more instructions. The argument has been that even if you have a CISC ISA that also happens to have a subset of instructions following the RISC philosophy, that the bloat and legacy instructions will hold CISC back. In other words, the weakness of CISC is that you can add, but never remove. Jim Keller disagrees with this assessment and it is blatantly obvious. You build a decoder that predicts that the instructions are going to have simple encodings and if they don't, then you have a slow fallback. Now you might say that this makes the code slow under the assumption that you make heavy use of the complex instructions, but compilers have a strong incentive to only emit fast instructions. If you can just add RISC style instructions to CISC ISAs, the entire argument collapses into irrelevance.
  - IshKebab 1y ago
    It's not just the complex encodings though, there's also the variable instruction length, and the instruction semantics that mean you need microcode. Obviously they've done an amazing job of working around it, but that adds a ton of complexity. At the very least it's going to mean you spend engineering resources on something that ARM & RISC-V don't even have to worry about. This seems a little like a Java programmer saying "garbage collection is solved". Like, yeah you've made an amazingly complicated concurrent compacting garbage collector that is really fast and certainly fast enough almost all of the time. But it's still not as fast as not doing garbage collection. If you didn't have the "we really want people to use x86 because my job depends on it" factor then why would you use CISC?
devnullbrain 1y ago
> fixed-length instructions seem really nice when you're building little baby computers, but if you're building a really big computer, to predict or to figure out where all the instructions are, it isn't dominating the die. So it doesn't matter that much. The notebooks of TFA aren't really big computers.
- adgjlsfhk1 1y ago
  they are. By "small" here, we're referring to the Core size, not the machine size. i.e. an in order, dual issue cpu is small, but a M1 chip is massive.

fanf2 1y ago

Apple’s ARM cores have wider decode than x86 M1 - 8 wide M4 - 10 wide Zen 4 - 4 wide Zen 5 - 8 wide

wmf 1y ago
Skymont - 9 wide
adgjlsfhk1 1y ago
pure decoder width isn't enough to tell you everything. X86 has some commonly used ridiculously compact instructions (e.g. lea) that would turn into 2-3 instructions on most other architectures.
- kimixa 1y ago
  Also the op cache - if it hits that the decoder is completely skipped.
- ajross 1y ago
  The whole ModRM addressing encoding (to which LEA is basically a front end) is actually really compact, and compilers have gotten frightently good at exploiting it. Just look at the disassembly for some non-trivial code sometime and see what it's doing.
- monocasa 1y ago
  Additionally, stuff llike rmw instructions are really like at least three, maybe four or five risc instructions.
- ack_complete 1y ago
  Yes, but so does ARM. ld1 {v0.16b,v1.16b,v2.16b,v3.16b},x0,#64 loads 4 x 128-bit vector registers and post-increments a pointer register.
mort96 1y ago
Wow, I had no idea we were up to 8 wide decoders in amd64 CPUs.
ryuuchin 1y ago
Is Zen 5 more like a 4x2 than a true 8 since it has dual decode clusters and one thread on a core can't use more than one? https://chipsandcheese.com/i/149874010/frontend https://chipsandcheese.com/i/149874010/frontend

AnotherGoodName 1y ago

For variable vs fixed width i have heard that fixed width is part of apple silicons performance. There’s literally gains to be had here for sure imho.

astrange 1y ago
It's easier but it's not that important. It's more important for security - you can reinterpret variable length instructions by jumping inside them.

ahartmetz 1y ago

Variable length decoding is more or less figured out, but it takes more design effort, transistors and energy. They cost, but not a lot, relatively, in a current state of the art super wide out-of-order CPU.

rasz 1y ago
Not a lot is not how I would describe it. Take a 64bit piece of fetched data. On ARM64 you will just push that into two decoder blocks and be done with it. On x86 you got what, 1 to 15 bytes range per instruction? I dont even want to think about possible permutations, its in the 10 ^ some two digit number order.
- ahartmetz 1y ago
  It's a lot for a decoder, but not for a whole core. Citation needed, but I remember that the decoder is about 10% of a Ryzen core's power budget, and of course that is with a few techniques better than complete brute force.
- mohinder 1y ago
  You don't need all the permutations. If there are 32 bytes in a cache line then each instruction can only start at one of 32 possible positions. Then if you want to decode N instructions per cycle you need N 32-to-1 muxes. You can reduce the number of inputs to the later muxes since instructions can't be zero size.
  - monocasa 1y ago
    It was even simpler until very recently where the decode stage would only look at a max 16 byte floating window.
- saagarjha 1y ago
  Yes, but you're not describing it from the right position. Is instruction decode hard? Yes, if you think about it in isolation (also, fwiw, it's not a permutation problem as you suggest). But the core has a bunch of other stuff it needs to do that is far harder. Even your lowliest Pentium from 2000 can do instruction decode.

codedokode 1y ago

x86 decoding must be a pain - I vaguely remember that they have trace caches (a cache of decoded micro-operations) to skip decoding in some cases. You probably don't make such caches when decoding is easy. Also, more complicated decoding and extra caches means longer pipeline, which means more price to pay when a branch is mispredicted (binary search is a festival of branch misprediction for example, and I got 3x acceleration of linear search on small arrays when I switched to the branchless algorithm). Also I am not a CPU designer, but branch prediction with wide decoder also must be a pain - imagine that while you are loading 16 or 32 bytes from instruction cache, you need to predict the address of next loaded chunk in the same cycle, before you even see what you got from cache. As for encoding efficiency, I played with little algorithms (like binary search or slab allocator) on godbolt, and RISC-V with compressed instruction generates similar amount of code as x86 - in rare cases, even slightly smaller. So x86 has a complex decoding that doesn't give any noticeable advantages. x86 also has flags, which add implicit dependencies between instructions, and must make designer's life harder.

phire 1y ago
Intel’s E cores decode x86 without a trace cache (μop cache), and are very efficient. The latest (Skymont) can decode 9 x86 instructions per cycle, more than the P core (which can only decode 8) AMD isn’t saying that decoding x86 is easy. They are just saying that decoding x86 doesn’t have a notable power impact.
- varispeed 1y ago
  Does that really say anything about efficiency? Why can't they decode 100 instructions per cycle?
  - ajross 1y ago
    > Why can't they decode 100 instructions per cycle? Well, obviously because there aren't 100 individual parallel execution units to which those instructions could be issued. And lower down the stack because a 3000 bit[1] wide cache would be extremely difficult to manage. An instruction fetch would be six (!) cache lines wide, causing clear latency and bottleneck problems (or conversely would demand your icache be 6x wider, causing locality/granularity problems as many leaf functions are smaller than that). But also because real world code just isn't that parallel. Even assuming perfect branch prediction the number of instructions between unpredictable things like function pointer calls or computed jumps is much less than 100 in most performance-sensitive algorithms. And even if you could, the circuit complexity of decoding variable length instructions is superlinear. In x86, every byte can be an instruction boundary, but most aren't, and your decoder needs to be able to handle that. [1] I have in my head somewhere that "the average x86_64 instruction is 3.75 bytes long", but that may be off by a bit. Somewhere around that range, anyway.

jorvi 1y ago

Nice followup to your link: https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter https://chipsandcheese.com/p/arm-or-x86-isa-doesnt-matter. Personally I do not entirely buy it. Intel and AMD have had plenty of years to catch up to Apple's M-architecture and they still aren't able to touch it in efficiency. The PC Snapdragon chips AFAIK also offer better performance-per-watt than AMD or Intel, with laptops offering them often having 10-30% longer battery life at similar performance. The same goes for GPUs, where Apple's M1 GPU completely smoked an RTX3090 in performance-per-watt, offering 320W of RTX 3090 performance in a 110W envelope: https://images.macrumors.com/t/xuN87vnxzdp_FJWcAwqFhl4IOXs=/1600x0/article-new/2022/03/m1-ultra-performance-chart.jpg https://images.macrumors.com/t/xuN87vnxzdp_FJWcAwqFhl4IOXs=/...

ohdeargodno 1y ago
[dead]
diddid 1y ago
I mean the M1 is nice but pretending that it can do in 110w what the 3090 does with 320w is Apple marketing nonsense. Like if your use case is playing games like cp2077, the 3090 will do 100fps in ultra ray tracing and an M4 Max will only do 30fps. Not to mention it’s trivial to undervolt nvidia cards and get 100% performance at 80% power. So 1/3 the power for 1/3 the performance? How is that smoking anything?
- deleted 1y ago
  [deleted]
- whatevaa 1y ago
  Apple fans drinking apple juice, nothing new with fans, sadly.
  - pjmlp 1y ago
    Indeed, like talking as if Apple mattered at all in server space, or digital workstations (studio is not a replacement for people willing to buy Mac Pros, which still keep being built with Intel Xeons).

topspin 1y ago

I've listened Keller's views on CPU design and the biggest takeaway I found is that performance is overwhelmingly dominated by predictors. Good predictors mitigate memory latency and keep pipelines full. Bad predictors stall everything while cores spin on useless cache lines. The rest, including ISA minutiae, rank well below predictors on the list of things that matter. At one time, ISA had a significant impact on predictors: variable length instructions complicated predictor design. The consensus is that this is no longer the case: decoders have grown to overcome this and now the difference is negligible.

IshKebab 1y ago

Yeah I'm not sure I buy it either. It doesn't matter if most instructions have simple encodings. You still need to design your front end to handle the crazy encodings. I doubt it makes a big difference, so until recently he would have been correct - why change your ISA when you can just wait a couple of months to get the same performance improvement. But Moore's law is dead now so small performance differences matter way more now.

imtringued 1y ago
The core argument in RISC vs CISC has never been that you can't add RISC style instructions to a CISC. If anything, the opposite is true, because CISC architectures just keep adding more and more instructions. The argument has been that even if you have a CISC ISA that also happens to have a subset of instructions following the RISC philosophy, that the bloat and legacy instructions will hold CISC back. In other words, the weakness of CISC is that you can add, but never remove. Jim Keller disagrees with this assessment and it is blatantly obvious. You build a decoder that predicts that the instructions are going to have simple encodings and if they don't, then you have a slow fallback. Now you might say that this makes the code slow under the assumption that you make heavy use of the complex instructions, but compilers have a strong incentive to only emit fast instructions. If you can just add RISC style instructions to CISC ISAs, the entire argument collapses into irrelevance.
- IshKebab 1y ago
  It's not just the complex encodings though, there's also the variable instruction length, and the instruction semantics that mean you need microcode. Obviously they've done an amazing job of working around it, but that adds a ton of complexity. At the very least it's going to mean you spend engineering resources on something that ARM & RISC-V don't even have to worry about. This seems a little like a Java programmer saying "garbage collection is solved". Like, yeah you've made an amazingly complicated concurrent compacting garbage collector that is really fast and certainly fast enough almost all of the time. But it's still not as fast as not doing garbage collection. If you didn't have the "we really want people to use x86 because my job depends on it" factor then why would you use CISC?

devnullbrain 1y ago

> fixed-length instructions seem really nice when you're building little baby computers, but if you're building a really big computer, to predict or to figure out where all the instructions are, it isn't dominating the die. So it doesn't matter that much. The notebooks of TFA aren't really big computers.

adgjlsfhk1 1y ago
they are. By "small" here, we're referring to the Core size, not the machine size. i.e. an in order, dual issue cpu is small, but a M1 chip is massive.