3 ms·

For example: Decode. One of the things Apple CPUs keep being praised for is how wide they are. To be wide effectively and keep your execution units busy, you ne

For example: Decode. One of the things Apple CPUs keep being praised for is how wide they are. To be wide effectively and keep your execution units busy, you need a wide decoder. With aarch64 it is easy. Every instruction is 4 bytes on a 4 byte boundary. Want a 4-wide decoder? Ingest 16 bytes and copy-and-paste the same decoder 4 times. X86 instructions are variable-length: 1-15 bytes. And you do not know how long any instruction is until you read almost every byte of it (there is no length byte up front - it is just a list of prefixes). Thus to make a 4-wide decoder you need to ingest up to 60 bytes, and your decoder cannot start on instruction #2 till it finds the end of # 1. (Or guess, and rewind as needed). Much harder and a rather long dependency chain for decoder #4 to even get started. Now say you want to go wider: 8-wide. Arm64: copy and paste same decoders again. Done X86: Now you need to ingest 120 bytes, and find even more boundaries, each depending on the previous - an even longer dependency chain. Decoder #8 cannot start till #1 - #7 are almost done. A mess nobody has managed to solve yet.

abc_lisper 4y ago
Excellent answer. I learnt something today!
gary_0 4y ago
And those bytes matter when most Intel CPU cores have only a few dozen KB of L1I cache. Interestingly, the M1 and M2 performance cores have a whopping 192KB of L1 instruction cache.
- kaba0 4y ago
  Isn’t it in part due to business reasons, namely that Apple is not a CPU vendor, it doesn’t have to turn a profit on chips so it can allow much larger die sizes and thus larger cache sizes as well. Is there something inherent in ARM here I’m missing?
- pclmulqdq 4y ago
  ARM instructions aren't as compact as their x86 counterparts, so they need a big cache to get similar performance on cache-bound applications.
  - gary_0 4y ago
    AFAIK, AArch64 instructions are roughly 2x longer than x64's average, but that's 3x as much cache as a comparable Intel core.
  - hajile 4y ago
    ARM aarch64 instructions are around 12-15% bigger overall, but cache sizes are much larger than that. Most likely that is so they can power the massive reorder buffer and deal with having code for tons of branches handy.
jcranmer 4y ago
That would be a dumb way to build a decoder. Which is why that's not how it works on x86 processors. What an x86 processor does is fetch an entire cacheline of instructions at once. Working out which byte corresponds to instructions is a straightforward state machine--and state machines can be parallelized as reduction tree, because function application is associative! (This is usually described as "x86 processors start decoding, in parallel, every byte as if it were the start of a new instruction.) My guess for x86 is that resolving instruction length takes a single cycle, a single stage of the instruction decode pipeline. It's actually hard to come up with x86 instructions that are 15 bytes long. Your typical general-purpose instruction is going to max out around 4 bytes (REX + other prefix + opcode + ModR/M byte), and even that's a little long. A "complex" memory instruction that moves a 4-byte constant into a register + constant offset will manage 12 bytes, but that's really the limit of practical instruction. So there's no point building a decoder capable of ingesting that many extra-long instructions a cycle because it is so synthetic a use case. Sure, you could design a better compressed ISA for decoding than x86, but the impact of its complexity on processors is vastly oversold. Just look at how much die space the x86 decoders actually take up to see how little of an impact it has.
- dmitrygr 4y ago
  > fetch an entire cacheline of instructions at once And how many instructions fit into it? Are you going to feed an 8-wide decoder with a single 64B cacheline? Especially with no promises that any instruction starts or ends on that boundary? > It's actually hard to come up with x86 instructions that are 15 bytes long. Something as simple and trivial as loading a 64-bit immediate into a register is a 10-byte instruction in x86... So for you to feed an 8-wide decoder even with such simple things as that, you're already at TWO cache lines ingested per cycle (80 bytes)... good luck with that... I hear intel is hiring. Their current top of the line decoder is IIRC 5-wide. I am sure they’d like to hear your ideas as to how easy it is to decode x86 8-way
  - jcranmer 4y ago
    > Something as simple and trivial as loading a 64-bit immediate into a register is a 10-byte instruction in x86... It will only take a 10 byte instruction if you can't zero- or sign-extend from a 32-bit immediate. Of course, if you were on ARM or most RISC processors, such an immediate likely requires at least two instructions anyways. From paging through an objdump output of a large program, I'd hazard that average instruction size is about 4 bytes, maybe a touch shy of that. After all, instructions like "add rax, rbx" is going to be 3 bytes (two if you're only using 32-bit registers!). > Their current top of the line decoder is IIRC 5-wide. Looking at wikichip for the block diagram [1], that's 5 instructions per 16-byte window. [1] https://en.wikichip.org/wiki/intel/microarchitectures/skylake_(client) https://en.wikichip.org/wiki/intel/microarchitectures/skylak...

abc_lisper 4y ago

Excellent answer. I learnt something today!

gary_0 4y ago

And those bytes matter when most Intel CPU cores have only a few dozen KB of L1I cache. Interestingly, the M1 and M2 performance cores have a whopping 192KB of L1 instruction cache.

kaba0 4y ago
Isn’t it in part due to business reasons, namely that Apple is not a CPU vendor, it doesn’t have to turn a profit on chips so it can allow much larger die sizes and thus larger cache sizes as well. Is there something inherent in ARM here I’m missing?
pclmulqdq 4y ago
ARM instructions aren't as compact as their x86 counterparts, so they need a big cache to get similar performance on cache-bound applications.
- gary_0 4y ago
  AFAIK, AArch64 instructions are roughly 2x longer than x64's average, but that's 3x as much cache as a comparable Intel core.
- hajile 4y ago
  ARM aarch64 instructions are around 12-15% bigger overall, but cache sizes are much larger than that. Most likely that is so they can power the massive reorder buffer and deal with having code for tons of branches handy.

jcranmer 4y ago

That would be a dumb way to build a decoder. Which is why that's not how it works on x86 processors. What an x86 processor does is fetch an entire cacheline of instructions at once. Working out which byte corresponds to instructions is a straightforward state machine--and state machines can be parallelized as reduction tree, because function application is associative! (This is usually described as "x86 processors start decoding, in parallel, every byte as if it were the start of a new instruction.) My guess for x86 is that resolving instruction length takes a single cycle, a single stage of the instruction decode pipeline. It's actually hard to come up with x86 instructions that are 15 bytes long. Your typical general-purpose instruction is going to max out around 4 bytes (REX + other prefix + opcode + ModR/M byte), and even that's a little long. A "complex" memory instruction that moves a 4-byte constant into a register + constant offset will manage 12 bytes, but that's really the limit of practical instruction. So there's no point building a decoder capable of ingesting that many extra-long instructions a cycle because it is so synthetic a use case. Sure, you could design a better compressed ISA for decoding than x86, but the impact of its complexity on processors is vastly oversold. Just look at how much die space the x86 decoders actually take up to see how little of an impact it has.

dmitrygr 4y ago
> fetch an entire cacheline of instructions at once And how many instructions fit into it? Are you going to feed an 8-wide decoder with a single 64B cacheline? Especially with no promises that any instruction starts or ends on that boundary? > It's actually hard to come up with x86 instructions that are 15 bytes long. Something as simple and trivial as loading a 64-bit immediate into a register is a 10-byte instruction in x86... So for you to feed an 8-wide decoder even with such simple things as that, you're already at TWO cache lines ingested per cycle (80 bytes)... good luck with that... I hear intel is hiring. Their current top of the line decoder is IIRC 5-wide. I am sure they’d like to hear your ideas as to how easy it is to decode x86 8-way
- jcranmer 4y ago
  > Something as simple and trivial as loading a 64-bit immediate into a register is a 10-byte instruction in x86... It will only take a 10 byte instruction if you can't zero- or sign-extend from a 32-bit immediate. Of course, if you were on ARM or most RISC processors, such an immediate likely requires at least two instructions anyways. From paging through an objdump output of a large program, I'd hazard that average instruction size is about 4 bytes, maybe a touch shy of that. After all, instructions like "add rax, rbx" is going to be 3 bytes (two if you're only using 32-bit registers!). > Their current top of the line decoder is IIRC 5-wide. Looking at wikichip for the block diagram [1], that's 5 instructions per 16-byte window. [1] https://en.wikichip.org/wiki/intel/microarchitectures/skylake_(client) https://en.wikichip.org/wiki/intel/microarchitectures/skylak...