3 ms·

> IMO, Intel (and AMD) did prove the impact of a legacy ISA was low enough to not be a competitive disadvantage. Not zero, but close enough for high-performance

> IMO, Intel (and AMD) did prove the impact of a legacy ISA was low enough to not be a competitive disadvantage. Not zero, but close enough for high-performance designs. And Apple proved that in fact it was a significant problem once you factored into account performance per watt allowing them to completely spank AMD and Intel once those hit a thermal limit. There’s a benefit from being able to decode and dispatch multiple instructions in parallel vs having to emulate that through heuristically guessing at instruction boundaries and backtrack when you make a mistake (among other things).

adgjlsfhk1 1y ago
I don't think Apple's battery life wins are primarily from isa. I think it's largely from better target and process optimization/ecosystem control. Intel (and somewhat AMD) make most of their money in servers where what matters is performance/watt in a 100% loaded system. they also are designing for jdec RAM and pcie connectivity (and lots of other industry standards). most of Apple's efficiency advantage comes at the edges, lowering max clock speed, integrating the ram to save power, using custom SSDs where the controller is on the CPU etc.
phire 1y ago
> having to emulate that through heuristically guessing at instruction boundaries and backtrack when you make a mistake Intel/AMD don't use heuristics-based decoding, or backtracking. They can decode 4 instructions in a single cycle. They implement this by starting a pre-decode at every single byte offset (within 16 bytes) and then resolving it to actual instructions at the end of the cycle. The actual decode is then done the following cycle, but the pre-decoder has already moved upto 4 instructions forwards, so the whole pipelined decoder can maintain 4 instructions per cycle on some code. This pre-decode approach does have limits. Due to propagation delays, 4 instructions over 16 bytes is probably the realistic limit that you can push it (while Apple can easily do 8 instructions over 32 bytes). Intel's Golden Cove did finally push it to 6 instructions over 32 bytes, but I'm not sure that's worth it. Intel's Skymont shows the way forwards. It only uses 3-wide decoders, but it has three of them running in parallel, leapfrogging over each other. They use the branch predictor to start each decoder running at a future instruction boundaries (inserting dummy branches to break up large branchless blocks). Skymont can maintain 9 instructions per cycle, which is more than the 8-wide that Apple currently is using. And unlike the previous "parallel pre-decode in a single cycle", this approach is scalable. Nothing stopping Intel adding a fourth decoder for 12 instructions per cycle, or a fifth decoder for 15. AMD is showing signs of going down the same path, zen5 has two 4-wide decoders though they can't work on the same thread, yet.
- vlovich123 1y ago
  You're arguing semantics IMHO. In my mind speculatively decoding every single byte offset & then resolving at the end of the cycle which to take is a form of heuristic execution because the heuristic is "decode all possible executions". And 4 vs 8 is a pretty sizeable difference. Moreover, the pre-decoder requiring knowing the op-codes at the end means all instruction decodes are serialized on decoding 16 instructions whereas Apple can just decode each op independently & only decodes 8.
  - phire 1y ago
    Oh, I see what you are saying. I don't consider it to be a heuristic because it's simply bruteforcing it. IMO a heuristic needs improve over brute force. > And 4 vs 8 is a pretty sizeable difference. True, but x86 was doing four instruction 20 years ago. As I mentioned the current state of the art (in a shipping product) is 9, and 9 is larger than 8. Importantly, this Skymont of leapfrogging decoders approach is scalable. > whereas Apple can just decode each op independently & only decodes 8. Apple isn't as free from serialisation as you suggest. Like X86, many instructions decode to multiple uops. According to research [1] instructions which decode to two uops are common and a few decode to as many as 12 uops. It also does instruction fusion, two neighbouring instructions can sometimes decode into a single uop. This all means that there is plenty of serialisation within Apple's decoder. And branching also creates serialisation. It's just not as simple as independently decoding eight instructions into eight uops every cycle. Simpler than what x86 implementations need to do, but not as brain-dead simple as you suggest. Actually, Skymont's approach has an advantage over Apple here, because it only needs to serialise within each 3-wide decoder. [1] https://dougallj.github.io/applecpu/firestorm.html https://dougallj.github.io/applecpu/firestorm.html
ahartmetz 1y ago
Reportedly, the decoder is like 10-20% of the power budget of Ryzen CPUs, so that rather contradicts that the ISA is the main issue that makes Ryzen's efficiency worse than Apple's A and M cores.
- phire 1y ago
  I looked around and couldn't find anything about Ryzen decoding power conniption. I'm only aware of one report for x86 decode power consumption, [1] and there are quite a few problems with trying to use it to justify that conclusion. First, it's covers Intel Haswell, which is "not ryzen", it's not even AMD. Plus, Haswell is 12 years old at this point, how much relevance does it even have to modern Intel CPUs? Second, the "instruction decoders" power zone was only 10%, not 20%. And still reported 3% on workloads that used very few instructions and always hit the uop cache. So really we are talking about 7% overhead for decoding instructions. They do speculate that other workloads use more power (they only tested two workloads), as the theoretical instruction throughput might be double. (which is where I suspect you got the 20% from), but they provide no evidence for that, and double the throughput doesn't mean double the power consumption. And double 7% + 3% base would be 17% at most. Third. Intel doesn't publish any details about what this "instruction decoder" zone actually covers. It's almost certainly more just the "decoding x86" part. Given there are only four zones, I'm almost certain this zone covers the entire frontend, which includes branch prediction, instruction fetch, the stack engine. It might include register renaming too. Maybe instruction TLB lookups? I am reasonably sure it includes the (dynamic) power cost of accessing the L1i cache too. So this 7% power usage is way more than just the decoding of decoding 86%. It's the entire frontend. Finally. I haven't seen any power numbers for the front end of an equivalent ARM processor (like Apple's M1). For all we know, they are also using 7% of their power budget to fetch ARM instructions from the L1 cache, decode them, do branch prediction, do all the fancy front-end stuff. The 7% number isn't x86 overhead as many people imply, it's just the cost of running Haswell's frontend. Without anything else to compare to, this 7% number is worthless. It's certainly an interesting paper, I don't have any major criticisms, but it simply cannot be used to support (or disprove) any arguments about the overhead of x86 decoding. [1] https://www.usenix.org/system/files/conference/cooldc16/cooldc16-paper-hirki.pdf https://www.usenix.org/system/files/conference/cooldc16/cool...

adgjlsfhk1 1y ago

I don't think Apple's battery life wins are primarily from isa. I think it's largely from better target and process optimization/ecosystem control. Intel (and somewhat AMD) make most of their money in servers where what matters is performance/watt in a 100% loaded system. they also are designing for jdec RAM and pcie connectivity (and lots of other industry standards). most of Apple's efficiency advantage comes at the edges, lowering max clock speed, integrating the ram to save power, using custom SSDs where the controller is on the CPU etc.

phire 1y ago

> having to emulate that through heuristically guessing at instruction boundaries and backtrack when you make a mistake Intel/AMD don't use heuristics-based decoding, or backtracking. They can decode 4 instructions in a single cycle. They implement this by starting a pre-decode at every single byte offset (within 16 bytes) and then resolving it to actual instructions at the end of the cycle. The actual decode is then done the following cycle, but the pre-decoder has already moved upto 4 instructions forwards, so the whole pipelined decoder can maintain 4 instructions per cycle on some code. This pre-decode approach does have limits. Due to propagation delays, 4 instructions over 16 bytes is probably the realistic limit that you can push it (while Apple can easily do 8 instructions over 32 bytes). Intel's Golden Cove did finally push it to 6 instructions over 32 bytes, but I'm not sure that's worth it. Intel's Skymont shows the way forwards. It only uses 3-wide decoders, but it has three of them running in parallel, leapfrogging over each other. They use the branch predictor to start each decoder running at a future instruction boundaries (inserting dummy branches to break up large branchless blocks). Skymont can maintain 9 instructions per cycle, which is more than the 8-wide that Apple currently is using. And unlike the previous "parallel pre-decode in a single cycle", this approach is scalable. Nothing stopping Intel adding a fourth decoder for 12 instructions per cycle, or a fifth decoder for 15. AMD is showing signs of going down the same path, zen5 has two 4-wide decoders though they can't work on the same thread, yet.

vlovich123 1y ago
You're arguing semantics IMHO. In my mind speculatively decoding every single byte offset & then resolving at the end of the cycle which to take is a form of heuristic execution because the heuristic is "decode all possible executions". And 4 vs 8 is a pretty sizeable difference. Moreover, the pre-decoder requiring knowing the op-codes at the end means all instruction decodes are serialized on decoding 16 instructions whereas Apple can just decode each op independently & only decodes 8.
- phire 1y ago
  Oh, I see what you are saying. I don't consider it to be a heuristic because it's simply bruteforcing it. IMO a heuristic needs improve over brute force. > And 4 vs 8 is a pretty sizeable difference. True, but x86 was doing four instruction 20 years ago. As I mentioned the current state of the art (in a shipping product) is 9, and 9 is larger than 8. Importantly, this Skymont of leapfrogging decoders approach is scalable. > whereas Apple can just decode each op independently & only decodes 8. Apple isn't as free from serialisation as you suggest. Like X86, many instructions decode to multiple uops. According to research [1] instructions which decode to two uops are common and a few decode to as many as 12 uops. It also does instruction fusion, two neighbouring instructions can sometimes decode into a single uop. This all means that there is plenty of serialisation within Apple's decoder. And branching also creates serialisation. It's just not as simple as independently decoding eight instructions into eight uops every cycle. Simpler than what x86 implementations need to do, but not as brain-dead simple as you suggest. Actually, Skymont's approach has an advantage over Apple here, because it only needs to serialise within each 3-wide decoder. [1] https://dougallj.github.io/applecpu/firestorm.html https://dougallj.github.io/applecpu/firestorm.html

ahartmetz 1y ago

Reportedly, the decoder is like 10-20% of the power budget of Ryzen CPUs, so that rather contradicts that the ISA is the main issue that makes Ryzen's efficiency worse than Apple's A and M cores.

phire 1y ago
I looked around and couldn't find anything about Ryzen decoding power conniption. I'm only aware of one report for x86 decode power consumption, [1] and there are quite a few problems with trying to use it to justify that conclusion. First, it's covers Intel Haswell, which is "not ryzen", it's not even AMD. Plus, Haswell is 12 years old at this point, how much relevance does it even have to modern Intel CPUs? Second, the "instruction decoders" power zone was only 10%, not 20%. And still reported 3% on workloads that used very few instructions and always hit the uop cache. So really we are talking about 7% overhead for decoding instructions. They do speculate that other workloads use more power (they only tested two workloads), as the theoretical instruction throughput might be double. (which is where I suspect you got the 20% from), but they provide no evidence for that, and double the throughput doesn't mean double the power consumption. And double 7% + 3% base would be 17% at most. Third. Intel doesn't publish any details about what this "instruction decoder" zone actually covers. It's almost certainly more just the "decoding x86" part. Given there are only four zones, I'm almost certain this zone covers the entire frontend, which includes branch prediction, instruction fetch, the stack engine. It might include register renaming too. Maybe instruction TLB lookups? I am reasonably sure it includes the (dynamic) power cost of accessing the L1i cache too. So this 7% power usage is way more than just the decoding of decoding 86%. It's the entire frontend. Finally. I haven't seen any power numbers for the front end of an equivalent ARM processor (like Apple's M1). For all we know, they are also using 7% of their power budget to fetch ARM instructions from the L1 cache, decode them, do branch prediction, do all the fancy front-end stuff. The 7% number isn't x86 overhead as many people imply, it's just the cost of running Haswell's frontend. Without anything else to compare to, this 7% number is worthless. It's certainly an interesting paper, I don't have any major criticisms, but it simply cannot be used to support (or disprove) any arguments about the overhead of x86 decoding. [1] https://www.usenix.org/system/files/conference/cooldc16/cooldc16-paper-hirki.pdf https://www.usenix.org/system/files/conference/cooldc16/cool...