3 ms·

> AMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache me

> AMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache memory. https://www.mersenneforum.org/node/21615#post614191 https://www.mersenneforum.org/node/21615#post614191 "Thus as many of us predicted, 512-bit instructions are split into 2 x 256-bit of the same instruction. And 512-bit is always half-throughput of their 256-bit versions." Is that wrong? There's a lot of it being described as "double pumped" going around… (tbh I couldn't care less about how wide the interface buses are, as long as they can deliver in sum total a reasonable bandwidth at a reasonable latency… especially on the further out cache hierarchies the latency overshadows the width so much it doesn't matter if it comes down to 1×512 or 2×256. The question at hand here is the total width of the ALUs and effective IPC.)

adrian_b 2y ago
Sorry, but you did not read carefully that good article and you did not read the AMD documentation and the Intel documentation. The AMD Zen cores had for several generations, until Zen 4, 4 (four) vector execution units with a width of 256 bits, i.e. a total datapath width of 1024 bits. On an 1024-bit datapath, you can execute either four 256-bit instructions per clock cycle or two 512-bit instructions per clock cycle. While the number of instructions executed per cycle varies, the data processing throughput is the same, 1024 bits per clock cycle, as determined by the datapath width. The use of the word "double-pumped" by the AMD CEO has been a very unfortunate choice, because it has been completely misunderstood by most people, who have never read the AMD technical documentation and who have never tested the behavior of the micro-architecture of the Zen CPU cores. On Zen 4, the advantage of using AVX-512 is not from a different throughput, but it is caused by a better instruction set and by the avoiding of bottlenecks in the CPU core front-end, at instruction fetching, decoding, renaming and dispatching. On the Intel P cores before Lion Cove, the datapath for 256-bit instructions had a width of 768 bits, as they had three 256-bit execution units. For most 256-bit instructions the throughput was of 768 bits per clock cycle. However the three execution units were not identical, so some 256-bit instructions had only a throughput of 512 bits per cycle. When the older Intel P cores executed 512-bit instructions, the instructions with a 512 bit/cycle throughput remained at that throughput, but most of the instructions with a 768 bit/cycle throughput had their throughput increased to 1024 bit/cycle, matching the AMD throughput, by using an additional 256-bit datapath section that stayed unused when executing 256-bit or narrower instructions. While what is said above applies to most vector instructions, floating-point multiplication and FMA have different rules, because their throughput is not determined by the width of the datapath, but it may be smaller, being determined by the number of available floating-point multipliers. Cheap Intel CPUs and AMD Zen 2/Zen 3/Zen 4 had FP multipliers with a total throughput of 512 bits of results per clock cycle, while the expensive Xeon Gold and Platinum had FP multipliers with a total throughput of 1024 bits of results per clock cycle. The "double-pumped" term is applicable only to FP multiplication, where Zen 4 and cheap Intel CPUs require a double number of clock cycles to produce the same results as expensive Intel CPUs. It may be also applied, even if that is even less appropriate, to vector load and store, where the path to the L1 data cache was narrower in Zen 4 than in Intel CPUs. The "double-pumped" term is not applicable to the very large number of other AVX-512 instructions, whose throughput is determined by the width of the vector datapath, not by the width of the FP multipliers or by the L1 data cache connection. Zen 5 doubles the vector datapath width to 2048 bits, so many 512-bit AVX-512 instructions have a 2048 bit/cycle throughput, except FMUL/FMA, which have an 1024 bit/cycle throughput, determined by the width of the FP multipliers. (Because there are only 4 execution units, 256-bit instructions cannot use the full datapath.) Intel Diamond Rapids, expected by the end of 2026, is likely to have the same vector throughput as Zen 5. Until then, the Lion Cove cores from consumer CPUs, like Arrow Lake S, Arrow Lake H and Lunar Lake, are crippled, by having a half-width datapath of 1024 bits, which cannot compete with a Zen 5 that executes AVX-512 instructions.
- crest 2y ago
  Isn't it misleading to just add up the output width of all SIMD ALU pipelines and call the sum "datapath width", because you can't freely mix and match when the available ALUs pipelines determine what operations you can compute at full width?
  - adrian_b 2y ago
    You are right that in most CPUs the 3 or 4 vector execution units are not completely identical. Therefore some operations may use the entire datapath width, while others may use only a fraction, e.g. only a half or only two thirds or only three quarters. However you cannot really discuss these details without listing all such instructions, i.e. reproducing the tables from the Intel or AMD optimization guides of from Agner Fog's optimization documents. For the purpose of this discussion thread, these details are not really relevant, because for Intel and AMD the classification of the instructions is mostly the same, i.e. the cheap instructions, like addition operations, can be executed in all execution units, using the entire datapath width, while certain more expensive operations, like multiplication/division/square root/shuffle may be done only in a subset of the execution units, so they can use only a fraction of the datapath width (but when possible they will be coupled with simple instructions using the remainder of the datapath, maintaining a total throughput equal with the datapath width). Because most instructions are classified by cost in the same way by AMD and Intel, the throughput ratio between AMD and Intel is typically the same both for instructions using the full datapath width and for those using only a fraction. Like I have said, with very few exceptions (including FMUL/FMA/LD/ST), the throughput for 512-bit instructions has been the same for Zen 4 and the Intel CPUs with AVX-512 support, as determined by the common 1024-bit datapath width, including for the instructions that could use only a half-width 512-bit datapath.
    - dzaima 2y ago
      Wouldn't it be 1536-bit for 2 256-bit FMA/cycle, with FMA taking 3 inputs? (applies equally to both so doesn't change anything materially; And even goes back to Haswell, which too is capable of 2 256-bit FMA/cycle)

adrian_b 2y ago

Sorry, but you did not read carefully that good article and you did not read the AMD documentation and the Intel documentation. The AMD Zen cores had for several generations, until Zen 4, 4 (four) vector execution units with a width of 256 bits, i.e. a total datapath width of 1024 bits. On an 1024-bit datapath, you can execute either four 256-bit instructions per clock cycle or two 512-bit instructions per clock cycle. While the number of instructions executed per cycle varies, the data processing throughput is the same, 1024 bits per clock cycle, as determined by the datapath width. The use of the word "double-pumped" by the AMD CEO has been a very unfortunate choice, because it has been completely misunderstood by most people, who have never read the AMD technical documentation and who have never tested the behavior of the micro-architecture of the Zen CPU cores. On Zen 4, the advantage of using AVX-512 is not from a different throughput, but it is caused by a better instruction set and by the avoiding of bottlenecks in the CPU core front-end, at instruction fetching, decoding, renaming and dispatching. On the Intel P cores before Lion Cove, the datapath for 256-bit instructions had a width of 768 bits, as they had three 256-bit execution units. For most 256-bit instructions the throughput was of 768 bits per clock cycle. However the three execution units were not identical, so some 256-bit instructions had only a throughput of 512 bits per cycle. When the older Intel P cores executed 512-bit instructions, the instructions with a 512 bit/cycle throughput remained at that throughput, but most of the instructions with a 768 bit/cycle throughput had their throughput increased to 1024 bit/cycle, matching the AMD throughput, by using an additional 256-bit datapath section that stayed unused when executing 256-bit or narrower instructions. While what is said above applies to most vector instructions, floating-point multiplication and FMA have different rules, because their throughput is not determined by the width of the datapath, but it may be smaller, being determined by the number of available floating-point multipliers. Cheap Intel CPUs and AMD Zen 2/Zen 3/Zen 4 had FP multipliers with a total throughput of 512 bits of results per clock cycle, while the expensive Xeon Gold and Platinum had FP multipliers with a total throughput of 1024 bits of results per clock cycle. The "double-pumped" term is applicable only to FP multiplication, where Zen 4 and cheap Intel CPUs require a double number of clock cycles to produce the same results as expensive Intel CPUs. It may be also applied, even if that is even less appropriate, to vector load and store, where the path to the L1 data cache was narrower in Zen 4 than in Intel CPUs. The "double-pumped" term is not applicable to the very large number of other AVX-512 instructions, whose throughput is determined by the width of the vector datapath, not by the width of the FP multipliers or by the L1 data cache connection. Zen 5 doubles the vector datapath width to 2048 bits, so many 512-bit AVX-512 instructions have a 2048 bit/cycle throughput, except FMUL/FMA, which have an 1024 bit/cycle throughput, determined by the width of the FP multipliers. (Because there are only 4 execution units, 256-bit instructions cannot use the full datapath.) Intel Diamond Rapids, expected by the end of 2026, is likely to have the same vector throughput as Zen 5. Until then, the Lion Cove cores from consumer CPUs, like Arrow Lake S, Arrow Lake H and Lunar Lake, are crippled, by having a half-width datapath of 1024 bits, which cannot compete with a Zen 5 that executes AVX-512 instructions.

crest 2y ago
Isn't it misleading to just add up the output width of all SIMD ALU pipelines and call the sum "datapath width", because you can't freely mix and match when the available ALUs pipelines determine what operations you can compute at full width?
- adrian_b 2y ago
  You are right that in most CPUs the 3 or 4 vector execution units are not completely identical. Therefore some operations may use the entire datapath width, while others may use only a fraction, e.g. only a half or only two thirds or only three quarters. However you cannot really discuss these details without listing all such instructions, i.e. reproducing the tables from the Intel or AMD optimization guides of from Agner Fog's optimization documents. For the purpose of this discussion thread, these details are not really relevant, because for Intel and AMD the classification of the instructions is mostly the same, i.e. the cheap instructions, like addition operations, can be executed in all execution units, using the entire datapath width, while certain more expensive operations, like multiplication/division/square root/shuffle may be done only in a subset of the execution units, so they can use only a fraction of the datapath width (but when possible they will be coupled with simple instructions using the remainder of the datapath, maintaining a total throughput equal with the datapath width). Because most instructions are classified by cost in the same way by AMD and Intel, the throughput ratio between AMD and Intel is typically the same both for instructions using the full datapath width and for those using only a fraction. Like I have said, with very few exceptions (including FMUL/FMA/LD/ST), the throughput for 512-bit instructions has been the same for Zen 4 and the Intel CPUs with AVX-512 support, as determined by the common 1024-bit datapath width, including for the instructions that could use only a half-width 512-bit datapath.
  - dzaima 2y ago
    Wouldn't it be 1536-bit for 2 256-bit FMA/cycle, with FMA taking 3 inputs? (applies equally to both so doesn't change anything materially; And even goes back to Haswell, which too is capable of 2 256-bit FMA/cycle)