5 ms·

> my interpretation is that AMD threw more R&D into this than Intel before shipping it to customers AMD had the benefit of learning from Intel's mistakes in th

> my interpretation is that AMD threw more R&D into this than Intel before shipping it to customers AMD had the benefit of learning from Intel's mistakes in their first generation of AVX-512 chips. It seemed unfair to compare an Intel chip that's so old (albeit long-lasting due to Intel's scaling problems). Skylake-X chips were released in 2017! [1] [1] https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#High-end_desktop_processors_(Skylake-X) https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#Hi...

eqvinox 2y ago
sure, but AMD's decision to start with a narrower datapath happened without insight from Intel's mistakes and could very well have backfired (if Intel had managed to produce a better-working implementation faster, that could've cost AMD a lot of market share). Intel had the benefit of designing the instructions along with the implementation as well, and also the choice of starting on a 2x256 datapath... And again, yeah it's not great were it a comparison, but it really just doesn't read as a comparison to me at all. It's a reference.
- adrian_b 2y ago
  AMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache memory. The most recent Intel and AMD CPU cores (Lion Cove and Zen 5) have identical vector datapath widths, but for many years, for 256-bit AVX Intel had a narrower datapath than AMD, 768-bit for Intel (3 x 256-bit) vs. 1024-bit for AMD (4 x 256-bit). Only when executing 512-bit AVX-512 instructions, the vector datapath of Intel was extended to 1024-bit (2 x 512-bit), matching the datapath used by AMD for any vector instructions. There were only 2 advantages of Intel AVX-512 vs. AMD executing AVX or the initial AVX-512 implementation of Zen 4. The first was that some Intel CPU models, but only the more expensive SKUs, i.e. most of the Gold and all of the Platinum, had 2 x 512-bit FMA units, while the cheap Intel CPUs and AMD Zen 4 had only one 512-bit FMA unit (but AMD Zen 4 still had 2 x 512-bit FADD units). Therefore Intel could do 2 FMUL or FMA per clock cycle, while Zen 4 could do only 1 FMUL or FMA (+ 1 FADD). The second was that Intel had a double width link to the L1 cache, so it could do 2 x 512-bit loads + 1 x 512-bit stores per clock cycle, while Zen 4 could do only 1 x 512-bit loads per cycle + 1 x 512-bit stores every other cycle. (In a balanced CPU core design the throughput for vector FMA and for vector loads from the L1 cache must be the same, which is true for both old and new Intel and AMD CPU cores.) With the exception of vector load/store and FMUL/FMA, Zen 4 had the same or better AVX-512 throughput for most instructions, in comparison with Intel Sapphire Rapids/Emerald Rapids. There were a few instructions with a poor implementation on Zen 4 and a few instructions with a poor implementation on Intel, where either Intel or AMD were significantly better than the other.
  - eqvinox 2y ago
    > AMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache memory. https://www.mersenneforum.org/node/21615#post614191 https://www.mersenneforum.org/node/21615#post614191 "Thus as many of us predicted, 512-bit instructions are split into 2 x 256-bit of the same instruction. And 512-bit is always half-throughput of their 256-bit versions." Is that wrong? There's a lot of it being described as "double pumped" going around… (tbh I couldn't care less about how wide the interface buses are, as long as they can deliver in sum total a reasonable bandwidth at a reasonable latency… especially on the further out cache hierarchies the latency overshadows the width so much it doesn't matter if it comes down to 1×512 or 2×256. The question at hand here is the total width of the ALUs and effective IPC.)
    - adrian_b 2y ago
      Sorry, but you did not read carefully that good article and you did not read the AMD documentation and the Intel documentation. The AMD Zen cores had for several generations, until Zen 4, 4 (four) vector execution units with a width of 256 bits, i.e. a total datapath width of 1024 bits. On an 1024-bit datapath, you can execute either four 256-bit instructions per clock cycle or two 512-bit instructions per clock cycle. While the number of instructions executed per cycle varies, the data processing throughput is the same, 1024 bits per clock cycle, as determined by the datapath width. The use of the word "double-pumped" by the AMD CEO has been a very unfortunate choice, because it has been completely misunderstood by most people, who have never read the AMD technical documentation and who have never tested the behavior of the micro-architecture of the Zen CPU cores. On Zen 4, the advantage of using AVX-512 is not from a different throughput, but it is caused by a better instruction set and by the avoiding of bottlenecks in the CPU core front-end, at instruction fetching, decoding, renaming and dispatching. On the Intel P cores before Lion Cove, the datapath for 256-bit instructions had a width of 768 bits, as they had three 256-bit execution units. For most 256-bit instructions the throughput was of 768 bits per clock cycle. However the three execution units were not identical, so some 256-bit instructions had only a throughput of 512 bits per cycle. When the older Intel P cores executed 512-bit instructions, the instructions with a 512 bit/cycle throughput remained at that throughput, but most of the instructions with a 768 bit/cycle throughput had their throughput increased to 1024 bit/cycle, matching the AMD throughput, by using an additional 256-bit datapath section that stayed unused when executing 256-bit or narrower instructions. While what is said above applies to most vector instructions, floating-point multiplication and FMA have different rules, because their throughput is not determined by the width of the datapath, but it may be smaller, being determined by the number of available floating-point multipliers. Cheap Intel CPUs and AMD Zen 2/Zen 3/Zen 4 had FP multipliers with a total throughput of 512 bits of results per clock cycle, while the expensive Xeon Gold and Platinum had FP multipliers with a total throughput of 1024 bits of results per clock cycle. The "double-pumped" term is applicable only to FP multiplication, where Zen 4 and cheap Intel CPUs require a double number of clock cycles to produce the same results as expensive Intel CPUs. It may be also applied, even if that is even less appropriate, to vector load and store, where the path to the L1 data cache was narrower in Zen 4 than in Intel CPUs. The "double-pumped" term is not applicable to the very large number of other AVX-512 instructions, whose throughput is determined by the width of the vector datapath, not by the width of the FP multipliers or by the L1 data cache connection. Zen 5 doubles the vector datapath width to 2048 bits, so many 512-bit AVX-512 instructions have a 2048 bit/cycle throughput, except FMUL/FMA, which have an 1024 bit/cycle throughput, determined by the width of the FP multipliers. (Because there are only 4 execution units, 256-bit instructions cannot use the full datapath.) Intel Diamond Rapids, expected by the end of 2026, is likely to have the same vector throughput as Zen 5. Until then, the Lion Cove cores from consumer CPUs, like Arrow Lake S, Arrow Lake H and Lunar Lake, are crippled, by having a half-width datapath of 1024 bits, which cannot compete with a Zen 5 that executes AVX-512 instructions.
- crest 2y ago
  AMD also has a long history of using half-width, double-pumped SIMD implementations. It worked each time. With Zen 4 the surprise wasn't that it worked at all, but how well it worked and how little the core grew in any relevant metric to support it: size, static and dynamic power. It makes me wonder why Intel didn't pair two efficiency cores with half width SIMD data paths per core sharing a single full width permute unit between them.

eqvinox 2y ago

sure, but AMD's decision to start with a narrower datapath happened without insight from Intel's mistakes and could very well have backfired (if Intel had managed to produce a better-working implementation faster, that could've cost AMD a lot of market share). Intel had the benefit of designing the instructions along with the implementation as well, and also the choice of starting on a 2x256 datapath... And again, yeah it's not great were it a comparison, but it really just doesn't read as a comparison to me at all. It's a reference.

adrian_b 2y ago
AMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache memory. The most recent Intel and AMD CPU cores (Lion Cove and Zen 5) have identical vector datapath widths, but for many years, for 256-bit AVX Intel had a narrower datapath than AMD, 768-bit for Intel (3 x 256-bit) vs. 1024-bit for AMD (4 x 256-bit). Only when executing 512-bit AVX-512 instructions, the vector datapath of Intel was extended to 1024-bit (2 x 512-bit), matching the datapath used by AMD for any vector instructions. There were only 2 advantages of Intel AVX-512 vs. AMD executing AVX or the initial AVX-512 implementation of Zen 4. The first was that some Intel CPU models, but only the more expensive SKUs, i.e. most of the Gold and all of the Platinum, had 2 x 512-bit FMA units, while the cheap Intel CPUs and AMD Zen 4 had only one 512-bit FMA unit (but AMD Zen 4 still had 2 x 512-bit FADD units). Therefore Intel could do 2 FMUL or FMA per clock cycle, while Zen 4 could do only 1 FMUL or FMA (+ 1 FADD). The second was that Intel had a double width link to the L1 cache, so it could do 2 x 512-bit loads + 1 x 512-bit stores per clock cycle, while Zen 4 could do only 1 x 512-bit loads per cycle + 1 x 512-bit stores every other cycle. (In a balanced CPU core design the throughput for vector FMA and for vector loads from the L1 cache must be the same, which is true for both old and new Intel and AMD CPU cores.) With the exception of vector load/store and FMUL/FMA, Zen 4 had the same or better AVX-512 throughput for most instructions, in comparison with Intel Sapphire Rapids/Emerald Rapids. There were a few instructions with a poor implementation on Zen 4 and a few instructions with a poor implementation on Intel, where either Intel or AMD were significantly better than the other.
- eqvinox 2y ago
  > AMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache memory. https://www.mersenneforum.org/node/21615#post614191 https://www.mersenneforum.org/node/21615#post614191 "Thus as many of us predicted, 512-bit instructions are split into 2 x 256-bit of the same instruction. And 512-bit is always half-throughput of their 256-bit versions." Is that wrong? There's a lot of it being described as "double pumped" going around… (tbh I couldn't care less about how wide the interface buses are, as long as they can deliver in sum total a reasonable bandwidth at a reasonable latency… especially on the further out cache hierarchies the latency overshadows the width so much it doesn't matter if it comes down to 1×512 or 2×256. The question at hand here is the total width of the ALUs and effective IPC.)
  - adrian_b 2y ago
    Sorry, but you did not read carefully that good article and you did not read the AMD documentation and the Intel documentation. The AMD Zen cores had for several generations, until Zen 4, 4 (four) vector execution units with a width of 256 bits, i.e. a total datapath width of 1024 bits. On an 1024-bit datapath, you can execute either four 256-bit instructions per clock cycle or two 512-bit instructions per clock cycle. While the number of instructions executed per cycle varies, the data processing throughput is the same, 1024 bits per clock cycle, as determined by the datapath width. The use of the word "double-pumped" by the AMD CEO has been a very unfortunate choice, because it has been completely misunderstood by most people, who have never read the AMD technical documentation and who have never tested the behavior of the micro-architecture of the Zen CPU cores. On Zen 4, the advantage of using AVX-512 is not from a different throughput, but it is caused by a better instruction set and by the avoiding of bottlenecks in the CPU core front-end, at instruction fetching, decoding, renaming and dispatching. On the Intel P cores before Lion Cove, the datapath for 256-bit instructions had a width of 768 bits, as they had three 256-bit execution units. For most 256-bit instructions the throughput was of 768 bits per clock cycle. However the three execution units were not identical, so some 256-bit instructions had only a throughput of 512 bits per cycle. When the older Intel P cores executed 512-bit instructions, the instructions with a 512 bit/cycle throughput remained at that throughput, but most of the instructions with a 768 bit/cycle throughput had their throughput increased to 1024 bit/cycle, matching the AMD throughput, by using an additional 256-bit datapath section that stayed unused when executing 256-bit or narrower instructions. While what is said above applies to most vector instructions, floating-point multiplication and FMA have different rules, because their throughput is not determined by the width of the datapath, but it may be smaller, being determined by the number of available floating-point multipliers. Cheap Intel CPUs and AMD Zen 2/Zen 3/Zen 4 had FP multipliers with a total throughput of 512 bits of results per clock cycle, while the expensive Xeon Gold and Platinum had FP multipliers with a total throughput of 1024 bits of results per clock cycle. The "double-pumped" term is applicable only to FP multiplication, where Zen 4 and cheap Intel CPUs require a double number of clock cycles to produce the same results as expensive Intel CPUs. It may be also applied, even if that is even less appropriate, to vector load and store, where the path to the L1 data cache was narrower in Zen 4 than in Intel CPUs. The "double-pumped" term is not applicable to the very large number of other AVX-512 instructions, whose throughput is determined by the width of the vector datapath, not by the width of the FP multipliers or by the L1 data cache connection. Zen 5 doubles the vector datapath width to 2048 bits, so many 512-bit AVX-512 instructions have a 2048 bit/cycle throughput, except FMUL/FMA, which have an 1024 bit/cycle throughput, determined by the width of the FP multipliers. (Because there are only 4 execution units, 256-bit instructions cannot use the full datapath.) Intel Diamond Rapids, expected by the end of 2026, is likely to have the same vector throughput as Zen 5. Until then, the Lion Cove cores from consumer CPUs, like Arrow Lake S, Arrow Lake H and Lunar Lake, are crippled, by having a half-width datapath of 1024 bits, which cannot compete with a Zen 5 that executes AVX-512 instructions.
crest 2y ago
AMD also has a long history of using half-width, double-pumped SIMD implementations. It worked each time. With Zen 4 the surprise wasn't that it worked at all, but how well it worked and how little the core grew in any relevant metric to support it: size, static and dynamic power. It makes me wonder why Intel didn't pair two efficiency cores with half width SIMD data paths per core sharing a single full width permute unit between them.