6 ms·
Zen 5's AVX-512 Frequency Behavior
- kristianp 2y agoI find it irritating that they are comparing clock scaling to the venerable Skylake-X. Surely Sapphire Rapids has been out for almost 2 years by now.
- fuhsnn 2y agoI think it's mostly the lack of comparable research other than the Skylake-X one by Travis Downs. I too would like to see how Zen 4 behaves in the situation with its double-pumping.
- eqvinox 2y agoSeemed appropriate to me as comparing the "first core to use full-width AVX-512 datapaths"; my interpretation is that AMD threw more R&D into this than Intel before shipping it to customers… (It's also not really a comparative article at all? Skylake-X is mostly just introduction…)
- kristianp 2y ago> my interpretation is that AMD threw more R&D into this than Intel before shipping it to customers AMD had the benefit of learning from Intel's mistakes in their first generation of AVX-512 chips. It seemed unfair to compare an Intel chip that's so old (albeit long-lasting due to Intel's scaling problems). Skylake-X chips were released in 2017! [1] [1] https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#High-end_desktop_processors_(Skylake-X) https://en.wikipedia.org/wiki/Skylake_(microarchitecture)#Hi...
- eqvinox 2y agosure, but AMD's decision to start with a narrower datapath happened without insight from Intel's mistakes and could very well have backfired (if Intel had managed to produce a better-working implementation faster, that could've cost AMD a lot of market share). Intel had the benefit of designing the instructions along with the implementation as well, and also the choice of starting on a 2x256 datapath... And again, yeah it's not great were it a comparison, but it really just doesn't read as a comparison to me at all. It's a reference.
- adrian_b 2y agoAMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache memory. The most recent Intel and AMD CPU cores (Lion Cove and Zen 5) have identical vector datapath widths, but for many years, for 256-bit AVX Intel had a narrower datapath than AMD, 768-bit for Intel (3 x 256-bit) vs. 1024-bit for AMD (4 x 256-bit). Only when executing 512-bit AVX-512 instructions, the vector datapath of Intel was extended to 1024-bit (2 x 512-bit), matching the datapath used by AMD for any vector instructions. There were only 2 advantages of Intel AVX-512 vs. AMD executing AVX or the initial AVX-512 implementation of Zen 4. The first was that some Intel CPU models, but only the more expensive SKUs, i.e. most of the Gold and all of the Platinum, had 2 x 512-bit FMA units, while the cheap Intel CPUs and AMD Zen 4 had only one 512-bit FMA unit (but AMD Zen 4 still had 2 x 512-bit FADD units). Therefore Intel could do 2 FMUL or FMA per clock cycle, while Zen 4 could do only 1 FMUL or FMA (+ 1 FADD). The second was that Intel had a double width link to the L1 cache, so it could do 2 x 512-bit loads + 1 x 512-bit stores per clock cycle, while Zen 4 could do only 1 x 512-bit loads per cycle + 1 x 512-bit stores every other cycle. (In a balanced CPU core design the throughput for vector FMA and for vector loads from the L1 cache must be the same, which is true for both old and new Intel and AMD CPU cores.) With the exception of vector load/store and FMUL/FMA, Zen 4 had the same or better AVX-512 throughput for most instructions, in comparison with Intel Sapphire Rapids/Emerald Rapids. There were a few instructions with a poor implementation on Zen 4 and a few instructions with a poor implementation on Intel, where either Intel or AMD were significantly better than the other.
- eqvinox 2y ago> AMD did not start with a narrower datapath, even if this is a widespread myth. It only had a narrower path between the inner CPU core and the L1 data cache memory. https://www.mersenneforum.org/node/21615#post614191 https://www.mersenneforum.org/node/21615#post614191 "Thus as many of us predicted, 512-bit instructions are split into 2 x 256-bit of the same instruction. And 512-bit is always half-throughput of their 256-bit versions." Is that wrong? There's a lot of it being described as "double pumped" going around… (tbh I couldn't care less about how wide the interface buses are, as long as they can deliver in sum total a reasonable bandwidth at a reasonable latency… especially on the further out cache hierarchies the latency overshadows the width so much it doesn't matter if it comes down to 1×512 or 2×256. The question at hand here is the total width of the ALUs and effective IPC.)
- adrian_b 2y agoTrue, but who would bother to pay a lot of money for a CPU that is known to be inferior to alternatives, only to be able to test the details of its performance.
- crest 2y agoA well funded investigative tech journalist?
- adrian_b 2y agoWith prices in the range $2,000 - $20,000 for the CPU, plus a couple grand for at least MB, cooler, memory and PSU, the journalist must be very well funded to spend that much for publishing one article analyzing the CPU. I would like to read such an article or to be able to test myself such a CPU, but my curiosity is not so great as to make me spend such money. For now, the best one can do is to examine the results of general-purpose benchmarks published on sites like: https://www.servethehome.com/ https://www.servethehome.com/ https://www.phoronix.com/ https://www.phoronix.com/ where Intel, AMD or Ampere send some review systems with Xeon, Epyc or Arm-based server CPUs. These sites are useful, but a more thorough micro-architectural investigation would have been nice.
- kristianp 2y agoAWS, dedicated EC2 instance. A few dollars an hour.
- formerly_proven 2y agoCascade Lake improved the situation a bit, but then you had Ice Lake where iirc the hard cutoffs were gone and you were just looking at regular power and thermal steering. IIRC, that was the first generation where we enabled AVX512 for all workloads.
- Earw0rm 2y agoDepending on which CPU category. I think Intel HEDT stops at Cascade Lake, which is essentially Skylake-X Refresh from 2019? Whereas AMD has full-fat AVX512 even in gaming laptop CPUs.
- deleted 2y ago[deleted]
- Remnant44 2y agoIt's interesting that Zen5's FPUs running in full 512bit wide mode doesn't actually seem to cause any trouble, but that lighting up the load store units does. I don't know enough about hardware-level design to know if this would be "expected". The fully investigation in this article is really interesting, but the TL;DR is: Light up enough of the core, and frequencies will have to drop to maintain power envelope. The transition period is done very smartly, but it still exists - but as opposed to the old intel avx512 cores that got endless (deserved?) bad press for their transition behavior, this is more or less seamless.
- eqvinox 2y agoReading the section under "Load Another FP Pipe?" I'm coming away with the impression that it's not the LSU but rather total overall load that causes trouble. While that section is focused on transition time, the end steady state is also slower…
- tanelpoder 2y agoI haven’t read the article yet, but back when I tried to get to over 100 GB/s IO rate from a bunch of SSDs on Zen4 (just fio direct IO workload without doing anything with the data), I ended up disabling Core Boost states (or maybe something else in BIOS too), to give more thermal allowance for the IO hub on the chip. As RAM load/store traffic goes through the IO hub too, maybe that’s it?
- eqvinox 2y agoI don't think these things are related, this is talking about the LSU right inside the core. I'd also expect oscillations if there were a thermal problem like you're describing, i.e. core clocks up when IO hub delivers data, IO hub stalls, causes core to stall as well, IO hub can run again delivering data, repeat from beginning. (Then again, boost clocks are an intentional oscillation anyway…)
- tanelpoder 2y ago
- Szpadel 2y agoI'm curious how changing some OC parameters would affect those results if that it caused by voltage drop, how load line calibration affects it? is that's power constraint then how PBO would affect it?
- menaerus 2y agoI don't understand why 2x FMAs in CPU design poses such a challenge when GPUs literally have hundreds of such ALUs? Both operate at similar TDP so where's the catch? Much lower GPU clock frequency?
- eqvinox 2y agoIt's not 2 FMAs, it's AVX-512 (and going with 32-bit words) ⇒ 2*512/32 = 32 FMAs per core, 256 on an 8-core CPU. The unit counts for GPUs - depending on which number you look at - count these separately. CPUs also have much more complicated program flow control, versatility, and AFAIK latency (⇒ flow control cost) of individual instructions. GPUs are optimized for raw calculation throughput meanwhile. Also note that modern GPUs and CPUs don't have a clear pricing relationship anymore, e.g. a desktop CPU is much cheaper than a high-end GPU, and large server CPUs are more expensive than either.
- menaerus 2y ago1x 512-bit FMA or 2x 256-bit FMAs or 4x 128-bit FMAs is irrelevant here - it's still a single physical unit in a CPU that consumes 512 bits of data bandwidth. The question is why the CPU budget allows for 2x 512-bit or 4x 256-bit while H100, for example, has 14592 FP32 CUDA cores - in AVX terminology that would translate, if I am not mistaken, to 7926x 512-bit or 14592x 256-bit FMAs per clock cycle. Even considering the obvious differences between GPUs and CPUs, this is still a large difference. Since GPU cores operate at much lower frequencies than CPU cores, it is what it made me believe where the biggest difference comes from.
- eqvinox 2y agoAIUI an FP32 core is only 32 bits wide, but this is outside my area of expertise really. Also note that CPUs also have additional ALUs that can't do FMAs, FMA is just the most capable one. You're also repeating 2×512 / 4×256 — that's per core, you need to multiply by CPU core count. [also, note e.g. an 8-core CPU is much cheaper than a H100 card ;) — if anything you'd be comparing the highest end server CPUs here. An 192-core Zen5c is 8.2~10.5k€ open retail, an H100 is 32~35k€…] [reading through some random docs, a CPU core seems vaguely comparable to a SM; a SM might have 128 or 64 lanes (=FP32 cores) while a CPU only has 16 with AVX-512, but there is indeed also a notable clock difference and far more flexibility otherwise in the CPU core (which consumes silicon area)]
- sylware 2y agoThey should put forward the fact that 512bits is the "sweet spot" as it is a data cache-line!
- ksec 2y agoI wonder if this will be improved or fix in Zen 6. Although personally I much rather they focus on IPC.
- Remnant44 2y agoNothing to fix here. The behavior in the transition regimes is already quite good. The overall throttling is dynamic and reactive based on heat and power draw - this is unavoidable and in fact desirable (the alternative is to simply run slower all the time, not to somehow be immune to physics and run faster all the time)
- mgaunard 2y agoIn practice everyone turns off AVX512 because they're afraid of the frequency throttling. The damage was made by Skylake-X and won't be healed for years.