4 ms·
Zen 4 has equivalent (in throughput) AVX-512 units with the best current Intel CPUs, for most AVX-512 instructions. It is a myth that Zen 4 is somehow slower. F
by adrian_b 3y ago
Zen 4 has equivalent (in throughput) AVX-512 units with the best current Intel CPUs, for most AVX-512 instructions. It is a myth that Zen 4 is somehow slower. For most AVX-512 instructions, both Zen 4 and Emerald Rapids do two 512-bit instructions per cycle (Zen 4 and Zen 3 do four 256-bit instructions per cycle, while Intel does only three 256-bit instructions per cycle, so Intel shows a greater increase in speed when using 512-bit instructions only because it has a lower throughput for 256-bit instructions).
The only exceptions are the loads/stores from/to the L1 cache, which have double throughput on Intel and the floating-point fused-multiply-add units, where the most expensive Xeon SKUs can do 2 FMAs per cycle, while Zen 4 can do only 1 FMA + 1 FADD per cycle.
Zen 4 implements the BF16 instruction set, which is likely to increase the speed many times for any AI/ML workload that uses BF16. It also implements the VNNI instruction set, which will accelerate any inference that uses INT8.
Even when these dedicated instructions are not used, AVX-512 is usually much faster on Zen 4, by eliminating bottlenecks caused by instruction fetch and decoding and by using the better designed AVX-512 instructions.
- vient 3y agoHuh, seems you are right. I've read about "not really real" AVX-512 implementation in Zen 4, and also saw that on my workload turning on AVX-512 on Zen 4 indeed gives almost nothing (compared to x2 speedup on Intel). But now that you've written it, I understood that my only test was very specific, full of 32-bit FMAs, and I was comparing Zen 4 with Skylake-X which has 2 FMA units.
- Lockal 3y agoIt is very likely that they compared newly supported BF16 instruction set with some other (float? fp16? quantized ints?). If you make simple 1-to-1 transition from AVX256 to AVX512, the speedup is usually <2x, unless there is a major bottleneck in instruction fetch and decoding. Also AVX512 in Zen is still double-pumped. Regarding FMA, again, if you compare FMA in AVX256 and AVX512, latencies and throughput are the same[1]. Comparing performance between different datatypes is probably fine, but it should be stated directly. Unless "Owners of CPUs like Zen4 can expect to see 10x faster prompt eval times" means comparison with Skylake. [1] https://uops.info/table.html?search=vfmadd132ps&cb_lat=on&cb_tp=on&cb_uops=on&cb_ports=on&cb_ZEN4=on&cb_measurements=on&cb_doc=on&cb_avx512=on&cb_fma=on https://uops.info/table.html?search=vfmadd132ps&cb_lat=on&cb...
- janwas 3y agoNice result, congrats Justine! The bf16 dot instruction replaces 6 instructions: https://github.com/google/highway/blob/master/hwy/ops/x86_128-inl.h#L9214 https://github.com/google/highway/blob/master/hwy/ops/x86_12... A 3-4x speedup vs SKX sounds quite plausible :)