4 ms·

I bet AMD itself is not able to get even close to the 204 GB/s bandwidth that they "promise" for these CPUs. AMD instructions for measuring peak bandwidth are

I bet AMD itself is not able to get even close to the 204 GB/s bandwidth that they "promise" for these CPUs. AMD instructions for measuring peak bandwidth are here: https://developer.amd.com/spack/stream-benchmark/ https://developer.amd.com/spack/stream-benchmark/ (essentially does a multi threaded memcpy). The script there will pin 1 thread per CCD, and with 8 threads you'll get close to peak bandwidth on CPUs with 8 CCDs (adding more threads doesn't help). I've run it in the same CPUs that AMD uses there, following their instructions to the letter, which should be able to achieve 204 GB/s, but the best I've seen is about 165 GB/s. That's 80% of what AMD advertises. HOWEVER, THIS NUMBER IS A LIE. It is using the AMD compiler with a flag called "-fnt-store" to turn all stores into non-temporal stores. This is something that no real-world application in practice would actually do because it does not make sense doing in general. Without this flag, the peak bandwidth that AMD scripts report is about 124 GB/s. That's 60% of what AMD advertises and the best that most applications can aim to obtain in practice (the blog post author is really close to it). The fine print (https://www.amd.com/en/products/cpu/amd-epyc-7742 https://www.amd.com/en/products/cpu/amd-epyc-7742) does say that 204 GB/s is a "theoretical" number. So I bought a "theoretical" 204 GB/s CPU that delivers 60% of what it "theoretically" promised. I feel scammed. --- Instead, if I run STREAM on my GPU, I get close to 100% of what the GPU promises, and about 90% or so of it in practice.

ashvardanian 5y ago
Those are great links, thanks for sharing! My memcpy-s were even slower :)
sliken 5y ago
You seem surprised, this is pretty much the rule for ALL CPUs. The 204GB/sec number is simply DDR4-3200 x 64 bits (width of one memory channel) x 8 (number of memory channels = 204,800MB/sec or "roughly" 204GB/sec (assuming GB = 10^9 not 2^30). This is called "peak" bandwidth, as is a never to exceed number. 60-80% of peak is common with tuned code, and less without. Sure GPUs, which are all about bandwidth do better. Much of a GPU design is about throughput and hiding latency with multiple requests/threads/whatever. CPUs have different design goals and worry much more about latency and often have to worry about NUMA/memory coherency. Not to mention in the general case often you have one cache miss per core, which isn't necessarily enough to saturate the memory bus. Try apple M1, other ARMs, Alpha, PA-risc, Sparc, Intel, etc and you'll find the same story.
- ggreg84 5y ago
  > This is called "peak" bandwidth, as is a never to exceed number. This is the peak bandwidth of DDR4, saying that would be fine. AMD claims that this is the theoretical peak bandwidth of their CPU, not of DDR4, yet their CPUs memory subsystem are too poor to even get close to this number. > Try apple M1, other ARMs, Alpha, PA-risc, Sparc, Intel, etc and you'll find the same story. I tried the Ampere Altra and Graviton and they achieve ~170-180 GB/s without non-temporal stores. You can get an AWS graviton instance and verify it yourself. That's 90% of "theoretical DDR4 peak" for ARM vs 60% for AMD. So no, i disagree. This is not the case for ALL cpus, this is the case for AMD cpus. And no, 60% vs 90% "out of the box" perf is not the same.
  - sliken 5y ago
    Arm does have an advantage with a looser memory model, which makes it easier to get good bandwidth. Graviton2 and Ampere Altra tend to have more cores than common x86-64 servers (64-128), which helps as well. 90% is more than I'd expect though, can you post the numbers and which code you used? Enough to replicate (compiler, compiler flags, code, etc). I've tracked down lower than expected numbers before, there's many causes. Compilers (temporal vs non-temporal), GCC vs Intel vs Other compilers, surprisingly large differences because of using new or malloc instead of static arrays, having two Dimms per channel (lowering the speed to DDR4-2666), various BIOS settings including a NUMA disable that stripes across all channels that apparently makes some windows codes run better. In a surprising number of cases just having dimms in the wrong slots will do it.
    - ggreg84 5y ago
      Virginia Tech Stream, stock, with OMP_NUM_THREADS set to number of cores (these don't have SMT), clang-12 compiler using -O3 -fopenmp -DNDEBUG. > Arm does have an advantage As I said, claiming that the theoretical peak of a CPU can actually achieve theoretical peak of DDR4 is ok as long as that's actually the case. Getting 50-60% just means AMD is lying. Particularly when on other vendors like Amazon and Ampere one gets 90+% of what they advertise with stock compilers and stock flags. Having to use an AMD proprietary compiler, with flags that are bad for programs in general (essentially a STREAM cheat mode), to get 70% of what they advertise, is just sad in 2022.
dragontamer 5y ago
I'm somewhat surprised that AMD's stream-benchmark uses "transparent_hugepage" rather than explicitly using huge-pages. Has Linux's transparent_hugepage feature gotten good enough to use in today's environments? Maybe its good enough for STREAM-benchmarking (simple memory patterns) ?? Or is "STREAM" unable to use huge-pages directly (so we require the OS to try and guess appropriate usage?) ??
- blihp 5y ago
  Yes, it works quite well. Explicit 1gb huge pages also provides significant improvements for the right workloads. You pretty much have to start taking advantage of features like these to get anywhere near the theoretical maximum throughput in real world usage even on the higher core count desktop chips.

ashvardanian 5y ago

Those are great links, thanks for sharing! My memcpy-s were even slower :)

sliken 5y ago

You seem surprised, this is pretty much the rule for ALL CPUs. The 204GB/sec number is simply DDR4-3200 x 64 bits (width of one memory channel) x 8 (number of memory channels = 204,800MB/sec or "roughly" 204GB/sec (assuming GB = 10^9 not 2^30). This is called "peak" bandwidth, as is a never to exceed number. 60-80% of peak is common with tuned code, and less without. Sure GPUs, which are all about bandwidth do better. Much of a GPU design is about throughput and hiding latency with multiple requests/threads/whatever. CPUs have different design goals and worry much more about latency and often have to worry about NUMA/memory coherency. Not to mention in the general case often you have one cache miss per core, which isn't necessarily enough to saturate the memory bus. Try apple M1, other ARMs, Alpha, PA-risc, Sparc, Intel, etc and you'll find the same story.

ggreg84 5y ago
> This is called "peak" bandwidth, as is a never to exceed number. This is the peak bandwidth of DDR4, saying that would be fine. AMD claims that this is the theoretical peak bandwidth of their CPU, not of DDR4, yet their CPUs memory subsystem are too poor to even get close to this number. > Try apple M1, other ARMs, Alpha, PA-risc, Sparc, Intel, etc and you'll find the same story. I tried the Ampere Altra and Graviton and they achieve ~170-180 GB/s without non-temporal stores. You can get an AWS graviton instance and verify it yourself. That's 90% of "theoretical DDR4 peak" for ARM vs 60% for AMD. So no, i disagree. This is not the case for ALL cpus, this is the case for AMD cpus. And no, 60% vs 90% "out of the box" perf is not the same.
- sliken 5y ago
  Arm does have an advantage with a looser memory model, which makes it easier to get good bandwidth. Graviton2 and Ampere Altra tend to have more cores than common x86-64 servers (64-128), which helps as well. 90% is more than I'd expect though, can you post the numbers and which code you used? Enough to replicate (compiler, compiler flags, code, etc). I've tracked down lower than expected numbers before, there's many causes. Compilers (temporal vs non-temporal), GCC vs Intel vs Other compilers, surprisingly large differences because of using new or malloc instead of static arrays, having two Dimms per channel (lowering the speed to DDR4-2666), various BIOS settings including a NUMA disable that stripes across all channels that apparently makes some windows codes run better. In a surprising number of cases just having dimms in the wrong slots will do it.
  - ggreg84 5y ago
    Virginia Tech Stream, stock, with OMP_NUM_THREADS set to number of cores (these don't have SMT), clang-12 compiler using -O3 -fopenmp -DNDEBUG. > Arm does have an advantage As I said, claiming that the theoretical peak of a CPU can actually achieve theoretical peak of DDR4 is ok as long as that's actually the case. Getting 50-60% just means AMD is lying. Particularly when on other vendors like Amazon and Ampere one gets 90+% of what they advertise with stock compilers and stock flags. Having to use an AMD proprietary compiler, with flags that are bad for programs in general (essentially a STREAM cheat mode), to get 70% of what they advertise, is just sad in 2022.

dragontamer 5y ago

I'm somewhat surprised that AMD's stream-benchmark uses "transparent_hugepage" rather than explicitly using huge-pages. Has Linux's transparent_hugepage feature gotten good enough to use in today's environments? Maybe its good enough for STREAM-benchmarking (simple memory patterns) ?? Or is "STREAM" unable to use huge-pages directly (so we require the OS to try and guess appropriate usage?) ??

blihp 5y ago
Yes, it works quite well. Explicit 1gb huge pages also provides significant improvements for the right workloads. You pretty much have to start taking advantage of features like these to get anywhere near the theoretical maximum throughput in real world usage even on the higher core count desktop chips.