3 ms·

You seem surprised, this is pretty much the rule for ALL CPUs. The 204GB/sec number is simply DDR4-3200 x 64 bits (width of one memory channel) x 8 (number of

by sliken 5y ago

You seem surprised, this is pretty much the rule for ALL CPUs. The 204GB/sec number is simply DDR4-3200 x 64 bits (width of one memory channel) x 8 (number of memory channels = 204,800MB/sec or "roughly" 204GB/sec (assuming GB = 10^9 not 2^30). This is called "peak" bandwidth, as is a never to exceed number. 60-80% of peak is common with tuned code, and less without. Sure GPUs, which are all about bandwidth do better. Much of a GPU design is about throughput and hiding latency with multiple requests/threads/whatever. CPUs have different design goals and worry much more about latency and often have to worry about NUMA/memory coherency. Not to mention in the general case often you have one cache miss per core, which isn't necessarily enough to saturate the memory bus. Try apple M1, other ARMs, Alpha, PA-risc, Sparc, Intel, etc and you'll find the same story.

ggreg84 5y ago
> This is called "peak" bandwidth, as is a never to exceed number. This is the peak bandwidth of DDR4, saying that would be fine. AMD claims that this is the theoretical peak bandwidth of their CPU, not of DDR4, yet their CPUs memory subsystem are too poor to even get close to this number. > Try apple M1, other ARMs, Alpha, PA-risc, Sparc, Intel, etc and you'll find the same story. I tried the Ampere Altra and Graviton and they achieve ~170-180 GB/s without non-temporal stores. You can get an AWS graviton instance and verify it yourself. That's 90% of "theoretical DDR4 peak" for ARM vs 60% for AMD. So no, i disagree. This is not the case for ALL cpus, this is the case for AMD cpus. And no, 60% vs 90% "out of the box" perf is not the same.
- sliken 5y ago
  Arm does have an advantage with a looser memory model, which makes it easier to get good bandwidth. Graviton2 and Ampere Altra tend to have more cores than common x86-64 servers (64-128), which helps as well. 90% is more than I'd expect though, can you post the numbers and which code you used? Enough to replicate (compiler, compiler flags, code, etc). I've tracked down lower than expected numbers before, there's many causes. Compilers (temporal vs non-temporal), GCC vs Intel vs Other compilers, surprisingly large differences because of using new or malloc instead of static arrays, having two Dimms per channel (lowering the speed to DDR4-2666), various BIOS settings including a NUMA disable that stripes across all channels that apparently makes some windows codes run better. In a surprising number of cases just having dimms in the wrong slots will do it.
  - ggreg84 5y ago
    Virginia Tech Stream, stock, with OMP_NUM_THREADS set to number of cores (these don't have SMT), clang-12 compiler using -O3 -fopenmp -DNDEBUG. > Arm does have an advantage As I said, claiming that the theoretical peak of a CPU can actually achieve theoretical peak of DDR4 is ok as long as that's actually the case. Getting 50-60% just means AMD is lying. Particularly when on other vendors like Amazon and Ampere one gets 90+% of what they advertise with stock compilers and stock flags. Having to use an AMD proprietary compiler, with flags that are bad for programs in general (essentially a STREAM cheat mode), to get 70% of what they advertise, is just sad in 2022.

by sliken 5y ago

ggreg84 5y ago

> This is called "peak" bandwidth, as is a never to exceed number. This is the peak bandwidth of DDR4, saying that would be fine. AMD claims that this is the theoretical peak bandwidth of their CPU, not of DDR4, yet their CPUs memory subsystem are too poor to even get close to this number. > Try apple M1, other ARMs, Alpha, PA-risc, Sparc, Intel, etc and you'll find the same story. I tried the Ampere Altra and Graviton and they achieve ~170-180 GB/s without non-temporal stores. You can get an AWS graviton instance and verify it yourself. That's 90% of "theoretical DDR4 peak" for ARM vs 60% for AMD. So no, i disagree. This is not the case for ALL cpus, this is the case for AMD cpus. And no, 60% vs 90% "out of the box" perf is not the same.

sliken 5y ago
Arm does have an advantage with a looser memory model, which makes it easier to get good bandwidth. Graviton2 and Ampere Altra tend to have more cores than common x86-64 servers (64-128), which helps as well. 90% is more than I'd expect though, can you post the numbers and which code you used? Enough to replicate (compiler, compiler flags, code, etc). I've tracked down lower than expected numbers before, there's many causes. Compilers (temporal vs non-temporal), GCC vs Intel vs Other compilers, surprisingly large differences because of using new or malloc instead of static arrays, having two Dimms per channel (lowering the speed to DDR4-2666), various BIOS settings including a NUMA disable that stripes across all channels that apparently makes some windows codes run better. In a surprising number of cases just having dimms in the wrong slots will do it.
- ggreg84 5y ago
  Virginia Tech Stream, stock, with OMP_NUM_THREADS set to number of cores (these don't have SMT), clang-12 compiler using -O3 -fopenmp -DNDEBUG. > Arm does have an advantage As I said, claiming that the theoretical peak of a CPU can actually achieve theoretical peak of DDR4 is ok as long as that's actually the case. Getting 50-60% just means AMD is lying. Particularly when on other vendors like Amazon and Ampere one gets 90+% of what they advertise with stock compilers and stock flags. Having to use an AMD proprietary compiler, with flags that are bad for programs in general (essentially a STREAM cheat mode), to get 70% of what they advertise, is just sad in 2022.