4 ms·

My money would be on greater memory bandwidth and probably better use of that bandwidth. If you look at the big graph later on in the article, even for similar

My money would be on greater memory bandwidth and probably better use of that bandwidth. If you look at the big graph later on in the article, even for similar _physical_ core counts the Graviton does better than the Skylake - this implies there's another bottleneck. It's worth pointing out as well that ARM's weaker memory model gives it many more opportunities for aggressive optimisations that can make better use of the available bandwidth.

kllrnohj 6y ago
> even for similar _physical_ core counts the Graviton does better than the Skylake - this implies there's another bottleneck. It doesn't, though, not until Skylake has already hit a bottleneck & graviton2 can eventually "catch up". Skylake caps out at around 12 cores, at which point performance goes flat & even regresses. At 12 cores, though, skylake has a huge lead over a 12 core Graviton2. Similarly, Cascade Lake caps out at around 18 cores. But again at 18 cores the gap between Cascade Lake & Graviton2 is huge. The bottleneck in play here is most likely memory bandwidth. The Graviton2 chip has 8-channel DDR4-3200 memory for a theoretical bandwidth of 204GB/s. That's going to be far more than any single CPU Xeon currently on the market. By comparison the Skylake would be 6 channel DDR4-2666 and Cascade Lake 6 channel DDR4-2933. So for this particular test the interesting comparison would be against Epyc Rome, which is also 8 channel DDR4-3200 and is up to 64 physical cores.
- StillBored 6y ago
  two words, "NUMA binding". The cross socket NUMA bandwidth is like a tiny straw. If the OS or application is failing to pin the memory, cpu and IO together on the correct numa node your going to have a major scalability problem. So, two things can really throw this off, having a Physical->VM topology mismatch, or using a language that is garbage collected and won't allow the programmer to control memory/node placement. I've literally gotten 10x perf improvements by fixing these kinds of problems in the past. Combined with comparing cores to threads and the results are somewhat predictable. In this way, the simpler gravaton topology is really going to shine for applications that don't scale well. That is until intel builds an actual high core count single socket machine.
  - kllrnohj 6y ago
    I was looking at the single-socket results for what I said above specifically to rule out any NUMA differences: https://blog.min.io/content/images/2020/06/single-socket-performance.png https://blog.min.io/content/images/2020/06/single-socket-per... That really looks like it's just sorting by memory bandwidth. Particularly with Cascade Lake's results vs. Skylake at the ~10 core number, since there's not really any other major difference between those two. Being that memory bandwidth constrained, and then not being NUMA aware, is also quite likely why the dual-socket results are a disaster.
  - redis_mlc 6y ago
    > The cross socket NUMA bandwidth is like a tiny straw. And the Internet is a series of tubes!
    - StillBored 6y ago
      How about a ballpark. The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel. Which is why you see postings on the intel forums about people complaining about their perf falling off a cliff if the hardware decides to flag the directory as requiring cross socket snoop (or whatever its actually doing). So even if you have done a reasonable job keeping your thread+data locally, doing a remote reference, or accidentally running the thread for a short while on the wrong socket can permanently lower the performance. See https://software.intel.com/en-us/forums/software-tuning-performance-optimization-platform-monitoring/topic/814772 https://software.intel.com/en-us/forums/software-tuning-perf... for an example about a contented cache line. Thankfully the latest products are mostly 10.4GT (gold and above) as it appears intel isn't doing as much product segmentation based on UPI link speeds anymore (silver/bronze are still slower though). https://www.microway.com/knowledge-center-articles/performance-characteristics-of-common-transports-buses/ https://www.microway.com/knowledge-center-articles/performan...
      - kllrnohj 6y ago
        > The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel Depends on the implementation. Epyc Rome's 2P socket design is 64 PCI-E 4.0 lanes between the two CPUs. That's 128 GB/s bandwidth, which is roughly equivalent to 6-channel DDR4 2666.
- karavelov 6y ago
  > So for this particular test the interesting comparison would be against Epyc Rome, which is also 8 channel DDR4-3200 and is up to 64 physical cores. Some different benchmarks that include the Epyc Rome and Graviton2 instances: https://www.phoronix.com/scan.php?page=article&item=epyc-ec2-c5a&num=1 https://www.phoronix.com/scan.php?page=article&item=epyc-ec2...
  - kllrnohj 6y ago
    Note that AWS seems to only offer up to 48-core Epyc's (96 VCPUs with SMT)

kllrnohj 6y ago

> even for similar _physical_ core counts the Graviton does better than the Skylake - this implies there's another bottleneck. It doesn't, though, not until Skylake has already hit a bottleneck & graviton2 can eventually "catch up". Skylake caps out at around 12 cores, at which point performance goes flat & even regresses. At 12 cores, though, skylake has a huge lead over a 12 core Graviton2. Similarly, Cascade Lake caps out at around 18 cores. But again at 18 cores the gap between Cascade Lake & Graviton2 is huge. The bottleneck in play here is most likely memory bandwidth. The Graviton2 chip has 8-channel DDR4-3200 memory for a theoretical bandwidth of 204GB/s. That's going to be far more than any single CPU Xeon currently on the market. By comparison the Skylake would be 6 channel DDR4-2666 and Cascade Lake 6 channel DDR4-2933. So for this particular test the interesting comparison would be against Epyc Rome, which is also 8 channel DDR4-3200 and is up to 64 physical cores.

StillBored 6y ago
two words, "NUMA binding". The cross socket NUMA bandwidth is like a tiny straw. If the OS or application is failing to pin the memory, cpu and IO together on the correct numa node your going to have a major scalability problem. So, two things can really throw this off, having a Physical->VM topology mismatch, or using a language that is garbage collected and won't allow the programmer to control memory/node placement. I've literally gotten 10x perf improvements by fixing these kinds of problems in the past. Combined with comparing cores to threads and the results are somewhat predictable. In this way, the simpler gravaton topology is really going to shine for applications that don't scale well. That is until intel builds an actual high core count single socket machine.
- kllrnohj 6y ago
  I was looking at the single-socket results for what I said above specifically to rule out any NUMA differences: https://blog.min.io/content/images/2020/06/single-socket-performance.png https://blog.min.io/content/images/2020/06/single-socket-per... That really looks like it's just sorting by memory bandwidth. Particularly with Cascade Lake's results vs. Skylake at the ~10 core number, since there's not really any other major difference between those two. Being that memory bandwidth constrained, and then not being NUMA aware, is also quite likely why the dual-socket results are a disaster.
- redis_mlc 6y ago
  > The cross socket NUMA bandwidth is like a tiny straw. And the Internet is a series of tubes!
  - StillBored 6y ago
    How about a ballpark. The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel. Which is why you see postings on the intel forums about people complaining about their perf falling off a cliff if the hardware decides to flag the directory as requiring cross socket snoop (or whatever its actually doing). So even if you have done a reasonable job keeping your thread+data locally, doing a remote reference, or accidentally running the thread for a short while on the wrong socket can permanently lower the performance. See https://software.intel.com/en-us/forums/software-tuning-performance-optimization-platform-monitoring/topic/814772 https://software.intel.com/en-us/forums/software-tuning-perf... for an example about a contented cache line. Thankfully the latest products are mostly 10.4GT (gold and above) as it appears intel isn't doing as much product segmentation based on UPI link speeds anymore (silver/bronze are still slower though). https://www.microway.com/knowledge-center-articles/performance-characteristics-of-common-transports-buses/ https://www.microway.com/knowledge-center-articles/performan...
    - kllrnohj 6y ago
      > The unidirectional bandwidth across a two socket system is roughly equal to a single DDR channel Depends on the implementation. Epyc Rome's 2P socket design is 64 PCI-E 4.0 lanes between the two CPUs. That's 128 GB/s bandwidth, which is roughly equivalent to 6-channel DDR4 2666.
karavelov 6y ago
> So for this particular test the interesting comparison would be against Epyc Rome, which is also 8 channel DDR4-3200 and is up to 64 physical cores. Some different benchmarks that include the Epyc Rome and Graviton2 instances: https://www.phoronix.com/scan.php?page=article&item=epyc-ec2-c5a&num=1 https://www.phoronix.com/scan.php?page=article&item=epyc-ec2...
- kllrnohj 6y ago
  Note that AWS seems to only offer up to 48-core Epyc's (96 VCPUs with SMT)