3 ms·

What’s interesting and surprising to me is that the new Epyc 2 chips by AMD have about the same cost per double precision teraflop as a GPU, even the ones with

What’s interesting and surprising to me is that the new Epyc 2 chips by AMD have about the same cost per double precision teraflop as a GPU, even the ones with good floating point hardware support. I assume the cost for these accelerators is probably a little cheaper for high performance computing folk able to buy in bulk, but still I was very surprised. I expected there to be an order of magnitude difference is cost per flop, even with doubles. Once AMD introduces AVX512, the cost per flop should improve even more. Also, in the same vein, I was surprised that double and single floating point cost per flop has stayed fairly stagnant the last couple years as NVIDIA seems focused on improving lower precision performance (ie for machine learning inference).

einpoklum 7y ago
This is probably because you're comparing them to consumer GPUs, which are designed this way - favoring integer and single-precision functional units on the SM cores. That's a sort of a marketing/tiering strategy by nVIDIA. The Teslas have good double-precision performance - but they are priced waaaay higher than the consumer cards.
- Robotbeat 7y ago
  I looked at both. Recent consumer cards have terrible double performance (with exception of Titan series) of course, but the extreme cost of the non-consumer cards offsets their double flop performance to be approximately no better[1] than Epyc 2 per dollar. [1]EDIT: Tesla v100s are still about 40% lower cost per Gflop than Epyc 7742, but still definitely the same order of magnitude, unlike my expectation.
- kllrnohj 7y ago
  > The Teslas have good double-precision performance - but they are priced waaaay higher than the consumer cards. Yup - the Tesla V100 is 7 TFLOPs of double-precision at around ~$9000 A huge split between consumer & HPC happened in the aftermath of the Fermi (2010) architecture. Fermi was really bad in the consumer space from all the wasted die spent on unused double-precision. It was late, hot, and loud. And barely even faster than the competition. With Maxwell Nvidia basically removed all the FP64 support from the architecture itself ( https://www.anandtech.com/show/9059/the-nvidia-geforce-gtx-titan-x-review/2 https://www.anandtech.com/show/9059/the-nvidia-geforce-gtx-t... - native FP64 rate is 1/32'd the FP32 rate) - the result was a huge boost to gaming performance. But it also meant that HPC users who want double-precision had to use Tesla cards. The actual architectures between GeForce & Tesla are different now, it's not "just" a lockout anymore.
  - Robotbeat 7y ago
    Exactly. About 780Gflops/$ for Tesla V100 and 460Gflops/$ (peak) for the Epyc 7742. I was very surprised it was this close. I thought the accelerator would be an order of magnitude cheaper per double Gflop. And AMD isn't using AVX512, yet.
    - jiggawatts 7y ago
      Imagine the very near future when AMD starts using TSCM's 5nm process, which has approximately double the transistor density of the current 7nm process used for the EPYC 7002 series. They could go to DDR5, PCIe 5, AVX 512, and still have a transistor budget left over for whatever they like. The 'whatever' is the interesting part. What exactly does a GPU do that a CPU doesn't? Typical GPUs have crazy high memory bandwidths and good latency hiding by using many (thousands) of threads. So if AMD does something like increase the number of memory channels and implement 4-way SMT, they're poised to upset NVIDIA in the HPC space in a big way. Many people would much rather program for a general-purpose processor than the CUDA platform with all of its quirks and limitations...
      - chapplap 7y ago
        That's not exactly true once you have to deal with vector extensions like AVX-512. It's quite a pain to write by hand (C intrinsics) and many of the ways to abstract it away end up giving you a GPU-like programming model (eg. Intel ISPC). Plus, this has largely been tried before with Xeon Phi and it didn't end so well. Huge vector units like AVX-512 are mainly useful for workloads that need huge amounts of RAM that you just can't get with a GPU, or for workloads that are very latency sensitive and incompatible with GPU task scheduling because they are in some other CPU-bound code.
        jiggawatts 7y ago
        > It's quite a pain to write by hand And we all know that autovectorisation is hit-and-miss at best. I wonder if there will be a new C-like language that has portable SIMD-like capabilities in the same sense that "C is a portable assembly language".

einpoklum 7y ago

This is probably because you're comparing them to consumer GPUs, which are designed this way - favoring integer and single-precision functional units on the SM cores. That's a sort of a marketing/tiering strategy by nVIDIA. The Teslas have good double-precision performance - but they are priced waaaay higher than the consumer cards.

Robotbeat 7y ago
I looked at both. Recent consumer cards have terrible double performance (with exception of Titan series) of course, but the extreme cost of the non-consumer cards offsets their double flop performance to be approximately no better[1] than Epyc 2 per dollar. [1]EDIT: Tesla v100s are still about 40% lower cost per Gflop than Epyc 7742, but still definitely the same order of magnitude, unlike my expectation.
kllrnohj 7y ago
> The Teslas have good double-precision performance - but they are priced waaaay higher than the consumer cards. Yup - the Tesla V100 is 7 TFLOPs of double-precision at around ~$9000 A huge split between consumer & HPC happened in the aftermath of the Fermi (2010) architecture. Fermi was really bad in the consumer space from all the wasted die spent on unused double-precision. It was late, hot, and loud. And barely even faster than the competition. With Maxwell Nvidia basically removed all the FP64 support from the architecture itself ( https://www.anandtech.com/show/9059/the-nvidia-geforce-gtx-titan-x-review/2 https://www.anandtech.com/show/9059/the-nvidia-geforce-gtx-t... - native FP64 rate is 1/32'd the FP32 rate) - the result was a huge boost to gaming performance. But it also meant that HPC users who want double-precision had to use Tesla cards. The actual architectures between GeForce & Tesla are different now, it's not "just" a lockout anymore.
- Robotbeat 7y ago
  Exactly. About 780Gflops/$ for Tesla V100 and 460Gflops/$ (peak) for the Epyc 7742. I was very surprised it was this close. I thought the accelerator would be an order of magnitude cheaper per double Gflop. And AMD isn't using AVX512, yet.
  - jiggawatts 7y ago
    Imagine the very near future when AMD starts using TSCM's 5nm process, which has approximately double the transistor density of the current 7nm process used for the EPYC 7002 series. They could go to DDR5, PCIe 5, AVX 512, and still have a transistor budget left over for whatever they like. The 'whatever' is the interesting part. What exactly does a GPU do that a CPU doesn't? Typical GPUs have crazy high memory bandwidths and good latency hiding by using many (thousands) of threads. So if AMD does something like increase the number of memory channels and implement 4-way SMT, they're poised to upset NVIDIA in the HPC space in a big way. Many people would much rather program for a general-purpose processor than the CUDA platform with all of its quirks and limitations...
    - chapplap 7y ago
      That's not exactly true once you have to deal with vector extensions like AVX-512. It's quite a pain to write by hand (C intrinsics) and many of the ways to abstract it away end up giving you a GPU-like programming model (eg. Intel ISPC). Plus, this has largely been tried before with Xeon Phi and it didn't end so well. Huge vector units like AVX-512 are mainly useful for workloads that need huge amounts of RAM that you just can't get with a GPU, or for workloads that are very latency sensitive and incompatible with GPU task scheduling because they are in some other CPU-bound code.
      - jiggawatts 7y ago
        > It's quite a pain to write by hand And we all know that autovectorisation is hit-and-miss at best. I wonder if there will be a new C-like language that has portable SIMD-like capabilities in the same sense that "C is a portable assembly language".