4 ms·

I'm very confused by the talk of "teraflops". Author references linpack (which is a double precision benchmark), but then says "if you really care about teraflo

I'm very confused by the talk of "teraflops". Author references linpack (which is a double precision benchmark), but then says "if you really care about teraflops, you need a graphics card" and installs a 4070 Ti, which does like 600 gigaflops in double (IIRC). This would make more sense if we were talking about single precision (4070 does ~40 TFlops single?); which is it? The original 1.2 teraflop CPU number also smells funny; 128 cores * 2 neon units / core * 2 doubles / neon unit * 2 flops / cycle * 2.8 billion cycles / second = 2.8 TFlops peak, an even halfway decent BLAS will get you to 80% of that, for 2.3 TFlops. Double that number for single precision. Either a completely untuned BLAS or benchmarking a problem that's much too small for the CPU under test.

geerlingguy 3y ago
Caveat: I consider myself a noob at benchmarking, after spending probably 1,000 hours on it over the years. The CPU benchmark used HPL linpack following a standard-ish Top500-style benchmark, mostly because I think it's fun to see how various single CPU systems compare to historic 'supercomputers' on the official lists. There are different ways to calculate (and benchmark) flops, the way I'm doing it is with this open source project: https://github.com/geerlingguy/top500-benchmark https://github.com/geerlingguy/top500-benchmark — which can use OpenBLAS, Blis, or ATLAS, and I've tried all three un-tuned. I also worked with some Ampere engineers to run their optimized version for Ampere Altra Max: https://github.com/AmpereComputing/HPL-on-Ampere-Altra https://github.com/AmpereComputing/HPL-on-Ampere-Altra On their own server systems with 8 channels of memory they topped out around 1.6 Tflops. There are other tests you can run and get more or less Tflops, but I based my own results on the top500 test. See more discussion about the results and testing in the following links: https://github.com/geerlingguy/top500-benchmark/issues/19 https://github.com/geerlingguy/top500-benchmark/issues/19 https://github.com/geerlingguy/top500-benchmark/issues/17 https://github.com/geerlingguy/top500-benchmark/issues/17 https://github.com/geerlingguy/top500-benchmark/issues/10 https://github.com/geerlingguy/top500-benchmark/issues/10 (And see some of the issues linked back to the Ampere repo in those issues.) Edit: And regarding the video card mention—it was meant more as a generic reference (e.g. running A100 or 4090 since you can go much further there than Altra Max), and not specifically to the 4070 Ti... but I can see how that is not as clear!
- dansalvato 3y ago
  Hey Jeff, I loved the video. I know it was more about "current state of the tech" and less about what we should actually be buying, but it would be very cool to hear more about how each of these setups are priced on the scale of "dollar per unit of performance" or something like that. (Or maybe that's not fair to do, since most consumer software can't handle all those cores yet?) I'm also curious whether you think Apple's decisions on memory architecture (despite being non-upgradeable) will have a leg up in the long run. You mentioned that memory bandwidth tops out around 174GB/s. Although you handily beat the Mac Pro in a multicore benchmark thanks to core count, one of the Mac Pro's claims to fame is its memory bandwidth of 800GB/s, as well as its unified memory architecture.
  - geerlingguy 3y ago
    As with all things, it's a tradeoff. HBM on servers is similar to Apple's choice, and Xeon, EPYC, Nvidia H100, and some other designs incorporate it. There are good things (performance) and bad things (price/non-upgradeability) about it. Best of both worlds would he chip on module plus expansion slots, so the fast RAM is like L4 cache.
    - jakobson14 3y ago
      That's essentially how things are likely to go with CXL, though the latency isn't likely to be quite as good as on a dedicated DIMM connection or even as "good" as it was with IBM's OMI. The future (imminent in enterprise and sometime around when PCIe 6.0 hits consumer machines) looks to be mostly a combination of HBM and CXL memory.
- RetroTechie 3y ago
  Hi Jeff, great article! Fast & efficient cpu cores, perhaps slightly memory bandwidth limited, sounds like a great compile/build machine. Any chance of doing a Linux kernel compile, and time that? Bonus points for 2nd, 3rd etc run with much of the stuff in RAM. Curious minds would like to know.

geerlingguy 3y ago

Caveat: I consider myself a noob at benchmarking, after spending probably 1,000 hours on it over the years. The CPU benchmark used HPL linpack following a standard-ish Top500-style benchmark, mostly because I think it's fun to see how various single CPU systems compare to historic 'supercomputers' on the official lists. There are different ways to calculate (and benchmark) flops, the way I'm doing it is with this open source project: https://github.com/geerlingguy/top500-benchmark https://github.com/geerlingguy/top500-benchmark — which can use OpenBLAS, Blis, or ATLAS, and I've tried all three un-tuned. I also worked with some Ampere engineers to run their optimized version for Ampere Altra Max: https://github.com/AmpereComputing/HPL-on-Ampere-Altra https://github.com/AmpereComputing/HPL-on-Ampere-Altra On their own server systems with 8 channels of memory they topped out around 1.6 Tflops. There are other tests you can run and get more or less Tflops, but I based my own results on the top500 test. See more discussion about the results and testing in the following links: https://github.com/geerlingguy/top500-benchmark/issues/19 https://github.com/geerlingguy/top500-benchmark/issues/19 https://github.com/geerlingguy/top500-benchmark/issues/17 https://github.com/geerlingguy/top500-benchmark/issues/17 https://github.com/geerlingguy/top500-benchmark/issues/10 https://github.com/geerlingguy/top500-benchmark/issues/10 (And see some of the issues linked back to the Ampere repo in those issues.) Edit: And regarding the video card mention—it was meant more as a generic reference (e.g. running A100 or 4090 since you can go much further there than Altra Max), and not specifically to the 4070 Ti... but I can see how that is not as clear!

dansalvato 3y ago
Hey Jeff, I loved the video. I know it was more about "current state of the tech" and less about what we should actually be buying, but it would be very cool to hear more about how each of these setups are priced on the scale of "dollar per unit of performance" or something like that. (Or maybe that's not fair to do, since most consumer software can't handle all those cores yet?) I'm also curious whether you think Apple's decisions on memory architecture (despite being non-upgradeable) will have a leg up in the long run. You mentioned that memory bandwidth tops out around 174GB/s. Although you handily beat the Mac Pro in a multicore benchmark thanks to core count, one of the Mac Pro's claims to fame is its memory bandwidth of 800GB/s, as well as its unified memory architecture.
- geerlingguy 3y ago
  As with all things, it's a tradeoff. HBM on servers is similar to Apple's choice, and Xeon, EPYC, Nvidia H100, and some other designs incorporate it. There are good things (performance) and bad things (price/non-upgradeability) about it. Best of both worlds would he chip on module plus expansion slots, so the fast RAM is like L4 cache.
  - jakobson14 3y ago
    That's essentially how things are likely to go with CXL, though the latency isn't likely to be quite as good as on a dedicated DIMM connection or even as "good" as it was with IBM's OMI. The future (imminent in enterprise and sometime around when PCIe 6.0 hits consumer machines) looks to be mostly a combination of HBM and CXL memory.
RetroTechie 3y ago
Hi Jeff, great article! Fast & efficient cpu cores, perhaps slightly memory bandwidth limited, sounds like a great compile/build machine. Any chance of doing a Linux kernel compile, and time that? Bonus points for 2nd, 3rd etc run with much of the stuff in RAM. Curious minds would like to know.