3 ms·

> magnitudes faster No...

by qeternity 4y ago

fxtentacle 4y ago
100+ vs <20 TFLOPS
- danaris 4y ago
  ...which is one order of magnitude.
  - xhkkffbf 4y ago
    Not if you use base 2. And sometimes I like to use base 1.00001 for a grand effect.
uniqueuid 4y ago
https://en.wikipedia.org/wiki/FLOPS https://en.wikipedia.org/wiki/FLOPS Intel Rocket Lake FP64: 32 Flops per core per cycle at 8 cores and 5GHz Nvidia Ampere: 1/32 Flops per core per cycle at 2048 cores and ~2 GHz Even though there are more architectural differences, there IS a difference in orders of magnitude ... for the things that you can do on a GPU (SIMD).
- bhouston 4y ago
  This is because FP64 is emulated on the NVIDIA Ampere as it is missing dedicated FP64 hardware. This is not a fair comparison. FP32 is 64x faster on NVIDIA Ampere than FP64 per: https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.pdf https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-... Ampere has 60 TFLOPs of Performance on its top GPU, the Hopper per: https://en.wikipedia.org/wiki/Ampere_(microarchitecture) https://en.wikipedia.org/wiki/Ampere_(microarchitecture). Which is roughly 50% to 100% faster than Rocket Lake depending and that doesn't account for the parallelism/memory benefits that comes with the GPU architecture.
  - hajile 4y ago
    Let's analyze 3 real cores. A100 has 6912 cores (excluding tensor as they don't do 32-bit float math). Each core does 2 (32-bit) floats per cycle and runs at 1410MHz peak frequency. It is 826mm^2 and has an official 300w TDP. This yields 19491.84 GFLOPS. Threadripper (Zen 3) has 64 cores. Each core does 32 (32-bit) floats per cycle and runs at 4300MHz. It has 8 CPU chiplets (80.7mm^2 each) and one IO die (416mm^2) for a total area of 1061.6mm^2. This chip yields 8806.4 GFLOPS. Alder Lake has 8 P-cores and 8 E-cores. If you have an unlocked chip with AVX-512, that yields 64 (32-bit) floats per cycle running at up to 5.3GHz along with 32 (32-bit) floats per cycle running at up to 4GHz. Die area is around 215.25mm^2 (though much of that die area is taken up by the GPU) and it has a TDP of 150w (peak 241w). This chip yields 2713.6 GFLOPS for the P-cores and 1024 GFLOPS for the E-cores for a total of 3737.6 GFLOPS. The first thing of note is that orders of magnitude is a gross overstatement as there isn't even a single order of magnitude of difference between the slowest and fastest systems here. Dividing GFLOPS by die area gives 23.60 GFLOPS/mm^2 for the A100, 8.30 GFLOPS/mm^2 for the Threadripper, and 17.36 GFLOPS?mm&2 for the Alder Lake. Considering the GPU size of the Alder Lake, I suspect that GFLOPS per die area actually favors Alder Lake over the A100 (though this is also somewhat offset by the presence of tensor cores and other non-related stuff in A100). Also noteworthy is that power is a much weirder metric due to how turbos affect all the things. I suspect this is where the A100 has a major advantage not to mention real GFLOPS dropping off steeply in long-running workloads (though this will affect all of these systems to greater or lesser degrees). On the flip side, I should add that branchy code dramatically slows down actual performance on a GPU while only slowing down a CPU when a misprediction happens (in theory this should be only 1-5% of cases). A more interesting question would be Alpha EV9 with it's proposed 1024-bit SIMD unit running at around 2GHz. That chip would have delivered 256 GFLOPS of performance per core around 2005. Scaling to a modest 32-core chip at 4GHz gives a very impressive 16384 GFLOPS while retaining all those CPU advantages too. This leads to the observation that adding large SIMD/vector units to a CPU isn't really hard. A small, in-order core with very wide vector units running at a slower speed to conserve energy seems like an interesting idea and several startups are working in this direction (not to mention this being the basis of Intel's Larabee and later Knight's Corner designs).
    - 4y ago
- qeternity 4y ago
  > here IS a difference in orders of magnitude No, that is a single order of magnitude.

> magnitudes faster No...

fxtentacle 4y ago

100+ vs <20 TFLOPS

danaris 4y ago
...which is one order of magnitude.
- xhkkffbf 4y ago
  Not if you use base 2. And sometimes I like to use base 1.00001 for a grand effect.

uniqueuid 4y ago

https://en.wikipedia.org/wiki/FLOPS https://en.wikipedia.org/wiki/FLOPS Intel Rocket Lake FP64: 32 Flops per core per cycle at 8 cores and 5GHz Nvidia Ampere: 1/32 Flops per core per cycle at 2048 cores and ~2 GHz Even though there are more architectural differences, there IS a difference in orders of magnitude ... for the things that you can do on a GPU (SIMD).

bhouston 4y ago
This is because FP64 is emulated on the NVIDIA Ampere as it is missing dedicated FP64 hardware. This is not a fair comparison. FP32 is 64x faster on NVIDIA Ampere than FP64 per: https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-architecture-whitepaper-v2.pdf https://www.nvidia.com/content/PDF/nvidia-ampere-ga-102-gpu-... Ampere has 60 TFLOPs of Performance on its top GPU, the Hopper per: https://en.wikipedia.org/wiki/Ampere_(microarchitecture) https://en.wikipedia.org/wiki/Ampere_(microarchitecture). Which is roughly 50% to 100% faster than Rocket Lake depending and that doesn't account for the parallelism/memory benefits that comes with the GPU architecture.
- hajile 4y ago
  Let's analyze 3 real cores. A100 has 6912 cores (excluding tensor as they don't do 32-bit float math). Each core does 2 (32-bit) floats per cycle and runs at 1410MHz peak frequency. It is 826mm^2 and has an official 300w TDP. This yields 19491.84 GFLOPS. Threadripper (Zen 3) has 64 cores. Each core does 32 (32-bit) floats per cycle and runs at 4300MHz. It has 8 CPU chiplets (80.7mm^2 each) and one IO die (416mm^2) for a total area of 1061.6mm^2. This chip yields 8806.4 GFLOPS. Alder Lake has 8 P-cores and 8 E-cores. If you have an unlocked chip with AVX-512, that yields 64 (32-bit) floats per cycle running at up to 5.3GHz along with 32 (32-bit) floats per cycle running at up to 4GHz. Die area is around 215.25mm^2 (though much of that die area is taken up by the GPU) and it has a TDP of 150w (peak 241w). This chip yields 2713.6 GFLOPS for the P-cores and 1024 GFLOPS for the E-cores for a total of 3737.6 GFLOPS. The first thing of note is that orders of magnitude is a gross overstatement as there isn't even a single order of magnitude of difference between the slowest and fastest systems here. Dividing GFLOPS by die area gives 23.60 GFLOPS/mm^2 for the A100, 8.30 GFLOPS/mm^2 for the Threadripper, and 17.36 GFLOPS?mm&2 for the Alder Lake. Considering the GPU size of the Alder Lake, I suspect that GFLOPS per die area actually favors Alder Lake over the A100 (though this is also somewhat offset by the presence of tensor cores and other non-related stuff in A100). Also noteworthy is that power is a much weirder metric due to how turbos affect all the things. I suspect this is where the A100 has a major advantage not to mention real GFLOPS dropping off steeply in long-running workloads (though this will affect all of these systems to greater or lesser degrees). On the flip side, I should add that branchy code dramatically slows down actual performance on a GPU while only slowing down a CPU when a misprediction happens (in theory this should be only 1-5% of cases). A more interesting question would be Alpha EV9 with it's proposed 1024-bit SIMD unit running at around 2GHz. That chip would have delivered 256 GFLOPS of performance per core around 2005. Scaling to a modest 32-core chip at 4GHz gives a very impressive 16384 GFLOPS while retaining all those CPU advantages too. This leads to the observation that adding large SIMD/vector units to a CPU isn't really hard. A small, in-order core with very wide vector units running at a slower speed to conserve energy seems like an interesting idea and several startups are working in this direction (not to mention this being the basis of Intel's Larabee and later Knight's Corner designs).
  - 4y ago
qeternity 4y ago
> here IS a difference in orders of magnitude No, that is a single order of magnitude.