4 ms·

So the 4090 is 38% faster (in FP32 FLOPS) than the current top server GPU (H100) and 105% faster than the current top desktop GPU (3090 Ti). And it's also more

by mrb 4y ago

So the 4090 is 38% faster (in FP32 FLOPS) than the current top server GPU (H100) and 105% faster than the current top desktop GPU (3090 Ti). And it's also more than twice as efficient (in FLOPS per watt) as all current top GPUs, even compared to the H100 which is manufactured on the same TSMC 4N process. This is impressive. The computing power of these new GPUs (in FP32 TFLOPS) is as below: Nvidia RTX 4090: 82.6 TFLOPS (450 watts¹) Nvidia RTX 4080-16GB: 48.8 TFLOPS (320 watts¹) Nvidia RTX 4080-12GB: 40.1 TFLOPS (285 watts¹) Compared to Nvidia's current top server and top desktop GPUs: Nvidia H100 (SXM card): 60.1 TFLOPS (700 watts) Nvidia RTX 3090 Ti: 40.0 TFLOPS (450 watts) Compared to AMD's current top server and top desktop GPUs: AMD Instinct MI250X: 47.9 TFLOPS (560 watts) AMD Radeon RX 6950 XT: 23.7 TFLOPS (335 watts) ¹ I didn't see the wattage listed in this page by Nvidia; my source is https://www.digitaltrends.com/computing/nvidia-rtx-4090-rtx-4080-launch-geforce-beyond/ https://www.digitaltrends.com/computing/nvidia-rtx-4090-rtx-...

xbar 4y ago
It's there: https://www.nvidia.com/en-us/geforce/graphics-cards/compare/ https://www.nvidia.com/en-us/geforce/graphics-cards/compare/
adrian_b 4y ago
The FP32 comparisons show only a very incomplete image about the RTX 40 series vs. server GPUs. Both the NVIDIA and the AMD server GPUs are about 30 times faster at FP64 operations. At equal FP32 Tflops, the server GPUs would be 32 times faster at FP64 operations. A 16 factor comes from having an 1:2 FP64/FP32 speed ratio instead of an 1:32 ratio, and an additional factor of 2 comes because the server GPUs have FP64 matrix operations, which do not exist in consumer GPUs. So the difference in FP64 speed is really huge. Moreover, the server GPUs have ECC memory. When you have a computation that takes a week, you would not want it to provide wrong results, forcing you to redo it.
- immmmmm 4y ago
  are they still using the same chip in server and gaming ranges and crippling f64 by a fuse or software?
  - terafo 4y ago
    No, they aren't even on the same architectures now.
- mrb 4y ago
  Of course, server GPUs remain kings of the hill on FP64 workloads, as they have extra FP64 units. I focus on FP32 performance because it is representative of general compute performance on all other workloads, such as FP32, FP16, FP8, or integer.
throwabro1000 4y ago
The more significant changes have not been detailed. - What will MIG look like on these GPUs? - Are there multiple graphics queues now? Otherwise, the RAM remains the greatest limitation of NVIDIA’s platform. They should really design a RAM add in standard and decouple it from the chip. IMO NVIDIA’s greatest competition is against Apple, who have achieved a unified memory architecture and a naturally multi queue (actually parallel) graphics and compute API. Everyone from normals to the most turbo “ML boys” do not comprehend yet how big of a deal the M1 GPU architecture is.

by mrb 4y ago

xbar 4y ago

It's there: https://www.nvidia.com/en-us/geforce/graphics-cards/compare/ https://www.nvidia.com/en-us/geforce/graphics-cards/compare/

adrian_b 4y ago

The FP32 comparisons show only a very incomplete image about the RTX 40 series vs. server GPUs. Both the NVIDIA and the AMD server GPUs are about 30 times faster at FP64 operations. At equal FP32 Tflops, the server GPUs would be 32 times faster at FP64 operations. A 16 factor comes from having an 1:2 FP64/FP32 speed ratio instead of an 1:32 ratio, and an additional factor of 2 comes because the server GPUs have FP64 matrix operations, which do not exist in consumer GPUs. So the difference in FP64 speed is really huge. Moreover, the server GPUs have ECC memory. When you have a computation that takes a week, you would not want it to provide wrong results, forcing you to redo it.

immmmmm 4y ago
are they still using the same chip in server and gaming ranges and crippling f64 by a fuse or software?
- terafo 4y ago
  No, they aren't even on the same architectures now.
mrb 4y ago
Of course, server GPUs remain kings of the hill on FP64 workloads, as they have extra FP64 units. I focus on FP32 performance because it is representative of general compute performance on all other workloads, such as FP32, FP16, FP8, or integer.

throwabro1000 4y ago

The more significant changes have not been detailed. - What will MIG look like on these GPUs? - Are there multiple graphics queues now? Otherwise, the RAM remains the greatest limitation of NVIDIA’s platform. They should really design a RAM add in standard and decouple it from the chip. IMO NVIDIA’s greatest competition is against Apple, who have achieved a unified memory architecture and a naturally multi queue (actually parallel) graphics and compute API. Everyone from normals to the most turbo “ML boys” do not comprehend yet how big of a deal the M1 GPU architecture is.