3 ms·

> 7,360-GPU cluster would be capable of about 100 .. petaflops How is an A100 worth "only" 17 tera-flops? There are multiple home gpus and even mobile chips r

> 7,360-GPU cluster would be capable of about 100 .. petaflops How is an A100 worth "only" 17 tera-flops? There are multiple home gpus and even mobile chips rated at tera-flops. And the A100 really is a gigantic chip. I realize all about bandwidth, and full vs half precision, and inflated marketing vs actual benchmark. But can a 10-node-iphone-cluster really match an A100 for pure flops?!

_kbh_ 4y ago
The A100 is not optimised for pure ieee-754 FP throughput it seems but for more specialised forms of FP and bandwidth. https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-us-nvidia-1758950-r4-web.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...
nodja 4y ago
> But can a 10-node-iphone-cluster really match an A100 for pure flops?! Only if the data is local, small, easily synchronizable, doesn't require fp32 precision and a dozen other caveats. Can a 10 iphone cluster match an A100 in calculating digits of pi? Yes. Can it do backpropagation in large machine learning models? The gradients won't even fit.
redox99 4y ago
Because its hardware is not optimized for FP32. If you're doing FP32 then a 3090 is literally faster. It's optimized for TF32, INT8, bfloat16, and other similar data types that the tensor cores like.
- make3 4y ago
  the number is for fp64. gpus most definitely are optimized for fp32
tuankiet65 4y ago
That 17 tera flops is in double precision (FP64) math as benchmarked by TOP500.
calaphos 4y ago
The number is for FP64 double precision terraflops. For vector operations, not matrix operations as well. Double precision is important for a lot of simulation codes because of numerical issues. For Deep learning applications it's basically unused, single precision or half precision formats dominate training. The same numerical issues don't show up, so this is a lot more efficient. Additionally a lot of the 'AI' flops are achieved through matrix cores, fitting Deep learning the workload well. This brings an A100 from only 17 to ~200 tflops. > But can a 10-node-iphone-cluster really match an A100 for pure flops?! The iphone number is probably fp32 if not lower precision. However the combined SoCs of 10 iphones have tripple the transistor count and a larger chip area among them. So the comparison is not totally off.

_kbh_ 4y ago

The A100 is not optimised for pure ieee-754 FP throughput it seems but for more specialised forms of FP and bandwidth. https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-us-nvidia-1758950-r4-web.pdf https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...

nodja 4y ago

> But can a 10-node-iphone-cluster really match an A100 for pure flops?! Only if the data is local, small, easily synchronizable, doesn't require fp32 precision and a dozen other caveats. Can a 10 iphone cluster match an A100 in calculating digits of pi? Yes. Can it do backpropagation in large machine learning models? The gradients won't even fit.

redox99 4y ago

Because its hardware is not optimized for FP32. If you're doing FP32 then a 3090 is literally faster. It's optimized for TF32, INT8, bfloat16, and other similar data types that the tensor cores like.

make3 4y ago
the number is for fp64. gpus most definitely are optimized for fp32

tuankiet65 4y ago

That 17 tera flops is in double precision (FP64) math as benchmarked by TOP500.

calaphos 4y ago

The number is for FP64 double precision terraflops. For vector operations, not matrix operations as well. Double precision is important for a lot of simulation codes because of numerical issues. For Deep learning applications it's basically unused, single precision or half precision formats dominate training. The same numerical issues don't show up, so this is a lot more efficient. Additionally a lot of the 'AI' flops are achieved through matrix cores, fitting Deep learning the workload well. This brings an A100 from only 17 to ~200 tflops. > But can a 10-node-iphone-cluster really match an A100 for pure flops?! The iphone number is probably fp32 if not lower precision. However the combined SoCs of 10 iphones have tripple the transistor count and a larger chip area among them. So the comparison is not totally off.