3 ms·
No, I already multiplied 624 TFLOPS / GPU * 8 GPU = 4992 TFLOPS (the 5 petaflops number). I'm saying that you are still missing another 8x on the way from 19.5
by sabalaba 6y ago
No, I already multiplied 624 TFLOPS / GPU * 8 GPU = 4992 TFLOPS (the 5 petaflops number).
I'm saying that you are still missing another 8x on the way from 19.5 TFLOPS / GPU to 624 TFLOPS / GPU. 19.5 (base FP32 theoretical peak performance) * 2 (FP16 instead of FP32) * 2 (counting sparse matrix ops as dense ops) * 8 (unknown) = 624 TFLOPS.
- rrss 6y agoFP16 tensorcore = 312 tflops x 2 (counting sparse as dense) = 624 tflops x 8 GPUs = 5 "pflops" The missing 8x you are looking for is just because tensorcore math is much faster than their normal fma path.