3 ms·
H100 SXM is 2000 TFLOPS at FP16. Multiply by 8. M3 Max is 28 TFLOPS at FP16. Based on FLOPS alone, it would be more like a year or two.
by mmoskal 2y ago
H100 SXM is 2000 TFLOPS at FP16. Multiply by 8.
M3 Max is 28 TFLOPS at FP16.
Based on FLOPS alone, it would be more like a year or two.
- Davidzheng 2y agoCan you estimate how long it would take to replicate alphago zero today on one set of 8xH100.
- karpathy 2y ago(H100 SXM is 1000 TFLOPS, *2 is from "with sparsity", which is not used here.)
- mmoskal 2y agoRight... and there are probably some communication overheads over NVLink that would not be present on single laptop. So a few months maybe :)
- latchkey 2y agoMI300x is 1300 TFLOPs at FP16 (without sparsity). Looking forward to seeing the results. https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html https://www.amd.com/en/products/accelerators/instinct/mi300/...