3 ms·
Ah so I couldn't just run this on my laptop for ~48 hours? That's too bad.
by jamestimmins 2y ago
Ah so I couldn't just run this on my laptop for ~48 hours? That's too bad.
- danielmarkbruce 2y agoHe does it on 8 H100's in 24 hours, ie 192 H100 hours. It's going to be thousands of laptop hours.
- deleted 2y ago[deleted]
- mmoskal 2y agoH100 SXM is 2000 TFLOPS at FP16. Multiply by 8. M3 Max is 28 TFLOPS at FP16. Based on FLOPS alone, it would be more like a year or two.
- Davidzheng 2y agoCan you estimate how long it would take to replicate alphago zero today on one set of 8xH100.
- karpathy 2y ago(H100 SXM is 1000 TFLOPS, *2 is from "with sparsity", which is not used here.)
- mmoskal 2y agoRight... and there are probably some communication overheads over NVLink that would not be present on single laptop. So a few months maybe :)
- latchkey 2y agoMI300x is 1300 TFLOPs at FP16 (without sparsity). Looking forward to seeing the results. https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html https://www.amd.com/en/products/accelerators/instinct/mi300/...