4 ms·
> 1026 > 1023 Should be 10^26 and 10^23.
by geon 3y ago
> 1026
> 1023
Should be 10^26 and 10^23.
- alchemist1e9 3y agoProbably I did this wrong but I’m getting an approximation of 300K H100s completes that in a month. At least they choose something fairly large it seems. Not sure how LoRA or other incremental training is handled.
- sbierwagen 3y agoDepends on which spec you used, since the law doesn't specify the floating point width. If you used FP8 ops on the H100 SXM then a single GPU would hit the limit in 25265285497.72612 seconds. 300,000 GPUs would pass 10^26 FP8 ops in 23 hours.