4 ms·
GPT3 benchmark, as others have said this means it would be ~46 hours to train GPT, still impressive! Also, going by other comments here, ~3500 of these or 13-14
by layoric 3y ago
GPT3 benchmark, as others have said this means it would be ~46 hours to train GPT, still impressive! Also, going by other comments here, ~3500 of these or 13-14 DGX GH200s at ~$10m each means we are talking about ~$135m worth of compute here. Still very impressive but holy hell that is a lot of money worth of compute hardware.