3 ms·
> It won’t be 100, you are underestimating it to make the number be small Make it a 1000 (I seriously doubt there are one thousand simultaneous training runs o
by M4v3R 1y ago
> It won’t be 100, you are underestimating it to make the number be small
Make it a 1000 (I seriously doubt there are one thousand simultaneous training runs of Mistral Large 2 scale models going on every second) and it's still a drop in the bucket.
> not counting the refresh rate of GPUs (every 3-5 years the whole infrastructure is renewed
I am accounting for this by citing annual usage instead of one-time cost.
- zekrioca 1y agoNot sure what you think demand is, but operators are building 10 GW AI datacenters. Assuming a GPU consumes ~1 kW, the number is potentially (upper bound) 10 GW / 1 kW, way larger than ‘1000’. For one company.
- lostmsu 1y agoStill drop in the bucket considering world total electricity production is about 10 TW. Where did you read one company? I found 10 GW new capacity next year for the entire industry.
- zekrioca 1y agoIt is not a drop in the bucket when we are talking about a factor of 1000000 (and not 100 as your initially calculated), on par with buildings and transportation, only behind agriculture.
- lostmsu 1y agoMe? Also, 100 was the number of trained LLMs. Do you think there will be 1 000 000 trained at the same time at some point?
- zekrioca 1y agoSorry, @OP, not you :) Yes, there will be. But the point is that potentially, all of these GPUs will be at 100% at all times, which makes the 1 000 000 realistic.