4 ms·
AI training demand should in theory be extremely elastic. Scale up when and where power is cheap, scale down when and where it is expensive. Training larger mod
by A1kmm 2y ago
AI training demand should in theory be extremely elastic. Scale up when and where power is cheap, scale down when and where it is expensive. Training larger models takes a long time, and storing or moving the current training checkpoint is not that expensive.
Inference is somewhat elastic - people want relatively low latency - they might be able to tolerate an extra round-trip around the world, but probably not waiting until a time when there is more total capacity.
However, the big impediment to using cheap and green power is the capital cost of the training hardware; that can't be moved around in a hurry, so its capacity goes unused when the sun isn't shining and the wind isn't blowing. Much of the cost of the high-end data centre oriented GPU hardware is likely not incremental cost for nvidia - it is recovering fixed costs, or profit. In addition, people buying the hardware fear it will depreciate, so they have a limited time frame to use it.
So it is fair to say that it is nvidia's pricing strategy that is a significant driver of Google, Microsoft and Amazon investing in nuclear.
- deleted 2y ago[deleted]
- epistasis 2y agoThe cost of the GPUs massively dwarfs the cost of the electricity. GPUs have a short lifetime of a few years before the next generation makes them mostly obsolete. Therefore it makes sense to use even very expensive electricity to run GPUs. Thus Musk insanely running generators to keep his new DC running. A little money can solve really really bad planning. And these SMRs, if ever built, will be very expensive electricity, a long long time from today. Investing in nuclear has nothing to do with satisfying current AI power needs. This announcement aims to have 5GW running in 15 years. A small amount of energy too far away. When has any tech company planned for energy 15 years away? Or any other specific R&D need 15 years away? Never. These announcements are about vibes and politics. I hope it means that the nuclear industry, for once in its history, actually delivers on its promises and doesn't fail. But the history of nuclear is to fail fail fail at every opportunity, so I'm not optimistic. Also, note the investment size, $500M. Basically nothing. That would buy a DC a bunch of power right now, in the form of solar and the batteries to convert the solar to constant output. Building the 5GW of SMR nuclear is going to cost a minimum of $50B, probably $100B or $150B. It's not going to be competitive with today's prices on solar and batteries. The nuclear purchase is all virtue signalling, not based in actual needs or solutions.