3 ms·
Doing calculations like these is a really bad idea because they muddy the waters, putting the focus on disputable numbers - unless you're willing to do a proper
by maeil 2y ago
Doing calculations like these is a really bad idea because they muddy the waters, putting the focus on disputable numbers - unless you're willing to do a proper study and really dive into it. It's extremely tiring seeing all these well-meaning people, including possibly this author (unless they're writing this for clout), making this mistake time and time again. You're actively not helping.
There's a much better heuristic. Just look at the contracts that the model providers (MS, Google et al) have recently, suddenly established to buy lots of electricity, spin up nuclear power plants and so on. That's the giveaway and one that doesn't need any calculations. Plus the hundreds of billions being invested in datacentres purely for these models.
You see it in this very comment section, lots of naysayers purely based off the provided, utterly meaningless numbers, when the above facts say it all and renders everything else moot. If the training and inference wouldn't cost insane amounts of energy, they wouldn't be hastily taking up these contracts and investing such obscene amounts in datacentres. That's all you need to know.
- ARandumGuy 2y agoWe also need to consider that AI is being used as a replacement for things that use a lot less energy. Like, someone "asking ChatGPT" for a simple question is using a lot more energy then using a traditional search engine. This gets worse when AI is shoehorned into random stuff, thus making that stuff less efficient.
- causal 2y agoThe calculations are also just wrong. The author basically makes up the 2W figure, doesn't take batching into account, has bogus DAU numbers. Energy use is worth discussing, but this is worse than just guesswork.
- datadrivenangel 2y agoAlso a lot of the datacentres are... just datacentres... AI is a small part of cloud
- hmmm-i-wonder 2y agoCompletely agree. There are so many additional layers of electrical costs (and other costs) not factored in The build/infrastructure costs. Especially the early models before the A100 and datacenters jumped on the bandwagon. Training costs of the model being used Training costs of failed models and tests Training costs of previous models that have been superseded before their cost could be recaptured from users. The actual energy used by AI is likely orders of magnitude higher than estimated here, but without really justifying all the numbers used and sourcing them its going to be a never ending argument between the pro and anti-AI crowed with the realists stuck in between trying to read the data.