3 ms·
Training language models does not contribute in any meaningful way to global CO2 emissions. Google consumes a vast amount of energy: "10.6 terawatt hours in 20
by joe_the_user 6y ago
Training language models does not contribute in any meaningful way to global CO2 emissions.
Google consumes a vast amount of energy: "10.6 terawatt hours in 2018, up from 2.86 terawatt hours in 2011."[1]
If training and retraining models is a significant and inefficient part of google's energy consumption, the point doesn't seem insignificant(edit: The most advanced AI model involve as much as computing and energy any programs ever created [2], btw). I'm biased by the impression Google's actual search results haven't improved very much but I don't think I'm alone in that impression.
[1] https://www.statista.com/statistics/788540/energy-consumption-of-google/ https://www.statista.com/statistics/788540/energy-consumptio...
[2] https://openai.com/blog/ai-and-compute/ https://openai.com/blog/ai-and-compute/
- juanbyrge 6y agoI obviously don't have the data, but I would argue that a tiny tiny fraction of Google's energy usage is dedicated to training language models. I'm guessing the vast majority of the energy usage is for serving billions of requests for various products, such as search, youtube, maps, gmail, photos, etc.., and the cpu, network, and storage requirements for those requests. Training and retraining ML models is definitely not on the hot path.