9 ms·
How much electricity does AI consume?
- Fripplebubby 3y agoAs I read this article I felt myself being extremely critical of it. Part of this may be that I'm enamored with AI and so I feel defensive about it, but I can't help the feeling that this article needed more work. For example: > Moreover, the organizations best placed to produce a bill — companies like Meta, Microsoft, and OpenAI — simply aren’t sharing the relevant information. To me this shows both unfamiliarity with large corporate structures and unfamiliarity with AI research. The former because, if you want an exact accounting at a company where there are several teams running dozens or hundreds of models, it becomes someone's job essentially to compile this information because it takes a lot of work. So, you are surprised that a corporation doesn't outlay $200k/year or more getting these figures to decorate your article with? The corporations do know, on a month-to-month basis, what they're spending on this stuff - since they settle the invoices. But doing the work to get these figures into a simple, digestible form is a lot of effort, and I think that quite frankly, that effort should fall on the journalist, since it's the journalist who gets the benefit from having those numbers... As to unfamiliarity with AI research, many papers I have been reading lately are very interested in measuring and minimizing the compute cost of models, and they often compare different methods and are often extremely precise about their training process and equipment. I feel like this article wants to sell the story that these faceless corpos don't care about energy consumption, but the researchers definitely do. Granted certain specific products / models such as ChatGPT 4 do not disclose exactly their process, but I feel like it would not be difficult to come up with a good estimate using similar models (mostly documented right out in the open in scientific papers). Tracking the energy consumption of AI is an important and emerging issue, but this article feels too partisan to be useful.
- zx8080 3y agoIs parent comment AI generated?
- smartmic 3y agoAt the very least, the article has brought to the forefront the awareness that, in addition to all the other effects of the hype, the research and use of AI on a large scale also consumes an extreme amount of energy. Since we should all be concerned about protecting our resources and thus the planet - and that means reducing unnecessary energy consumption - this figure should be clearly included in the general costs of AI. And by that I don't mean in the electricity bills of corporations. Of course, it is in the interest of researchers to increase the efficiency of modelling and application, but for other reasons. As long as the big, well-known companies are in charge, the target figure towards which everything is optimised and maximised is commercial profit. To cut a long story short, the only way to gain some ground here is through independent regulation, for example to get more transparency into this issue.
- sdenton4 3y agoTo be sure, we only care about electricity usage because of externalities, most importantly CO2 emissions. Many of the big players have gone to great lengths to build out renewables capacity to power their data centers, which tends not to be considered in these articles.
- Gormo 3y ago> Since we should all be concerned about protecting our resources and thus the planet To my knowledge, most of the researchers and users of AI are using their own resources, not "ours", so I'm not sure there's much to be worried about. > And by that I don't mean in the electricity bills of corporations. What else would it be accounted for in? The users of electricity are purchasing it from the producers, who in turn purchase equipment and services from vendors, etc. It's all a chain of specific transactions among specific parties all the way down. There's no point at which some fuzzy collection of arbitrarily aggregated people is involved as one of the parties. > As long as the big, well-known companies are in charge, the target figure towards which everything is optimised and maximised is commercial profit. And they make that profit by delivering value to their customers -- what's the problem there? > To cut a long story short, the only way to gain some ground here is through independent regulation, for example to get more transparency into this issue. What is "independent regulation"? Who is conducting it, what makes them "independent" and what is their incentive to be involved in the first place?
- mepiethree 3y agoI work in the sustainability industry so this is obviously important to me, but I agree the article needed more work. Comparing the energy required to train GPT3 to watching Netflix is nonsensical. Training is a global task. A better comparison would be the cost to train GPT vs that to upload all Netflix movies. Plus, there are weird things with the figures. It lists the 0.012kwh figure required to charge a smartphone, but the source it cites for that number explicitly says that the correct number is 0.022kwh. Was this article hallucinated by AI? Overall though, frankly I found it comforting that AI would “only” use 0.5% of global electricity, as I expected much higher. Google and several other peers are committed to being 100% renewable anyway. Still, credit to the author for starting discussion about this.
- Fripplebubby 3y agoYes, I ultimately I guess it does not hurt anything to have this article in circulation even if I have some critiques, better to discuss the elephant in the room.
- lynndotpy 3y ago> As to unfamiliarity with AI research, many papers I have been reading lately are very interested in measuring and minimizing the compute cost of models, and they often compare different methods and are often extremely precise about their training process and equipment It's worth knowing this is a very recent phenomena. Before ~2020, it was extremely rare to see any such paper, with most of the focus in publishing being on maximizing metrics like accuracy. The biggest force pushing people toward smaller models were researchers at smaller institutions who lacked access to big GPU clusters. > So, you are surprised that a corporation doesn't outlay $200k/year or more getting these figures to decorate your article with? I think it's incorrect to suggest these figures would only serve to make for a better "The Verge" article. One of the externalized costs here is climate change.
- Fripplebubby 3y agoFair point about recency. Actually my point is that to specifically isolate costs that are from AI is a lot of work at a company of this scale. These companies are absolutely accountable for their overall energy usage and externalities!
- loeg 3y agoYeah. Meta has internally widely shared the expected size of a new very large cluster they are building in 2024 and from the number of H100s or whatever you can probably calculate a reasonable low estimate on power draw (there will be other machines to support the compute, e.g., storage). I don't know that it's public yet so I won't say the exact size but I expect it to be leaked or spoken about publicly shortly anyway. It's a lot of MW. They are semi-joking talking about building nuclear reactors to power future DCs.
- rossdavidh 3y agoIt is difficult to say for sure, but one thing is definite, and that is that it consumes more than it is worth, at this point anyway.
- x86x87 3y agoMore than the entire country of Argentina?
- infecto 3y agoNot definite at all. That is only your world view.
- monkaiju 3y agoSeems like a pretty worldview to say generating garbage text and 5 legged horse images isn't worth a measurable amount of our electricity, especially in a time where we so clearly need to be reducing consumption.
- infecto 3y agoPerhaps your own view but obviously not everyone's. I am not a doomer and don't live in the same doomer vision as your own, sorry. I am excited for the future and part of that future is growing energy consumption which will fuel new innovations.
- INTPenis 3y agoI was just thinking a few weeks ago that there is a risk LLMs will bring on a wave of big computer builds similar to crypto mining. For storing all the data, and for calculating all the queries from their users. I'm not familiar with what is required but I do believe that the larger the dataset the better it is, and that specialized GPUs have already been released.
- Gormo 3y agoIt's odd to characterize that as a "risk". It seems to follow that increases in demand for computing use cases would in turn generate more demand for the hardware required to implement them.
- INTPenis 3y agoThe article was about electricity consumption, so in the context of environmental impact, or consumer market impact, it is a risk.
- Gormo 3y agoConsumption of electricity has negligible environmental impact. The relevant discussion here would pertain to considerations around generating power, regardless of what downstream use cases the power is being applied to. Pretty much all activity in modern society is going to consume electricity, and overall demand is not going to be decreasing in the first place, so it seems a bit silly to look at this from the demand side: we're always going to need more and more power, and the focus is properly on how to generate power in a clean and scalable way regardless of what it's being used for. Consumer market impact is an interesting topic, though, if there are massive spikes in demand that could drive prices up for other users of electricity. It will be important to ensure to minimize artificial impediments to the expansion of the supply side to mitigate that risk.
- verdverm 3y agoMore data & compute still produces noticeably better results, and probably always will (?) The difference is that the AI field is - rapidly producing better algos and hardware, L2 scaling in crypto can be seen similarly, but nowhere near the same pace of innovation - AI is producing business & consumer value now, there is a sustainable business model, crypto still seems like a separated economic system
- verdverm 3y agoFrom looking at a variety of sources, my best, rough understanding is that 1. Data centers & Bitcoin are using about the same amount of electricity, about 2% of the US energy production each 2. AI is a subset of data center usage, but expect this portion to drastically rise and drive up the energy usage 3. Data centers & AI are continuingly becoming more efficient while Bitcoin becomes less so by design, L2 crypto is an efficiency play for crypto more generally
- x86x87 3y agoNot sure where bitcoin got dragged into the conversation. Also not sure how BTC becomes less efficient by design. The hardware BTC is mined on has improved exponentially over time moving from general purpose cpus to gpus to custom asics.
- ta988 3y agoBut what's the relation between that efficiency and the number of hashes computed? My understanding was that difficulty is increasing with efficiency so you end up having to try harder to get a block mined successfully.
- x86x87 3y agoThe difficulty gets periodically adjusted to generate a block roughly every 10 minutes. It's not really about efficiency as it's about computing power and available miners. If it's not economically beneficial to mine (costs more than reward) miners will stop mining leading to the difficulty decreasing which in turns makes it profitable to mine again.
- serf 3y ago>Also not sure how BTC becomes less efficient by design. one of the major key principles behind bitcoin is that the hashing becomes more difficult over time. hardware can't constantly keep parity with difficulty. [0]: https://www.blockchain.com/explorer/charts/difficulty https://www.blockchain.com/explorer/charts/difficulty
- jdmoreira 3y agoThis is day one. This is the least efficiency it will ever be. All the software optimisations are still ahead of us and even hardware will get more optimised. Computers slower than my pencil arithmetic used to run on valves and now here I am typing for half a day and blazing on the internet in my cordless, battery powered MacBook Air.
- x86x87 3y agoOr another way to look at it: this is day one. Usage will just continue to growth.
- jdmoreira 3y agoYes, of course but not per unit of output. It will become more efficient and more widespread. I don't really see a long-term problem in that. Maybe a short-term problem yes
- monkaiju 3y agoAssuming it doesnt crash because people realize its not a good tool (which I'm hoping for), this still means its overall energy consumption will rise. Thats Jevons paradox and this scenario fits it quite well
- surajrmal 3y agoYou're living under a rock if you think this is a fad. You're seeing ai used all around you constantly. Autocorrect on your phone keyboard, suggestions in Google News, YouTube, etc, translation, speech to text captioning, voice navigation, etc. These are all useful applications in the consumer space, not to mention all of the industrial and medical applications such as helping diagnose health issues, fraud/anomaly detection, etc. I would say there is a bit of stick on the wall happening where everyone tries to apply it everywhere and it's generally not always helpful. I consider that more of an evolutionary necessity. Hopefully we will learn where it works best and hone out usage in those applications.
- rvz 3y agoYet AI still cannot even find a sustainable alternative to their inferencing, training and fine-tuning processes to combat their mass consumption of electricity and water for years. At least, cryptocurrencies have managed to combat this criticism with alternatives to proof-of-work and even Ethereum made it possible for a wasteful PoW blockchain to migrate to proof-of-stake which is an energy efficient alternative consensus mechanism [0][1] and have reduced their consumption by 99%. The field of Deep Learning has made little efficient alternatives with any measurable impact and have always needed tons of GPUs in data-centers and the demand is made even worse with generative AI whilst continuing to green-wash the public with faux green proposals for years. Not much progress in these so-called practical alternatives to this waste that AI has produced or any reduction of energy usage. As it data and model scales, the energy consumption and costs will only just get worse even by 2027. [0] https://digiconomist.net/ethereum-energy-consumption https://digiconomist.net/ethereum-energy-consumption [1] https://www.cell.com/patterns/fulltext/S2666-3899(22)00265-3 https://www.cell.com/patterns/fulltext/S2666-3899(22)00265-3
- deleted 3y ago[deleted]
- CollinEMac 3y agoFrom the article: “The tl;dr is we just don’t know.”
- aaron695 3y ago[dead]
- megaman821 3y agoAt numbers this small on the global scale, is this something that merits mass concern? So there is more energy in a tanker truck full of gasoline than it takes to train an AI model. And a person uses more energy to go to the grocery store than then will reasonably use generating things with AI all week.
- bonton89 3y agoA common HN bugbear is all the energy crypto wastes so it seems reasonable that AI energy usage would be of interest as well. This is the part where people say Crypto is only for fraud, scams and just making money so it is different. I think AI will open all new avenues of fraud that will make the ransomware mess look pleasant in comparison. AI may destroy the very concepts of truth and trust even for those looking for it, and it ALSO will waste lots of energy doing it.
- andybak 3y agoAI will be a mixed bag. Crypto is probably a net negative. You obviously disagree but opening with "A common HN bugbear is all the energy crypto wastes" is no more disingenuous than my reply.
- jijijijij 3y ago> AI may destroy the very concepts of truth and trust Oh, hell nah! Dark visions: AI's mere application will be solving problems AI created in the first place - and, incidentally, the erosion of trust brought up a use case for crypto, at last. I will be in my hut eating moss.
- myaccountonhn 3y ago1% is huuuuuuuuge on the global scale. Especially since it’s questionable how useful it is. Are we really allocating our limited resources well?
- prpl 3y ago700W per customer dedicated for Netflix seems incredibly high - basically saying a 1u is dedicated per customer (no TV power consumption) or half of a 1U factoring in a 52” TV (admittedly I’m omitting network power costs, transcoding, etc…) Anyway, given previous examples of the netflix arch, I’d expect most of the cost of streaming is mostly TLS session management.
- 1-6 3y agoWith today’s AI, we’re taking a big hammer to the problem using unoptimized but vastly flexible machines like GPUs. Once the code settles down, expect ASICs to run much of the show and lower energy consumption.
- mwhitfield 3y agoYou're maybe not wrong, but I'm pretty sure I was reading this same comment 10 years ago, when we were just calling it deep learning.
- bongodongobob 3y agoYeah and it actually works now. It's not just weird dog faces hallucinated with Starry Night.
- quonn 3y agoThis is really unlikely, because the operations are pretty basic math and the GPUs do these about as efficiently as possible. Surely those GPUs will be improved further, but a custom ASIC probably won‘t help much with training. And NVIDIA has been producing GPUs that are specifically targeted at deep learning and they control both the software library and the hardware. I don‘t quite see how they couldn‘t optimize whatever they want to optimize.
- xadhominemx 3y agoThe chips won't be that much more efficient on a like for like basis but the models will be much smaller so the chips can be smaller or run much larger batches
- pixl97 3y agoBeing that us meatbags operate on a pretty limited power budget I'm guessing there is still a lot of algorithmic optimization to be done, and those algorithms will need hardware to run on.
- londons_explore 3y ago
- pxeger1 3y agoOnly 130 US homes’ annual power consumption to train GPT-3 is surprisingly small to me, considering it only needs to be done once.
- moffkalast 3y agoTbf, GPT-3 was made before the Chinchilla paper and was only trained on 0.3T tokens which is basically nothing for its size or for any current model (Mistral 7B, a 25x smaller model was trained on 8T). Doing it properly would require much more power.
- morphle 3y agoWe don't know exactly how much electricity the AI/ML part of global energy use is, but it is being counted as part of the global datacenter energy use. The estimates are between 1% and 5% worldwide depending on the used definition AI/ML and the definition of world energy use (for example global electricity versus global energy). [1] Integrative design for radical energy efficiency - Amory Lovins https://youtu.be/Na3qhrMHWuY?t=1026 https://youtu.be/Na3qhrMHWuY?t=1026 [2] "Energy. Estimated global data centre electricity consumption in 2022 was 240-340 TWh, or around 1-1.3% of global final electricity demand. This excludes energy used for cryptocurrency mining, which was estimated to be around 110 TWh in 2022, accounting for 0.4% of annual global electricity demand." https://www.iea.org/energy-system/buildings/data-centres-and-data-transmission-networks# https://www.iea.org/energy-system/buildings/data-centres-and... [3] Stanford Seminar - Saving energy and increasing density in information processing using photonics - David B. Miller https://www.youtube.com/watch?v=7hWWyuesmhs https://www.youtube.com/watch?v=7hWWyuesmhs
- hristov 3y agoYou have a citation for 2022 which is right before use of AI exploded and NVIDIA started selling AI machines like hotcakes. I expect energy use to be much larger this year.
- morphle 3y agoI'm sure you are right but the numbers for 2023 will be published next year. We can also estimate [2] and extrapolate from the recent shift of 30% (?) of bitcoin mining from China to the US [1]. ARPA-E funded the tabulation of all US energy uses in Sankey Diagrams, which included datacenter use [2] but also does not split out NVDIA's, Bitcoin mining or AI's parts. [1] https://www.nytimes.com/2023/10/13/us/bitcoin-mines-china-united-states.html https://www.nytimes.com/2023/10/13/us/bitcoin-mines-china-un... [2] Energy, Following the Numbers - Saul Griffith Stanford talk https://www.youtube.com/watch?v=1ewEaTlGz4s https://www.youtube.com/watch?v=1ewEaTlGz4s
- mmoskal 3y agoFWIW a recent The Economist report [0] lists data center usage at 1-2% of global electricity consumption (which is 1/3 of energy IIRC) and AI at 8% of that in 2023 projected to grow to 15-20% by 2028. Given how careful the hyper scalers are about using renewables and offsets (edit: compared to other big energy users that is) I refuse to feel bad about it... [0] https://www.economist.com/technology-quarterly/2024/01/29/da https://www.economist.com/technology-quarterly/2024/01/29/da...
- Havoc 3y ago>a single Balenciaga pope I approve of this unit of measure
- verticalscaler 3y agoWhat if instead of all the gnashing of teeth about the "co2 output of AI" we let it go wild and have it solve fusion for us.
- bgnn 3y agoThey don't even consider the energy consumed to produce the chips AI/ML is running on at all. The biggest chip producer TSMC ysed 22000 gigawatt hour last year. This doesn't include oder foundries, memory etc.. AI is now morebthan 15% of TSMC revenue. AI/GPU chips ate often in advanced nodes consuming orders of magnitude more energy than older nodes. So one can assume energy consumption share is much more than 15% of that 22000 gigawatt hour. Let's say 25%. That would be 5500 gigawatt hour. That's like 0.15% of whole US energy consumption of 2020. So 1-2% during operation is totally fathomable.
- littlestymaar 3y agoDoes TSMC produce the pure silicon themselves from silica or is the refinery done by a supplier? I suspect it's the latter, but idk. It's a very energy intensive process, and huge source of CO2 emissions as well because you use coal to provide carbon in order to reduce the silica and get silicon + CO2.
- bgnn 3y agoThey get raw wafers from other suppliers AFAIK. It comes pre-doped even.
- Engineering-MD 3y agoRealistically the hidden costs of AI are huge. Microchips are the very peak end of manufacturing complexity and require an entire global supply chain in order to produce. While there are other uses for microchips, automation is the ultimate use, and AI is the current pursuit. I would argue you could consider the entire global tech economy as being part of the cost of AI. The real costs are huge.
- pixl97 3y agoSo, let me counter with, what is the cost of efficiency? For example the 'entire tech economy' produces more efficient engines, devices that save on fertilizer planting crops, machines that sequence genes, etc. Attempting to handwave AI as a cost of this, unless you can otherwise explain, seems like you're saying "Climbing down from the trees was a mistake"
- Ferret7446 3y agoThe question no one wants to ask; how much resources is consumed by the people whose economic output would be replaced by AI?
- defrost 3y agoDon't stop there; * how many resources will they still consume after being replaced by AI, * is AI just more resource consumption on top, * will the people replaced be deported to "elsewhere", converted to Soylent Green, or supported by UBI funded by an AI tax? Never go ankle deep in these questions.
- danjc 3y agoAnyone else think that the Netflix estimate of 0.8kWh seems far too high?