15 ms·

AI's energy footprint

312 points by pseudolus 1y ago

mentalgear 1y ago
When companies make ESG claims, sensible measurement and open traceability should always be the first proof they must provide. Without these, and validation from a credible independent entity such as a non-profit or government agency, all ESG claims from companies are merely PR puff pieces to keep the public at bay (especially in "AI").
- stevage 1y ago
  esg?
  - JohnFen 1y ago
    Environmental/Social/Governance. From Wikipedia: Environmental, social, and governance (ESG) is shorthand for an investing principle that prioritizes environmental issues, social issues, and corporate governance.
- kkarakk 1y ago
  when has ESG not been FUD or a way to bypass sanctions from poorly thought out climate change targets?
mg 1y ago
The brain uses 20% of the human body's energy. I wouldn't be surprised if mankind will evolve similar to an organism and use 20% of all energy it produces on AI. Which is about 10x of what we use for software at the moment. But then more AI also means more physical activity. When robots drive cars, we will have more cars driving around. When robots build houses, we will have more houses being built, etc. So energy usage will probably go up exponentially. At the moment, the sun sends more energy to earth in an hour than humans use in a year. So the sun alone will be able to power this for the foreseeable future.
- amelius 1y ago
  One problem: all this energy is eventually turned into heat ...
  - mg 1y ago
    Most of the sunlight that hits a roof is already turned into heat. Whether you use that for calculations or not does not make a difference. Not sure about the exact numbers, but I guess that at the moment normal roofs and solar panels absorb very roughly about the same percentage of sunlight. So if in the future solar panels become more efficient, then yes, the amount of sunlight turned into heat could double. Maybe that can be offset by covering other parts of earth with reflective materials or finding a way to send the heat back into the universe more effectively.
    - amelius 1y ago
      What if you put a solar farm in a desert, though? And also, people should paint their roofs white.
      - carunenjoyerlp 1y ago
        >And also, people should paint their roofs white. Some could, but a dark roof can be beneficial in the winter
- vmg12 1y ago
  > When robots drive cars, we will have more cars driving around This doesn't seem true. In SF, waymo with 300 cars does more rides than lyft with 45k drivers. If self driving cars interleave different tasks based on their routes I imagine they would be much more efficient per mile.
  - mg 1y ago
    Existing rides will be done more efficiently but since rides are so much cheaper without a driver, much more rides will be done. A car driving from A to B will cost less than 50% of the current price. Which will unlock a huge amount of new rides.
adrianN 1y ago
The important part remains internalizing emission costs into the price of electricity. Fussing over individual users seems like a distraction to me. Rapid decarbonization of electricity is necessary regardless of who uses it. Demand will soar anyway as we electrify transportation, heating, and industry.
- seb1204 1y ago
  I agree but reducing consumption or increase of efficiency are still very important aspects of the energy transition. What is not consumed does not need to be generated.
  - adrianN 1y ago
    Yeah, but pricing signals are a good way of reaching those goals.
  - Scarblac 1y ago
    Businesses will only start doing that in significant amounts when carbon emissions are priced according to their environmental impact.
  - Lerc 1y ago
    I don't think it a given that reducing energy consumption is a required part of the transition. Increasing demand can lead to stimulus of green energy production.
    - Uehreka 1y ago
      There’s no rule that increased demand will necessarily stimulate green energy production, only that it will stimulate energy production. And getting people to care about climate gets tougher, not easier, when energy demand goes up.
  - 0_____0 1y ago
    Sometimes I see chatter about using solar or nuclear or whatever power for data centers, thereby making them "clean," and it's frustrating that there isn't always the acknowledgement that the clean energy could displace other dirty generation. Even with things like orphaned natural gas that gets flared otherwise - rescuing the energy is great but we could use it for many things, not just LLMs or bitcoin mining!
Proofread0592 1y ago
> When you ask an AI model to write you a joke or generate a video of a puppy, that query comes with a small but measurable energy toll and an associated amount of emissions spewed into the atmosphere. Given that each individual request often uses less energy than running a kitchen appliance for a few moments, it may seem insignificant. > But as more of us turn to AI tools, these impacts start to add up. And increasingly, you don’t need to go looking to use AI: It’s being integrated into every corner of our digital lives. Forward looking, I imagine this will be the biggest factor in increasing energy demands for AI: companies shoving it into products that nobody wants or needs.
- Uehreka 1y ago
  In the short term perhaps, but even without carbon pricing the raw electricity prices will probably tamp down the enthusiasm. At someone point it’ll become cool for activist investors to demand to see ROI for AI features on earnings calls, and then the fat will get trimmed just like any other trend that goes too far. I think the bigger underrated concern is if LLMs fall into an unfortunate bucket where they are in fact generally useful, but not in ways that help us decarbonize our energy supply (or that do, but not enough to offset their own energy usage).
- keybored 1y ago
  Try to buy something that isn’t wrapped in three layers of plastic. Or that isn’t made of plastic itself. Then go to the checkout and see their “PSA” about how asking for a plastic bag to carry your plastic merchandise kills the planet. I’m sorry. I’m being blocked by some mysterious force from understanding what “actual human” means. And I don’t know how to get you in contact with your car manufacturer. Would you like me to repeat my 20 step suggestion on how to troubleshoot “why does my shitty car put the A/C on freezer mode whenever “Indian Summer” tops the charts in Bulgaria”, but with more festive emojis?
- Nition 1y ago
  > increasing energy demands for AI: companies shoving it into products that nobody wants or needs I think of this a little every time Google gives me another result with the AI summary and no option for anyone to turn it off. Apparently worldwide there are 8+ billion searches every day.
BewareTheYiga 1y ago
http://archive.today/mnHb8 http://archive.today/mnHb8
mitch_said 1y ago
[dead]
mark_l_watson 1y ago
The book “AI Atlas” covers the energy and other costs of AI.
- emushack 1y ago
  Link?
- simonw 1y ago
  You mean this one? https://en.m.wikipedia.org/wiki/Atlas_of_AI https://en.m.wikipedia.org/wiki/Atlas_of_AI It's from 2021 so won't cover the 2022-onwards generative AI boom. From the Wikipedia summary it sounds like it's about machine learning algorithms like classification, AlphaGo and concerns about ethics of training and bias.
  - mark_l_watson 1y ago
    thank you! I got the name wrong. “Atlas of AI” by Kate Crawford.
briandear 1y ago
What’s the net energy footprint of an employee working in an office whose job was made redundant by AI? Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. In other words, let’s calculate the “economic output per energy unit expended.” On that note, what’s the energy footprint of the return to office initiatives that many companies have initiated?
- Scarblac 1y ago
  When human civilization crashes due to yearly climate change caused famines it won't matter how useful the work done by the AIs was.
- folkrav 1y ago
  > Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do That’s a lot of big assumptions - that the job getting replaced was tedious in the first place, that those other “more productive” job exists, that the thing AI can’t necessarily do will stay that way long enough for it not to be taken over by AI as well, that the tediousness was not part of the point (e.g. art)…
- carunenjoyerlp 1y ago
  Net energy change of people doing work on their desk versus browsing the internet versus playing games, you will likely not see difference at all. They're all at rest, more or less thinking something. People at home sofa always have metabolic processes running regardless of whether it produces additional value to some corporation
- lm28469 1y ago
  > a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. Like driving a uber or delivering food on a bicycle ? Amazing!
est31 1y ago
I wonder how the energy requirements are distributed between training and inference. Training should be extremely flexible, so one can only train when the sun shines and nobody uses the huge amount of solar power, or only when the wind turbines turn.
- jnsaff2 1y ago
  AFAICT the energy cost of training is still fairly low compared to cost of GPU's themselves so especially during a land grab it's important to drive as near as possible full utilization of the GPU's, energy be damned. I doubt this is going to change. That said, the flip side of energy cost being not a big factor is that you could probably eat the increase of energy cost by a factor of say 2 and this could possibly enable installation of short term (say 12h) battery storage to enable you to use only intermittent clean energy AND drive 100% utilization.
  - deleted 1y ago
    [deleted]
carunenjoyerlp 1y ago
>you might think it’s like measuring a car’s fuel economy or a dishwasher’s energy rating: a knowable value with a shared methodology for calculating it. You’d be wrong. But everyone knows fuel economy is everything but a knowable value. Everything from if it has rained in the past four hours to temperature to loading of the vehicle to the chemical composition of the fuel (HVO vs traditional), how worn are your tires? Are they installed the right way? Are your brakes lagging? The possibilities are endless. You could end up with twice the consumption. By the way, copy-pasting from the website is terrible on desktop firefox, the site just lags every second, for a second.
- GuinansEyebrows 1y ago
  fuel economy, like blood glucose levels, is impacted by many factors, but you can measure it over time. you might not be able to prescribe a course of action but you can make corrections to the course you're already on.
emushack 1y ago
I would like to see more data centers make use of large-scale Oil Immersion-Cooling. I feel like the fresh water use for cooling is a huge issue. https://par.nsf.gov/servlets/purl/10101126 https://par.nsf.gov/servlets/purl/10101126
- carunenjoyerlp 1y ago
  Isn't water just the transfer medium between the server and the heat exchangers outside? How would changing that to oil help?
  - chneu 1y ago
    It wouldn't really help. Oil might be able to carry more heat but it's more expensive to use. Oil immersion is something nerds like to think is amazing but it's just a pain in the ass for negligible benefits. Imagine the annoyance of doing maintenance.
    - asdff 1y ago
      Wouldn't it be no different but your hands get a little oily? Say you take out a ram stick, oil goes into the empty dimm slot, but so what because its displaced again when you put in the new ram stick.
  - emushack 1y ago
    True, but water, especially fresh water, is used in so many applications. I don't have hard numbers on this because I'm not THAT big of a nerd, but intuitively, I would think that if we can find alternatives to water, that would free up some fresh water resources, for I don't know, growing food? This is actually a major issue in my area - A data center is being built right now, and competing with municipalities for fresh water. Maybe they don't have to?
- lxgr 1y ago
  There are various data center cooling techniques, and not all of them use water. As a result, water usage, and water usage efficiency, vary wildly: https://dgtlinfra.com/data-center-water-usage/ https://dgtlinfra.com/data-center-water-usage/ https://datacenters.microsoft.com/sustainability/efficiency/ https://datacenters.microsoft.com/sustainability/efficiency/
giantg2 1y ago
With all the issues and inefficiencies listed, there is a lot of room for improvement. I'm hopeful that just as the stat they give for data center energy not rising from 2005-2017, so to will the AI energy needs flatten in a few years. GPUs are not very efficient. Switching to more task specific hardware will provide more efficiency eventually. This is already happening a little with stuff like TPUs.
Lerc 1y ago
The point that stood out to me as concerning was "The carbon intensity of electricity used by data centers was 48% higher than the US average." I'd be fine with as many data centers as they want if they stimulated production of clean energy to run them. But that quote links to another article by the same author. Which says "Notably, the sources for all this power are particularly “dirty.” Since so many data centers are located in coal-producing regions, like Virginia, the “carbon intensity” of the energy they use is 48% higher than the national average. The paper, which was published on arXiv and has not yet been peer-reviewed, found that 95% of data centers in the US are built in places with sources of electricity that are dirtier than the national average. " Which in turn links to https://arxiv.org/abs/2411.09786 https://arxiv.org/abs/2411.09786 Which puts the bulk of that 48% higher claim on "The average carbon intensity of the US data centers in our study (weighted by the energy they consumed) was 548 grams of CO2e per kilowatt hour (kWh), approximately 48% higher than the US national average of 369 gCO2e / kWh (26)." Which points to https://ourworldindata.org/grapher/carbon-intensity-electricity?tab=chart&time=latest..2023&country=EU-27~KHM~OWID_UMC~USA https://ourworldindata.org/grapher/carbon-intensity-electric... For the average of 369g/KWh. That's close enough to the figure in the table at https://www.epa.gov/system/files/documents/2024-01/egrid2022_summary_tables.pdf https://www.epa.gov/system/files/documents/2024-01/egrid2022... which shows 375g/KWh (after converting from lb/MWh) But the table they compare against shows. VA 576g/KWh TX 509g/KWh CA 374g/KWh and the EPA table shows VA 268g/KWh TX 372g/KWh CA 207g/KWh Which seem more likely to be true. The paper has California at only marginally better than the national average for renewables (Which I guess they needed to support their argument given the number of data centers there) I like arxiv, It's a great place to see new ideas, the fields I look at have things that I can test myself to see if the idea actually works. I would not recommend it as a source of truth. Peer review still has a place. If they were gathering emissions data from states themselves, they should have caclulated the average from that data, not pulled the average from another potentially completely different measure. Then their conclusions would have been valid regardless what weird scaling factor they bought in to their state calculations. The numbers might have been wrong but the proportion would have been accurate, and it is the proportion that is being highlighted.
- GuinansEyebrows 1y ago
SoftTalker 1y ago
If you are old enough you remember posting to Usenet and the warning that would accompany each new submission: This program posts news to thousands of machines throughout the entire civilized world. Your message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. Are you absolutely sure that you want to do this? [ny] Maybe we meed something similar in LLM clients. Could be phrased in terms of how many pounds of atmospheric carbon the request will produce.
- mirekrusin 1y ago
  Or gigantic footer explaining you should consider impact before trying to print this email – which was always eating next page when printed.
- wmf 1y ago
  Only if that same warning is attached to literally everything else you do. It's unfair to single out AI for using energy.
  - SoftTalker 1y ago
    But "asking ChatGPT" is something people do so casually without there being any apparent clues of the costs associated with that. I guess that is true of pretty much everything online though. Even driving your car around you at least are somewhat aware of the gas you are burning.
    - asdff 1y ago
      That could be solved by charging more for the service. That is the only reason you are aware of the gas burning after all, you aren't conducting your own aq test you are noticing you are filling up twice a week at $50 a tank.
    - appreciatorBus 1y ago
      I would bet most ppl drive around with very little awareness of how much it’s costing, either in money or environmental impact. Many people I’ve met seem to measure efficiency by how much it costs to fill up the tank.
teekert 1y ago
I wonder what the carbon footprint of all those ads is.
- amelius 1y ago
  Not just the ads, but also the overconsumption which they cause.
  - __MatrixMan__ 1y ago
    Not just overconsumption, but also waste due to supply chain fragility. If you can induce demand anywhere then supply has to do crazy things to keep up.
sfpityparty 1y ago
[flagged]
Ericson2314 1y ago
We really need to improve the power grid. I don't think about "A. I." very much, but I am glad that something is making us upgrade the grid.
- lucb1e 1y ago
  > I am glad that something is making us upgrade the grid A few big consumers in centralized locations isn't changing the grid as much as the energy transition from fuels to electricity is
  - Ericson2314 1y ago
    The transition to electric vehicles in the US has been disappointing, to say the least.
    - lucb1e 1y ago
      I thought everyone over there has an air conditioning system on electricity?
      - Ericson2314 1y ago
        Yes, but we already do. We need new electricity demand.
acidburnNSA 1y ago
I'm glad that the needs of AI and the sustainable capabilities of nuclear fission go well together.
simonw 1y ago
I found this bit interesting: > In 2017, AI began to change everything. Data centers started getting built with energy-intensive hardware designed for AI, which led them to double their electricity consumption by 2023. As we all know, the generative AI boom only really kicked into high gear in November 2022 with ChatGPT. That's five years of "AI" growth between 2017 and 2022 which presumably was mostly not generative AI.
- SoftTalker 1y ago
  People started using GPUs for ML at around that time.
- mattnewton 1y ago
  2017 is the year after AlphaGo beats Lee Sedol and is when the attention is all you need paper was published. The writing was on the wall. OpenAI just found product market fit in november 2022, but the industry wasn't wandering aimlessly until then.
- NitpickLawyer 1y ago
  Meta was in the process of AI-ing everything on their stacks, with search, similarity, graphs, recommendation, etc. everywhere on their properties. Incidentally that's why they already had tens/hundreds of thousands of GPUs when the LLM craze hit, and why they were in a good place to work on llama and other stuff.
phillipcarter 1y ago
Well, this was disappointing: > There is a significant caveat to this math. These numbers cannot serve as a proxy for how much energy is required to power something like ChatGPT 4o. Otherwise this is an excellent article critiquing the very real problem that is opacity of these companies regarding model sizes and deployments. Not having an honest accounting of computing deployed worldwide is a problem, and while it's true that we didn't really do this in the past (early versions of Google searches were undoubtedly inefficient!), it's not an excuse today. I also wish this article talked about the compute trends. That is, compute per token is going significantly down, but that also means use of that compute can spread more. Where does that lead us?
blkhawk 1y ago
the numbers in the article are all over the place. I mean the article seems to try and some of the more general calculations on paper should work out but especially the image gen ones I can sorta disprove with my own experiences in local gen. Even were it matches sorta (the 400 feet e-bike thing) that only works out for me because I use an AMD card. An NVIDIA card can have several times the generation speed at the same power draw so it all falls down again. And the parameters they tried to standardize their figures with (the 1024x1024 thing) is also a bit meh because the SAME amount of pixels in a different aspect ratio can have huge variations in gen speed and thus power usage. for instance for most illustrious type checkpoints the speed is about 60% higher at aspect ratios other than 1024x1024. Its all a bit of a mess.
bbor 1y ago
Interesting, thanks for sharing! I share some concerns others have about this piece, but I’m most shocked about their finding that image generation is cheaper than text. As someone who’s gone down this rabbit hole multiple times, this runs against every single paper I’ve ever cited on the topic. Anyone know why? Maybe this is a recent change? It also doesn’t help that multimodal transformers are now blurring the lines between image and text, of course… this article doesn’t even handle that though, treating all image models as diffusion models.
fred69 1y ago
Might have missed it but was disappointed to see no mention of externalized costs like the scraping burden imposed on every IP-connected server. From discussions on HN this sounds quite substantial. And again, why exactly should the few AI companies reap all the value when other companies and individuals are incurring costs for it?
Henchman21 1y ago
I work in DCO, thats Data Center Operations if you’re not aware. I’ve tried explaining the amount of power used to my elderly mom; it isn’t easy! But here’s my best take: The racks I am personally responsible for consume 17.2kW. That’s consistent across the year; sure things dip a bit when applications are shut down, but in general 17.2kW is the number. Presuming a national average of 1.2kW per home, each rack of equipment I oversee could potentially power 14 houses. I am responsible for hundreds of these racks, while my larger organization has many thousands of these racks in many locations worldwide. I’ve found no other way to let the scale of this sink in. When put this way she is very clear: the price isn’t worth it to humanity. Being able to get, say, Door Dash, is pretty neat! But not at the cost of all our hoarded treasure and certainly not at the cost of the environment on the only planet we have access to. The work done by AI will only ever benefit the people at the top. Because to be frank: they won’t share. Because the very wealthy have hoarding disorder.
- jeffbee 1y ago
  It seems like you are having an emotional response to not understanding the general energy picture. For example, an A320 aloft uses the same energy as two thousand of your hypothetical racks (2.5 tons of kerosene per hour). Each! We are in no meaningful sense torching the biosphere to get AI.
  - globnomulous 1y ago
    > It seems like you are having an emotional response to not understanding the general energy picture. This is condescending and rude. It also strikes me as obviously wrong. The 'response' isn't emotional in the slightest; it just explains the emotional and cognitive experience of acquiring understanding. It's a reasonable, well-reasoned explanation of the difficulty of intuitively grasping how much energy these data centers, and thus the companies that use them, consume -- and then of the shock many experience when it dawns on them. > For example, an A320 aloft uses the same energy as two thousand of your hypothetical racks (2.5 tons of kerosene per hour). > Each! > We are in no meaningful sense torching the biosphere to get AI. What exactly is the reasoning here? That airplanes use a lot of energy, just like data centers or compared to data centers, and therefore that AI isn't ecologically damaging -- or isn't "meaningfully" damaging, whatever that means? That's not just wrong. It's a nonsequitur. There's a simpler, easier, better way to wrap our heads around the data, one that doesn't require false dichotomies (or something like that): humans are torching the biosphere both to get AI and to travel.
floxy 1y ago
40% of electricity consumption in Virgina will be data centers in 2030? Table A1 , PDF page 29: https://www.epri.com/research/products/000000003002028905 https://www.epri.com/research/products/000000003002028905
- coolcase 1y ago
  Virginia, us-east-2, U.S.A. (P.S. check your spelling!)
  - floxy 1y ago
    Whoops!
deleted 1y ago
[deleted]
djoldman 1y ago
> This leaves even those whose job it is to predict energy demands forced to assemble a puzzle with countless missing pieces, making it nearly impossible to plan for AI’s future impact on energy grids and emissions. Worse, the deals that utility companies make with the data centers will likely transfer the costs of the AI revolution to the rest of us, in the form of higher electricity bills. ... So don't? Explicitly shift the cost to the customer. If I want to hook up to the energy grid with 3-phase power, I pay the utility to do it. If a business wants more power and it isn't available, then the business can pay for it. Then only businesses that really need it will be willing to step up to the plate. No amount of "accounting" or "energy needs prediction" will guard against regulatory capture.
cco 1y ago
Today Google launched a model, Gemma 3n, that performs about as good as SOTA models from 1-2 years ago that runs locally on a cell phone. Training SOTA models will, like steel mills or other large industrial projects, require a lot of environmental footprint to produce. But my prediction is that over time the vast majority of use cases in the hands of users will be essentially run on device and be basically zero impact, both in monetary cost and environment.
jeffbee 1y ago
This series of articles is driving me insane. The authors or editors are using inappropriate units to shock readers: billions of gallons, millions of square feet. But they are not putting the figures into context that the reader can directly comprehend. Because if they said the Nevada data centers would use 2% as much water as the hay/alfalfa industry in Nevada then the entire article loses its shock value.
- panstromek 1y ago
  I agree. Even first few paragraphs strike me as intentionaly misleading. The whole "AI energy saga" seems to be full of manipulative claims. I can't help to feel that the intent is just to color the AI as bad in whatever way possible. It feels like it's deriving the research from predetermined conclusions
protocolture 1y ago
Weird I was assured that Bitcoin would be using all of the worlds electricity by now. Which I already thought was odd, because London would need all that electricity to see through the giant mountain of poop piled up by all the horses the british use for transportation.
gitroom 1y ago
[dead]
scottcha 1y ago
Shameless plug . . . I run a startup who is working to help this https://neuralwatt.com https://neuralwatt.com We are starting with an os level (as in no model changes/no developer changes required) component which uses RL to run AI with a ~25% energy efficiency improvement w/out sacrificing UX. Feel free to dm me if you are interested in chatting either about problems you face with energy and ai or if you'd like to learn more.
mNovak 1y ago
This gives me the silly idea to go try to measure the power consumption of the local data center by measuring the magnetic field coming off the utility lines.
Quenby 1y ago
[dead]
teleforce 1y ago
> unprecedented and comprehensive look at how much energy the AI industry uses Not sure about comprehensive claim here if end-to-end query chains were not considered. For example the mobile wireless node (that're being used by the majority of the users) contribution to the energy consumption are totally ignored. The wireless power amplifier or PA for both sides of users and base-stations are notorious for their inefficiency being only less than than 50% in practice although in theory can be around 80%. Almost all of the current AI applications are cloud based not local-first thus the end users energy consumption and contribution are necessary.
neves 1y ago
Best article I've ever read about the energy needs of AI. Impressive how Big Tech refuses to share data with society for collective decisions. I'd also recommend the Data Vampires podcast series: https://techwontsave.us/episode/241_data_vampires_going_hyperscale_episode_1 https://techwontsave.us/episode/241_data_vampires_going_hype... https://techwontsave.us/episode/243_data_vampires_opposing_data_centers_episode_2 https://techwontsave.us/episode/243_data_vampires_opposing_d... https://techwontsave.us/episode/245_data_vampires_sacrificing_for_ai_episode_3 https://techwontsave.us/episode/245_data_vampires_sacrificin... https://techwontsave.us/episode/247_data_vampires_fighting_for_control_episode_4 https://techwontsave.us/episode/247_data_vampires_fighting_f...
johnny_________ 1y ago
I ponder this a lot, but the interface of "MIT technology Review" is unbearably overdesigned, its got that annoying narrow smartphone format where you can't zoom out, and then all these fancy graphics. Can't we have crisp, easy-to-read HTML? The format annoyed me so much I didn't read the article because this kind of design makes me doubt the source. Alas
- myaccountonhn 1y ago
  The article works fine with the browser's reader-mode for me.
paulcole 1y ago
Are these the same people who claimed that crypto was going to use more energy than the entire world by 2030?
- kleiba 1y ago
  This reminds me of the quote: "Félix L'Herbier learns that there are more links in his brain than atoms in the universe."
LordDragonfang 1y ago
> The new model uses [energy] equivalent to riding 38 miles on an e-bike... AI companies have defended these numbers saying that generative video has a smaller footprint than the film shoots and travel that go into typical video production. That claim is hard to test and doesn’t account for the surge in video generation that might follow if AI videos become cheap to produce. "Hard to test", but very obviously true if you make any attempt at guessing based on making a few assumptions... like they seem comfortable doing for all the closed source models they don't have access to being run in conditions they're not testing for. Especially considering they're presenting their numbers as definitive, and then just a couple paragraphs down admit that, yeah, they're just guessing. Regardless, I know for a fact that a typical commercial shoot uses way more energy than driving across the TMZ in an e-bike (considering they're definitely using cars to transport gear, which gives you less than 4 miles for the same energy).
thedevilslawyer 1y ago
What's the energy imprint of the human doing the same work that AI is now going to do? If AI's imprint is less, what is the right thing for us to do?
- ahtihn 1y ago
  Kill the human? If you don't want to go there, it doesn't really matter how much energy the human uses because the human will just use the same energy to do something else.
  - zavec 1y ago
    Not necessarily. I think the point of comparison here is how much energy does AI use to e.g. generate a video, compared to the energy used not by the human themselves, but by running XYZ software on a computer with a beefy graphics card for however many hours it'd take a human to do the same work.
    - thedevilslawyer 1y ago
      Indeed. Everything taken into account - work getting done takes energy. And if an agent can do a task for less energy than a human, then indeed it's a benefit. This would be the apt comparison, instead of looking at some overall datacenter energy consumption.
    - shmeeed 1y ago
      While that's a valid point of view, as long as the human is the bottleneck, it's not going to scale to infinity and beyond. Human's got to exist and needs to work to eat. They don't really, necessarily, existentially need to be 10x productive with the help of AI. But I'll be honest, that's not really a solid argument, because it could rapidly lead to the question of why they do this exact job in the first place, instead of e.g. farming or whatever else there might be that can be called a net positive for humanity without reservations.
    - HPsquared 1y ago
      Or a team going to a location and filming, etc.
xeox538 1y ago
I believe we're currently seeing AI in the "mainframe" era, much like the early days of computing, where a single machine occupied an entire room and consumed massive amounts of power, yet offered less compute than what now fits in a smartphone. I expect rapid progress in both model efficiency and hardware specialization. Local inference on edge devices, using chips designed specifically for AI workloads, will drastically reduce energy consumption for the majority of tasks. This shift will free up large-scale compute resources to focus on truly complex scientific problems, which seems like a worthwhile goal to me.
- panstromek 1y ago
  It sure seems like that to me. I was pretty impressed by how easily I could run small Gemma on 7 year old laptop and get a decent chat experience. I can imagine that doing some clever offloading to a normal programs and using the LLM as a sort of "fuzzy glue" for the rest could improve the efficiency on many common tasks.
  - coolcase 1y ago
    Big tech ain't investing heavily so you can run local, what data does that leave them to sell, and what power and control does that give them. Zilch.
    - panstromek 1y ago
      I mean.. cute conspiracy but it doesn't correspond with reality. Just look what's Google releasing, they are trying to make these things fit on consumer hardware.
- whatnow37373 1y ago
  The CPU development curve is often thrown around but it very seldomly fits anything else in reality. It was a very rare and extraordinary set of coincidences that got it us here. Computation using silicon turned out to have massive growth potential for a variety of lucky reasons but say battery tech is not so lucky, nor is fusion nor is quantum computing. The low hanging fruit has been plucked by said silicon development process and while remarkable improvement in AI efficiency is likely it is highly unlikely for that to follow a similar curve. More likely is slow, incremental process taking decades. We cannot just wish away billions of parameters and the need for trillions of operations. It’s not like we have some open path of possible improvement like with silicon. We walked that path already. Maybe photonics..
tempfile 1y ago
Jesus, who writes this stuff? > AI is unavoidable > We will speak to models in voice mode, chat with companions for 2 hours a day, and point our phone cameras at our surroundings in video mode This is surely meant to be an objective assessment, not a fluff piece.
dr_dshiv 1y ago
Solving climate change will take a lot of energy. I found this article to be a little too one sided. For instance, it didn’t talk about the 10x reductions in power achieved this past year — essentially how gpt4 can now run on a laptop. Viz, via sama “The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use. You can see this in the token cost from GPT-4 in early 2023 to GPT-4o in mid-2024, where the price per token dropped about 150x in that time period. Moore’s law changed the world at 2x every 18 months; this is unbelievably stronger.” https://blog.samaltman.com/three-observations https://blog.samaltman.com/three-observations
- Juliate 1y ago
  It's still _additional_ power usage that 1/ did not exist before 2/ does not replace/reduce previous/other power (some, very much more critical and essential) usages. 3/ a LOT of tasks are still way more energy/time-efficiently done with regular existing methods (dedicated software, or even by hand), but are still asked/improperly routed to AI chatbots that ... statistically guess the answer.
  - dr_dshiv 1y ago
    It is also additional value creation. It also leads to automation and efficiency, even if it isn’t a fully linear path. AI isnt a waste. We can’t let environmental consciousness get in the way of rather natural human development. Especially CO2. (I have different opinions about biodiversity because of the irreversible loss. I also believe that we have the technology to pause and reverse climate change — but don’t pursue it because of degrowth ideologies)
    - t_tsonev 1y ago
      No amount of value, in the economic sense, will solve climate change.
      - SirHumphrey 1y ago
        Well, somebody must buy solar panels, batteries, (nuclear reactors) and finance the whole research process. While “economic value” =/= “solving climate change” without enough tax revenue costly transitions are impossible.
panstromek 1y ago
There's a certain irony here in the fact that this page is maxing out my CPU on idle, doing some unclear work in javascript, while I'm just reading text.
- guappa 1y ago
  I would like browsers to have very strict limits on the amount of cycles a page can use, and have some permission like for camera to use more CPU.
  - jve 1y ago
    What a nice idea.
    - tessellated 1y ago
      top joke!
  - sligor 1y ago
    I would like also that devs run their site/apps on downgraded machine/VM so that they can early detect performance issues.
    - guappa 1y ago
      In my company the frontend team has shiny new powerful machines. I on the other hand don't have enough RAM to compile our server.
      - coolcase 1y ago
        Does that create a sort of Conway's law pushing complexity to the FE?
        guappa 1y ago
        They offset that risk by taking 2 months to design and implement complex features like "add a new text box in the form, to enter an IP address". I haven't seen them merge a pull request in less than 3 days. However simple. The team lead, in 2 years, contributed less than 20 pull requests, all of which were 1 to 3 line changes to CSS or similar.
liendolucas 1y ago
> ...and many buildings use millions of gallons of water (often fresh, potable water) per day in their cooling operations. This is outrageous. People still struggle to access fresh water (and power), but hey "sustainability is all to our company" is always promoted as if something nice is being done on from the behemoth's sides. BS. What a waste of resources. I truly condemn all this. To this day I do still refuse to use any of this technology and hope that all this ends in the near future. It's madness. I see this as nothing more than next-gen restrictive lousy search engines, and as many have pointed out ads are going to roll soon. The more people adopt it the worse will be for everyone. I always emphasize this: 10-15 years ago I could find everything through simple web searches. Everything. In many cases even landing on niche and unexpected but useful and interesting websites. Today that is a difficult/impossible task. Perhaps there is still room for a well-done traditional search engine (haven't tried Kagi but people in general do say nice things about it) to surface and take the lead but I doubt it, when hype arrives especially in the tech industry people follow blindly. There are still flourishing "ai" startups and from night to day everyone has become a voice or expert on the subject. Again: BS. Traditional web engines and searches were absolutely just fine and quite impressed with their outputs. I remember it. What the heck has happened?
- bogtog 1y ago
  > ...and many buildings use millions of gallons of water (often fresh, potable water) per day in their cooling operations. Of note, cooling is water evaporation, so the water will inevitably come back to us as good as new. This contrasts uses that will actually pollute water
bogtog 1y ago
> The largest model we tested has 405 billion parameters, but others, such as DeepSeek, have gone much further, with over 600 billion parameters. Very quickly skimming, I have some trouble taking this post seriously when it omits that the larger DeepSeek one is a mixture-of-experts that will only use 12.5% (iirc) of its components for each token. The best summary of text energy use I've seen is this (seemingly more rigorous, although its estimates are consistent with the final numbers made by the present post): epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use Estimates for a given response widely for a "typical" query (0.3 Wh; 1080 joules) and a maximal-context query (40 Wh; 144k joules). Assuming most uses don't come close to maximizing the context, the energy use of text seems very small compared to the benefits. That being said, the energy use for video generation seems substantial I would be interested in seeing the numbers corresponding to how LLMs are typically used for code generation
James_K 1y ago
The fact that none of these companies want to tell you how much power they're using should be enough to reason that it's utterly horrible. If it were anywhere reasonable, they'd be boasting about how low the number is.
janaagaard 1y ago
- "4.4% of all the energy in the US now goes toward data centers" - "by 2028 [...] AI alone could consume as much electricity annually as 22% of all US households." What would the 22% be if compared against all US energy instead of just all US household?
- EdiX 1y ago
  US households constitute 21% of all energy use [1], so 22% of this is 0.21 * 0.22, which is 4%. [1] https://rpsc.energy.gov/energy-data-facts https://rpsc.energy.gov/energy-data-facts
southernplaces7 1y ago
The number of people in this comment thread defending this gargantuan energy footprint for a technology that currently in large measure is being used for a tremendous amount of dogshit things (and oceans of visual/text spam) is amusing considering the hysterics that this same energy use problem caused when it came to crypto. I guess it becomes okay when the companies guzzling the energy are some of the biggest tech employers in the world, buttering your bread in some way.
- admiralrohan 1y ago
  Good that you connected it with crypto boom. But problem is it's hard to rewind these kind of trends once started. Human nature.
  - pjc50 1y ago
    NFTs are dying and link rotting. ICOs are out of fashion. VR is back in its niche. 3DTV .. I've not seen that marketed for quite a while now. Things that are fads will die out eventually. Crypto meanwhile is in an odd space: everyone knows blockchains are too much of a hassle and that the volatility is too high, so they're using centralized exchanges and "stablecoins" (shadow dollars). There's still a huge amount of money there but not quite as much as the stadium ads / FTX peak.
    - alnwlsn 1y ago
      I wish you could still buy 3D TVs. Why? We used them to build a truck simulator which had a driver and passenger. The view out the window is different for each person. They were perfect for that, but it's too small a niche to keep making new TVs.
      - ben_w 1y ago
        Have you considered DIY-ing it with a normal TV/monitor and a lenticular lens?
    - dragonwriter 1y ago
deleted 1y ago
[deleted]
polski-g 1y ago
Seems like a solved problem. We have an unlimited sources of clean power from nuclear and solar. China alone now has almost 1TW of solar capacity. Build more nuclear, build more solar. Tax carbon.
troyvit 1y ago
> Tech companies like Meta, Amazon, and Google have responded to this fossil fuel issue by announcing goals to use more nuclear power. Those three have joined a pledge to triple the world’s nuclear capacity by 2025. Erm ... that's a weird date considering this article came out yesterday. They actually pledge to triple the world's nuclear capacity by 2050[1] There are a couple of weird things like that in this article, including the classic reference to "experts" for some of its data points. Still ... at least somebody's trying to quantify this. [1] https://www.world-nuclear-news.org/articles/amazon-google-meta-and-dow-back-tripling-nuclear-goal https://www.world-nuclear-news.org/articles/amazon-google-me...
- neves 1y ago
  The weirdest thing in the article is the refusal of Big Tech to release this data. We wouldn't experts to guess. Society must have information to make decisions about what affects all the world.
- WillPostForFood 1y ago
  Do you think it is weird or just a typo?
  - troyvit 1y ago
    Heh, when I first started typing it I thought it was weird, then I discovered it was a typo, but didn't edit my first sentence. Weird huh?
palmfacehn 1y ago
From "The Limits to Growth" to peak oil and beyond: As a rule, scarcity doom and neo-malthusianism hasn't played out the way proponents claim. I've been very critical of the AI hype cycle here and elsewhere, but this isn't it. In the long run technological advancements increase our productivity and quality of life. Yes, much of what is being promoted is slop. Yes, this bubble is driven by an overly financialized economy. That doesn't preclude the possibility of AI models precipitating meaningful advancements in the human condition. From refrigeration to transportation, cheap and abundant energy has been one of the major driving forces in human advancement. Paradoxically, consuming cheap energy doesn't reduce the amount of energy available on the market. Instead it increases the size of the market.
thedudeabides5 1y ago
long silver for solar
pizzuh 1y ago
Energy use is going to continue to climb. I'm worried about the environmental impacts of this, but from everything I've seen society values model output more. Curious to watch this over the rest of the decade.
- jes5199 1y ago
  solar and battery costs are falling so fast that emissions are likely to go down, even with growth in energy use
  - edhelas 1y ago
    Sure sure, you did the maths ? How many batteries do you need to power-up a datacenter at night ?
    - Filligree 1y ago
      Not that many, if you pair it with offshore wind, a large grid (which covers multiple time zones), and ideally nuclear. The conditions for that don’t exist everywhere, but we don’t need these datacentres everywhere. LLMs aren’t latency-sensitive.
      - p_j_w 1y ago
        >if you pair it with offshore wind Unfortunately the current government of the US is entirely beholden to fossil fuel interests so they're actively hostile to offshore wind.
    - jes5199 1y ago
      Tony Seba did the maths: https://www.rethinkx.com/publications/understandingstellarenergy2025.en https://www.rethinkx.com/publications/understandingstellaren...
  - pizzuh 1y ago
    Great point. Assuming we can adopt them at the correct pace and on a global scale, then energy doesn't have to be a limiting factor
some_random 1y ago
We shouldn't be hand wringing on energy usage, we should be generating energy too cheap to bother metering and too clean to bother thinking about. Nuclear baseline, renewables everywhere they make sense, we know how to do this we just need to do it.
airylizard 1y ago
As more and more people use brute force loops to make their AI agents more reliable, this hidden inference giant will only continue to grow. This is why I put my framework together, using just 2 passes as opposed to n+ can increase accuracy and reliability by a far greater amount than brute force loops while using significantly less resources. Supportive evidence and data can be found in my repo: https://github.com/AutomationOptimization/tsce_demo https://github.com/AutomationOptimization/tsce_demo
benlivengood 1y ago
I find it weird that on the whole there wasn't as huge a pushback against AWS and Google and Facebook (by far the biggest fleets of data centers) for power and carbon emissions over the last couple decades. Maybe they managed propaganda better about paying for future renewable resources or how low their PUEs are. Turning 4% of the U.S.'s electricity into cat videos and online shopping and advertisements and heat already sounds like a lot of use. Maybe the rapid rise of AI use is what's alarming people? Contrast the 2016 study[0] of data center energy use where use was recently flat because of efficiency improvements in 2010-2020 but historically there was a ton of growth in energy consumption since ~1990; basically we have always been on a locally exponential growth curve in data center energy use but our constant factors were being optimized by the hyperscalers in that 2010-2020 period. We also need to compare the efficiency of AI with other modes of computation/work. The article goes into detail on the supposed actual energy use but there's a simple metric; All the large companies provide costs per unit of inference which can put a hard ceiling on actual energy cost. Something like $20/1M tokens for the best models. METR used a 2M token budget. So you can currently price out N hours of work at $40 from whichever latest METR benchmarks come out and have a worst case cost for efficiency comparison. Lastly, if we're not on a trend toward having Dyson swarms of compute in the long run then what are we even doing as a species? Of course energy spent on compute is going to grow quadratically into the distant future as we expand. People are complaining about compute for AI but compute is how we figure things out and get things done. AI is the latest tool. [0] https://eta.lbl.gov/publications/united-states-data-center-energy https://eta.lbl.gov/publications/united-states-data-center-e...
insane_dreamer 1y ago
It's hard to believe that as a society we are prioritizing AI queries (which are fine, nothing wrong with them in principle, and they do increase productivity in some cases) over the future wellbeing of our species (and planet). Winter is Coming, and these big companies and their investors couldn't care less. I can't decide whether they think it won't be that bad, or the scientific forecasts are wrong, or that they just don't care because whatever turmoil results from serious climate change, they'll be able to rise above it, and well, fuck the rest of humanity. I'm disgusted by our sense of priorities. (Though maybe I shouldn't be since we live in a country that values subsidizing the industrial-military complex over the health and education of its citizens.)

312 points by pseudolus 1y ago

When companies make ESG claims, sensible measurement and open traceability should always be the first proof they must provide. Without these, and validation from a credible independent entity such as a non-profit or government agency, all ESG claims from companies are merely PR puff pieces to keep the public at bay (especially in "AI").

stevage 1y ago
esg?
- JohnFen 1y ago
  Environmental/Social/Governance. From Wikipedia: Environmental, social, and governance (ESG) is shorthand for an investing principle that prioritizes environmental issues, social issues, and corporate governance.
kkarakk 1y ago
when has ESG not been FUD or a way to bypass sanctions from poorly thought out climate change targets?

mg 1y ago

The brain uses 20% of the human body's energy. I wouldn't be surprised if mankind will evolve similar to an organism and use 20% of all energy it produces on AI. Which is about 10x of what we use for software at the moment. But then more AI also means more physical activity. When robots drive cars, we will have more cars driving around. When robots build houses, we will have more houses being built, etc. So energy usage will probably go up exponentially. At the moment, the sun sends more energy to earth in an hour than humans use in a year. So the sun alone will be able to power this for the foreseeable future.

amelius 1y ago
One problem: all this energy is eventually turned into heat ...
- mg 1y ago
  Most of the sunlight that hits a roof is already turned into heat. Whether you use that for calculations or not does not make a difference. Not sure about the exact numbers, but I guess that at the moment normal roofs and solar panels absorb very roughly about the same percentage of sunlight. So if in the future solar panels become more efficient, then yes, the amount of sunlight turned into heat could double. Maybe that can be offset by covering other parts of earth with reflective materials or finding a way to send the heat back into the universe more effectively.
  - amelius 1y ago
    What if you put a solar farm in a desert, though? And also, people should paint their roofs white.
    - carunenjoyerlp 1y ago
      >And also, people should paint their roofs white. Some could, but a dark roof can be beneficial in the winter
vmg12 1y ago
> When robots drive cars, we will have more cars driving around This doesn't seem true. In SF, waymo with 300 cars does more rides than lyft with 45k drivers. If self driving cars interleave different tasks based on their routes I imagine they would be much more efficient per mile.
- mg 1y ago
  Existing rides will be done more efficiently but since rides are so much cheaper without a driver, much more rides will be done. A car driving from A to B will cost less than 50% of the current price. Which will unlock a huge amount of new rides.

adrianN 1y ago

The important part remains internalizing emission costs into the price of electricity. Fussing over individual users seems like a distraction to me. Rapid decarbonization of electricity is necessary regardless of who uses it. Demand will soar anyway as we electrify transportation, heating, and industry.

seb1204 1y ago
I agree but reducing consumption or increase of efficiency are still very important aspects of the energy transition. What is not consumed does not need to be generated.
- adrianN 1y ago
  Yeah, but pricing signals are a good way of reaching those goals.
- Scarblac 1y ago
  Businesses will only start doing that in significant amounts when carbon emissions are priced according to their environmental impact.
- Lerc 1y ago
  I don't think it a given that reducing energy consumption is a required part of the transition. Increasing demand can lead to stimulus of green energy production.
  - Uehreka 1y ago
    There’s no rule that increased demand will necessarily stimulate green energy production, only that it will stimulate energy production. And getting people to care about climate gets tougher, not easier, when energy demand goes up.
- 0_____0 1y ago
  Sometimes I see chatter about using solar or nuclear or whatever power for data centers, thereby making them "clean," and it's frustrating that there isn't always the acknowledgement that the clean energy could displace other dirty generation. Even with things like orphaned natural gas that gets flared otherwise - rescuing the energy is great but we could use it for many things, not just LLMs or bitcoin mining!

Proofread0592 1y ago

> When you ask an AI model to write you a joke or generate a video of a puppy, that query comes with a small but measurable energy toll and an associated amount of emissions spewed into the atmosphere. Given that each individual request often uses less energy than running a kitchen appliance for a few moments, it may seem insignificant. > But as more of us turn to AI tools, these impacts start to add up. And increasingly, you don’t need to go looking to use AI: It’s being integrated into every corner of our digital lives. Forward looking, I imagine this will be the biggest factor in increasing energy demands for AI: companies shoving it into products that nobody wants or needs.

Uehreka 1y ago
In the short term perhaps, but even without carbon pricing the raw electricity prices will probably tamp down the enthusiasm. At someone point it’ll become cool for activist investors to demand to see ROI for AI features on earnings calls, and then the fat will get trimmed just like any other trend that goes too far. I think the bigger underrated concern is if LLMs fall into an unfortunate bucket where they are in fact generally useful, but not in ways that help us decarbonize our energy supply (or that do, but not enough to offset their own energy usage).
keybored 1y ago
Try to buy something that isn’t wrapped in three layers of plastic. Or that isn’t made of plastic itself. Then go to the checkout and see their “PSA” about how asking for a plastic bag to carry your plastic merchandise kills the planet. I’m sorry. I’m being blocked by some mysterious force from understanding what “actual human” means. And I don’t know how to get you in contact with your car manufacturer. Would you like me to repeat my 20 step suggestion on how to troubleshoot “why does my shitty car put the A/C on freezer mode whenever “Indian Summer” tops the charts in Bulgaria”, but with more festive emojis?
Nition 1y ago
> increasing energy demands for AI: companies shoving it into products that nobody wants or needs I think of this a little every time Google gives me another result with the AI summary and no option for anyone to turn it off. Apparently worldwide there are 8+ billion searches every day.

BewareTheYiga 1y ago

http://archive.today/mnHb8 http://archive.today/mnHb8

mitch_said 1y ago

[dead]

mark_l_watson 1y ago

The book “AI Atlas” covers the energy and other costs of AI.

emushack 1y ago
Link?
simonw 1y ago
You mean this one? https://en.m.wikipedia.org/wiki/Atlas_of_AI https://en.m.wikipedia.org/wiki/Atlas_of_AI It's from 2021 so won't cover the 2022-onwards generative AI boom. From the Wikipedia summary it sounds like it's about machine learning algorithms like classification, AlphaGo and concerns about ethics of training and bias.
- mark_l_watson 1y ago
  thank you! I got the name wrong. “Atlas of AI” by Kate Crawford.

briandear 1y ago

What’s the net energy footprint of an employee working in an office whose job was made redundant by AI? Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. In other words, let’s calculate the “economic output per energy unit expended.” On that note, what’s the energy footprint of the return to office initiatives that many companies have initiated?

Scarblac 1y ago
When human civilization crashes due to yearly climate change caused famines it won't matter how useful the work done by the AIs was.
folkrav 1y ago
> Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do That’s a lot of big assumptions - that the job getting replaced was tedious in the first place, that those other “more productive” job exists, that the thing AI can’t necessarily do will stay that way long enough for it not to be taken over by AI as well, that the tediousness was not part of the point (e.g. art)…
carunenjoyerlp 1y ago
Net energy change of people doing work on their desk versus browsing the internet versus playing games, you will likely not see difference at all. They're all at rest, more or less thinking something. People at home sofa always have metabolic processes running regardless of whether it produces additional value to some corporation
lm28469 1y ago
> a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. Like driving a uber or delivering food on a bicycle ? Amazing!

est31 1y ago

I wonder how the energy requirements are distributed between training and inference. Training should be extremely flexible, so one can only train when the sun shines and nobody uses the huge amount of solar power, or only when the wind turbines turn.

jnsaff2 1y ago
AFAICT the energy cost of training is still fairly low compared to cost of GPU's themselves so especially during a land grab it's important to drive as near as possible full utilization of the GPU's, energy be damned. I doubt this is going to change. That said, the flip side of energy cost being not a big factor is that you could probably eat the increase of energy cost by a factor of say 2 and this could possibly enable installation of short term (say 12h) battery storage to enable you to use only intermittent clean energy AND drive 100% utilization.
- deleted 1y ago
  [deleted]

carunenjoyerlp 1y ago

>you might think it’s like measuring a car’s fuel economy or a dishwasher’s energy rating: a knowable value with a shared methodology for calculating it. You’d be wrong. But everyone knows fuel economy is everything but a knowable value. Everything from if it has rained in the past four hours to temperature to loading of the vehicle to the chemical composition of the fuel (HVO vs traditional), how worn are your tires? Are they installed the right way? Are your brakes lagging? The possibilities are endless. You could end up with twice the consumption. By the way, copy-pasting from the website is terrible on desktop firefox, the site just lags every second, for a second.

GuinansEyebrows 1y ago
fuel economy, like blood glucose levels, is impacted by many factors, but you can measure it over time. you might not be able to prescribe a course of action but you can make corrections to the course you're already on.

emushack 1y ago

I would like to see more data centers make use of large-scale Oil Immersion-Cooling. I feel like the fresh water use for cooling is a huge issue. https://par.nsf.gov/servlets/purl/10101126 https://par.nsf.gov/servlets/purl/10101126

carunenjoyerlp 1y ago
Isn't water just the transfer medium between the server and the heat exchangers outside? How would changing that to oil help?
- chneu 1y ago
  It wouldn't really help. Oil might be able to carry more heat but it's more expensive to use. Oil immersion is something nerds like to think is amazing but it's just a pain in the ass for negligible benefits. Imagine the annoyance of doing maintenance.
  - asdff 1y ago
    Wouldn't it be no different but your hands get a little oily? Say you take out a ram stick, oil goes into the empty dimm slot, but so what because its displaced again when you put in the new ram stick.
- emushack 1y ago
  True, but water, especially fresh water, is used in so many applications. I don't have hard numbers on this because I'm not THAT big of a nerd, but intuitively, I would think that if we can find alternatives to water, that would free up some fresh water resources, for I don't know, growing food? This is actually a major issue in my area - A data center is being built right now, and competing with municipalities for fresh water. Maybe they don't have to?
lxgr 1y ago
There are various data center cooling techniques, and not all of them use water. As a result, water usage, and water usage efficiency, vary wildly: https://dgtlinfra.com/data-center-water-usage/ https://dgtlinfra.com/data-center-water-usage/ https://datacenters.microsoft.com/sustainability/efficiency/ https://datacenters.microsoft.com/sustainability/efficiency/

giantg2 1y ago

With all the issues and inefficiencies listed, there is a lot of room for improvement. I'm hopeful that just as the stat they give for data center energy not rising from 2005-2017, so to will the AI energy needs flatten in a few years. GPUs are not very efficient. Switching to more task specific hardware will provide more efficiency eventually. This is already happening a little with stuff like TPUs.

Lerc 1y ago

The point that stood out to me as concerning was "The carbon intensity of electricity used by data centers was 48% higher than the US average." I'd be fine with as many data centers as they want if they stimulated production of clean energy to run them. But that quote links to another article by the same author. Which says "Notably, the sources for all this power are particularly “dirty.” Since so many data centers are located in coal-producing regions, like Virginia, the “carbon intensity” of the energy they use is 48% higher than the national average. The paper, which was published on arXiv and has not yet been peer-reviewed, found that 95% of data centers in the US are built in places with sources of electricity that are dirtier than the national average. " Which in turn links to https://arxiv.org/abs/2411.09786 https://arxiv.org/abs/2411.09786 Which puts the bulk of that 48% higher claim on "The average carbon intensity of the US data centers in our study (weighted by the energy they consumed) was 548 grams of CO2e per kilowatt hour (kWh), approximately 48% higher than the US national average of 369 gCO2e / kWh (26)." Which points to https://ourworldindata.org/grapher/carbon-intensity-electricity?tab=chart&time=latest..2023&country=EU-27~KHM~OWID_UMC~USA https://ourworldindata.org/grapher/carbon-intensity-electric... For the average of 369g/KWh. That's close enough to the figure in the table at https://www.epa.gov/system/files/documents/2024-01/egrid2022_summary_tables.pdf https://www.epa.gov/system/files/documents/2024-01/egrid2022... which shows 375g/KWh (after converting from lb/MWh) But the table they compare against shows. VA 576g/KWh TX 509g/KWh CA 374g/KWh and the EPA table shows VA 268g/KWh TX 372g/KWh CA 207g/KWh Which seem more likely to be true. The paper has California at only marginally better than the national average for renewables (Which I guess they needed to support their argument given the number of data centers there) I like arxiv, It's a great place to see new ideas, the fields I look at have things that I can test myself to see if the idea actually works. I would not recommend it as a source of truth. Peer review still has a place. If they were gathering emissions data from states themselves, they should have caclulated the average from that data, not pulled the average from another potentially completely different measure. Then their conclusions would have been valid regardless what weird scaling factor they bought in to their state calculations. The numbers might have been wrong but the proportion would have been accurate, and it is the proportion that is being highlighted.

GuinansEyebrows 1y ago

SoftTalker 1y ago

If you are old enough you remember posting to Usenet and the warning that would accompany each new submission: This program posts news to thousands of machines throughout the entire civilized world. Your message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. Are you absolutely sure that you want to do this? [ny] Maybe we meed something similar in LLM clients. Could be phrased in terms of how many pounds of atmospheric carbon the request will produce.

mirekrusin 1y ago
Or gigantic footer explaining you should consider impact before trying to print this email – which was always eating next page when printed.
wmf 1y ago
Only if that same warning is attached to literally everything else you do. It's unfair to single out AI for using energy.
- SoftTalker 1y ago
  But "asking ChatGPT" is something people do so casually without there being any apparent clues of the costs associated with that. I guess that is true of pretty much everything online though. Even driving your car around you at least are somewhat aware of the gas you are burning.
  - asdff 1y ago
    That could be solved by charging more for the service. That is the only reason you are aware of the gas burning after all, you aren't conducting your own aq test you are noticing you are filling up twice a week at $50 a tank.
  - appreciatorBus 1y ago
    I would bet most ppl drive around with very little awareness of how much it’s costing, either in money or environmental impact. Many people I’ve met seem to measure efficiency by how much it costs to fill up the tank.

teekert 1y ago

I wonder what the carbon footprint of all those ads is.

amelius 1y ago
Not just the ads, but also the overconsumption which they cause.
- __MatrixMan__ 1y ago
  Not just overconsumption, but also waste due to supply chain fragility. If you can induce demand anywhere then supply has to do crazy things to keep up.

sfpityparty 1y ago

[flagged]

Ericson2314 1y ago

We really need to improve the power grid. I don't think about "A. I." very much, but I am glad that something is making us upgrade the grid.

lucb1e 1y ago
> I am glad that something is making us upgrade the grid A few big consumers in centralized locations isn't changing the grid as much as the energy transition from fuels to electricity is
- Ericson2314 1y ago
  The transition to electric vehicles in the US has been disappointing, to say the least.
  - lucb1e 1y ago
    I thought everyone over there has an air conditioning system on electricity?
    - Ericson2314 1y ago
      Yes, but we already do. We need new electricity demand.

acidburnNSA 1y ago

I'm glad that the needs of AI and the sustainable capabilities of nuclear fission go well together.

simonw 1y ago

I found this bit interesting: > In 2017, AI began to change everything. Data centers started getting built with energy-intensive hardware designed for AI, which led them to double their electricity consumption by 2023. As we all know, the generative AI boom only really kicked into high gear in November 2022 with ChatGPT. That's five years of "AI" growth between 2017 and 2022 which presumably was mostly not generative AI.

SoftTalker 1y ago
People started using GPUs for ML at around that time.
mattnewton 1y ago
2017 is the year after AlphaGo beats Lee Sedol and is when the attention is all you need paper was published. The writing was on the wall. OpenAI just found product market fit in november 2022, but the industry wasn't wandering aimlessly until then.
NitpickLawyer 1y ago
Meta was in the process of AI-ing everything on their stacks, with search, similarity, graphs, recommendation, etc. everywhere on their properties. Incidentally that's why they already had tens/hundreds of thousands of GPUs when the LLM craze hit, and why they were in a good place to work on llama and other stuff.

phillipcarter 1y ago

Well, this was disappointing: > There is a significant caveat to this math. These numbers cannot serve as a proxy for how much energy is required to power something like ChatGPT 4o. Otherwise this is an excellent article critiquing the very real problem that is opacity of these companies regarding model sizes and deployments. Not having an honest accounting of computing deployed worldwide is a problem, and while it's true that we didn't really do this in the past (early versions of Google searches were undoubtedly inefficient!), it's not an excuse today. I also wish this article talked about the compute trends. That is, compute per token is going significantly down, but that also means use of that compute can spread more. Where does that lead us?

blkhawk 1y ago

the numbers in the article are all over the place. I mean the article seems to try and some of the more general calculations on paper should work out but especially the image gen ones I can sorta disprove with my own experiences in local gen. Even were it matches sorta (the 400 feet e-bike thing) that only works out for me because I use an AMD card. An NVIDIA card can have several times the generation speed at the same power draw so it all falls down again. And the parameters they tried to standardize their figures with (the 1024x1024 thing) is also a bit meh because the SAME amount of pixels in a different aspect ratio can have huge variations in gen speed and thus power usage. for instance for most illustrious type checkpoints the speed is about 60% higher at aspect ratios other than 1024x1024. Its all a bit of a mess.

bbor 1y ago

Interesting, thanks for sharing! I share some concerns others have about this piece, but I’m most shocked about their finding that image generation is cheaper than text. As someone who’s gone down this rabbit hole multiple times, this runs against every single paper I’ve ever cited on the topic. Anyone know why? Maybe this is a recent change? It also doesn’t help that multimodal transformers are now blurring the lines between image and text, of course… this article doesn’t even handle that though, treating all image models as diffusion models.

fred69 1y ago

Might have missed it but was disappointed to see no mention of externalized costs like the scraping burden imposed on every IP-connected server. From discussions on HN this sounds quite substantial. And again, why exactly should the few AI companies reap all the value when other companies and individuals are incurring costs for it?

Henchman21 1y ago

I work in DCO, thats Data Center Operations if you’re not aware. I’ve tried explaining the amount of power used to my elderly mom; it isn’t easy! But here’s my best take: The racks I am personally responsible for consume 17.2kW. That’s consistent across the year; sure things dip a bit when applications are shut down, but in general 17.2kW is the number. Presuming a national average of 1.2kW per home, each rack of equipment I oversee could potentially power 14 houses. I am responsible for hundreds of these racks, while my larger organization has many thousands of these racks in many locations worldwide. I’ve found no other way to let the scale of this sink in. When put this way she is very clear: the price isn’t worth it to humanity. Being able to get, say, Door Dash, is pretty neat! But not at the cost of all our hoarded treasure and certainly not at the cost of the environment on the only planet we have access to. The work done by AI will only ever benefit the people at the top. Because to be frank: they won’t share. Because the very wealthy have hoarding disorder.

jeffbee 1y ago
It seems like you are having an emotional response to not understanding the general energy picture. For example, an A320 aloft uses the same energy as two thousand of your hypothetical racks (2.5 tons of kerosene per hour). Each! We are in no meaningful sense torching the biosphere to get AI.
- globnomulous 1y ago
  > It seems like you are having an emotional response to not understanding the general energy picture. This is condescending and rude. It also strikes me as obviously wrong. The 'response' isn't emotional in the slightest; it just explains the emotional and cognitive experience of acquiring understanding. It's a reasonable, well-reasoned explanation of the difficulty of intuitively grasping how much energy these data centers, and thus the companies that use them, consume -- and then of the shock many experience when it dawns on them. > For example, an A320 aloft uses the same energy as two thousand of your hypothetical racks (2.5 tons of kerosene per hour). > Each! > We are in no meaningful sense torching the biosphere to get AI. What exactly is the reasoning here? That airplanes use a lot of energy, just like data centers or compared to data centers, and therefore that AI isn't ecologically damaging -- or isn't "meaningfully" damaging, whatever that means? That's not just wrong. It's a nonsequitur. There's a simpler, easier, better way to wrap our heads around the data, one that doesn't require false dichotomies (or something like that): humans are torching the biosphere both to get AI and to travel.

floxy 1y ago

40% of electricity consumption in Virgina will be data centers in 2030? Table A1 , PDF page 29: https://www.epri.com/research/products/000000003002028905 https://www.epri.com/research/products/000000003002028905

coolcase 1y ago
Virginia, us-east-2, U.S.A. (P.S. check your spelling!)
- floxy 1y ago
  Whoops!

deleted 1y ago

[deleted]

djoldman 1y ago

> This leaves even those whose job it is to predict energy demands forced to assemble a puzzle with countless missing pieces, making it nearly impossible to plan for AI’s future impact on energy grids and emissions. Worse, the deals that utility companies make with the data centers will likely transfer the costs of the AI revolution to the rest of us, in the form of higher electricity bills. ... So don't? Explicitly shift the cost to the customer. If I want to hook up to the energy grid with 3-phase power, I pay the utility to do it. If a business wants more power and it isn't available, then the business can pay for it. Then only businesses that really need it will be willing to step up to the plate. No amount of "accounting" or "energy needs prediction" will guard against regulatory capture.

cco 1y ago

Today Google launched a model, Gemma 3n, that performs about as good as SOTA models from 1-2 years ago that runs locally on a cell phone. Training SOTA models will, like steel mills or other large industrial projects, require a lot of environmental footprint to produce. But my prediction is that over time the vast majority of use cases in the hands of users will be essentially run on device and be basically zero impact, both in monetary cost and environment.

jeffbee 1y ago

This series of articles is driving me insane. The authors or editors are using inappropriate units to shock readers: billions of gallons, millions of square feet. But they are not putting the figures into context that the reader can directly comprehend. Because if they said the Nevada data centers would use 2% as much water as the hay/alfalfa industry in Nevada then the entire article loses its shock value.

panstromek 1y ago
I agree. Even first few paragraphs strike me as intentionaly misleading. The whole "AI energy saga" seems to be full of manipulative claims. I can't help to feel that the intent is just to color the AI as bad in whatever way possible. It feels like it's deriving the research from predetermined conclusions

protocolture 1y ago

Weird I was assured that Bitcoin would be using all of the worlds electricity by now. Which I already thought was odd, because London would need all that electricity to see through the giant mountain of poop piled up by all the horses the british use for transportation.

gitroom 1y ago

[dead]

scottcha 1y ago

Shameless plug . . . I run a startup who is working to help this https://neuralwatt.com https://neuralwatt.com We are starting with an os level (as in no model changes/no developer changes required) component which uses RL to run AI with a ~25% energy efficiency improvement w/out sacrificing UX. Feel free to dm me if you are interested in chatting either about problems you face with energy and ai or if you'd like to learn more.

mNovak 1y ago

This gives me the silly idea to go try to measure the power consumption of the local data center by measuring the magnetic field coming off the utility lines.

Quenby 1y ago

[dead]

teleforce 1y ago

> unprecedented and comprehensive look at how much energy the AI industry uses Not sure about comprehensive claim here if end-to-end query chains were not considered. For example the mobile wireless node (that're being used by the majority of the users) contribution to the energy consumption are totally ignored. The wireless power amplifier or PA for both sides of users and base-stations are notorious for their inefficiency being only less than than 50% in practice although in theory can be around 80%. Almost all of the current AI applications are cloud based not local-first thus the end users energy consumption and contribution are necessary.

neves 1y ago

Best article I've ever read about the energy needs of AI. Impressive how Big Tech refuses to share data with society for collective decisions. I'd also recommend the Data Vampires podcast series: https://techwontsave.us/episode/241_data_vampires_going_hyperscale_episode_1 https://techwontsave.us/episode/241_data_vampires_going_hype... https://techwontsave.us/episode/243_data_vampires_opposing_data_centers_episode_2 https://techwontsave.us/episode/243_data_vampires_opposing_d... https://techwontsave.us/episode/245_data_vampires_sacrificing_for_ai_episode_3 https://techwontsave.us/episode/245_data_vampires_sacrificin... https://techwontsave.us/episode/247_data_vampires_fighting_for_control_episode_4 https://techwontsave.us/episode/247_data_vampires_fighting_f...

johnny_________ 1y ago

I ponder this a lot, but the interface of "MIT technology Review" is unbearably overdesigned, its got that annoying narrow smartphone format where you can't zoom out, and then all these fancy graphics. Can't we have crisp, easy-to-read HTML? The format annoyed me so much I didn't read the article because this kind of design makes me doubt the source. Alas

myaccountonhn 1y ago
The article works fine with the browser's reader-mode for me.

paulcole 1y ago

Are these the same people who claimed that crypto was going to use more energy than the entire world by 2030?

kleiba 1y ago
This reminds me of the quote: "Félix L'Herbier learns that there are more links in his brain than atoms in the universe."

LordDragonfang 1y ago

> The new model uses [energy] equivalent to riding 38 miles on an e-bike... AI companies have defended these numbers saying that generative video has a smaller footprint than the film shoots and travel that go into typical video production. That claim is hard to test and doesn’t account for the surge in video generation that might follow if AI videos become cheap to produce. "Hard to test", but very obviously true if you make any attempt at guessing based on making a few assumptions... like they seem comfortable doing for all the closed source models they don't have access to being run in conditions they're not testing for. Especially considering they're presenting their numbers as definitive, and then just a couple paragraphs down admit that, yeah, they're just guessing. Regardless, I know for a fact that a typical commercial shoot uses way more energy than driving across the TMZ in an e-bike (considering they're definitely using cars to transport gear, which gives you less than 4 miles for the same energy).

thedevilslawyer 1y ago

What's the energy imprint of the human doing the same work that AI is now going to do? If AI's imprint is less, what is the right thing for us to do?

ahtihn 1y ago
Kill the human? If you don't want to go there, it doesn't really matter how much energy the human uses because the human will just use the same energy to do something else.
- zavec 1y ago
  Not necessarily. I think the point of comparison here is how much energy does AI use to e.g. generate a video, compared to the energy used not by the human themselves, but by running XYZ software on a computer with a beefy graphics card for however many hours it'd take a human to do the same work.
  - thedevilslawyer 1y ago
    Indeed. Everything taken into account - work getting done takes energy. And if an agent can do a task for less energy than a human, then indeed it's a benefit. This would be the apt comparison, instead of looking at some overall datacenter energy consumption.
  - shmeeed 1y ago
    While that's a valid point of view, as long as the human is the bottleneck, it's not going to scale to infinity and beyond. Human's got to exist and needs to work to eat. They don't really, necessarily, existentially need to be 10x productive with the help of AI. But I'll be honest, that's not really a solid argument, because it could rapidly lead to the question of why they do this exact job in the first place, instead of e.g. farming or whatever else there might be that can be called a net positive for humanity without reservations.
  - HPsquared 1y ago
    Or a team going to a location and filming, etc.

xeox538 1y ago

I believe we're currently seeing AI in the "mainframe" era, much like the early days of computing, where a single machine occupied an entire room and consumed massive amounts of power, yet offered less compute than what now fits in a smartphone. I expect rapid progress in both model efficiency and hardware specialization. Local inference on edge devices, using chips designed specifically for AI workloads, will drastically reduce energy consumption for the majority of tasks. This shift will free up large-scale compute resources to focus on truly complex scientific problems, which seems like a worthwhile goal to me.

panstromek 1y ago
It sure seems like that to me. I was pretty impressed by how easily I could run small Gemma on 7 year old laptop and get a decent chat experience. I can imagine that doing some clever offloading to a normal programs and using the LLM as a sort of "fuzzy glue" for the rest could improve the efficiency on many common tasks.
- coolcase 1y ago
  Big tech ain't investing heavily so you can run local, what data does that leave them to sell, and what power and control does that give them. Zilch.
  - panstromek 1y ago
    I mean.. cute conspiracy but it doesn't correspond with reality. Just look what's Google releasing, they are trying to make these things fit on consumer hardware.
whatnow37373 1y ago
The CPU development curve is often thrown around but it very seldomly fits anything else in reality. It was a very rare and extraordinary set of coincidences that got it us here. Computation using silicon turned out to have massive growth potential for a variety of lucky reasons but say battery tech is not so lucky, nor is fusion nor is quantum computing. The low hanging fruit has been plucked by said silicon development process and while remarkable improvement in AI efficiency is likely it is highly unlikely for that to follow a similar curve. More likely is slow, incremental process taking decades. We cannot just wish away billions of parameters and the need for trillions of operations. It’s not like we have some open path of possible improvement like with silicon. We walked that path already. Maybe photonics..

tempfile 1y ago

Jesus, who writes this stuff? > AI is unavoidable > We will speak to models in voice mode, chat with companions for 2 hours a day, and point our phone cameras at our surroundings in video mode This is surely meant to be an objective assessment, not a fluff piece.

dr_dshiv 1y ago

Solving climate change will take a lot of energy. I found this article to be a little too one sided. For instance, it didn’t talk about the 10x reductions in power achieved this past year — essentially how gpt4 can now run on a laptop. Viz, via sama “The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use. You can see this in the token cost from GPT-4 in early 2023 to GPT-4o in mid-2024, where the price per token dropped about 150x in that time period. Moore’s law changed the world at 2x every 18 months; this is unbelievably stronger.” https://blog.samaltman.com/three-observations https://blog.samaltman.com/three-observations

Juliate 1y ago
It's still _additional_ power usage that 1/ did not exist before 2/ does not replace/reduce previous/other power (some, very much more critical and essential) usages. 3/ a LOT of tasks are still way more energy/time-efficiently done with regular existing methods (dedicated software, or even by hand), but are still asked/improperly routed to AI chatbots that ... statistically guess the answer.
- dr_dshiv 1y ago
  It is also additional value creation. It also leads to automation and efficiency, even if it isn’t a fully linear path. AI isnt a waste. We can’t let environmental consciousness get in the way of rather natural human development. Especially CO2. (I have different opinions about biodiversity because of the irreversible loss. I also believe that we have the technology to pause and reverse climate change — but don’t pursue it because of degrowth ideologies)
  - t_tsonev 1y ago
    No amount of value, in the economic sense, will solve climate change.
    - SirHumphrey 1y ago
      Well, somebody must buy solar panels, batteries, (nuclear reactors) and finance the whole research process. While “economic value” =/= “solving climate change” without enough tax revenue costly transitions are impossible.

panstromek 1y ago

There's a certain irony here in the fact that this page is maxing out my CPU on idle, doing some unclear work in javascript, while I'm just reading text.

guappa 1y ago
I would like browsers to have very strict limits on the amount of cycles a page can use, and have some permission like for camera to use more CPU.
- jve 1y ago
  What a nice idea.
  - tessellated 1y ago
    top joke!
- sligor 1y ago
  I would like also that devs run their site/apps on downgraded machine/VM so that they can early detect performance issues.
  - guappa 1y ago
    In my company the frontend team has shiny new powerful machines. I on the other hand don't have enough RAM to compile our server.
    - coolcase 1y ago
      Does that create a sort of Conway's law pushing complexity to the FE?
      - guappa 1y ago
        They offset that risk by taking 2 months to design and implement complex features like "add a new text box in the form, to enter an IP address". I haven't seen them merge a pull request in less than 3 days. However simple. The team lead, in 2 years, contributed less than 20 pull requests, all of which were 1 to 3 line changes to CSS or similar.

liendolucas 1y ago

> ...and many buildings use millions of gallons of water (often fresh, potable water) per day in their cooling operations. This is outrageous. People still struggle to access fresh water (and power), but hey "sustainability is all to our company" is always promoted as if something nice is being done on from the behemoth's sides. BS. What a waste of resources. I truly condemn all this. To this day I do still refuse to use any of this technology and hope that all this ends in the near future. It's madness. I see this as nothing more than next-gen restrictive lousy search engines, and as many have pointed out ads are going to roll soon. The more people adopt it the worse will be for everyone. I always emphasize this: 10-15 years ago I could find everything through simple web searches. Everything. In many cases even landing on niche and unexpected but useful and interesting websites. Today that is a difficult/impossible task. Perhaps there is still room for a well-done traditional search engine (haven't tried Kagi but people in general do say nice things about it) to surface and take the lead but I doubt it, when hype arrives especially in the tech industry people follow blindly. There are still flourishing "ai" startups and from night to day everyone has become a voice or expert on the subject. Again: BS. Traditional web engines and searches were absolutely just fine and quite impressed with their outputs. I remember it. What the heck has happened?

bogtog 1y ago
> ...and many buildings use millions of gallons of water (often fresh, potable water) per day in their cooling operations. Of note, cooling is water evaporation, so the water will inevitably come back to us as good as new. This contrasts uses that will actually pollute water

bogtog 1y ago

> The largest model we tested has 405 billion parameters, but others, such as DeepSeek, have gone much further, with over 600 billion parameters. Very quickly skimming, I have some trouble taking this post seriously when it omits that the larger DeepSeek one is a mixture-of-experts that will only use 12.5% (iirc) of its components for each token. The best summary of text energy use I've seen is this (seemingly more rigorous, although its estimates are consistent with the final numbers made by the present post): epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use Estimates for a given response widely for a "typical" query (0.3 Wh; 1080 joules) and a maximal-context query (40 Wh; 144k joules). Assuming most uses don't come close to maximizing the context, the energy use of text seems very small compared to the benefits. That being said, the energy use for video generation seems substantial I would be interested in seeing the numbers corresponding to how LLMs are typically used for code generation

James_K 1y ago

The fact that none of these companies want to tell you how much power they're using should be enough to reason that it's utterly horrible. If it were anywhere reasonable, they'd be boasting about how low the number is.

janaagaard 1y ago

- "4.4% of all the energy in the US now goes toward data centers" - "by 2028 [...] AI alone could consume as much electricity annually as 22% of all US households." What would the 22% be if compared against all US energy instead of just all US household?

EdiX 1y ago
US households constitute 21% of all energy use [1], so 22% of this is 0.21 * 0.22, which is 4%. [1] https://rpsc.energy.gov/energy-data-facts https://rpsc.energy.gov/energy-data-facts

southernplaces7 1y ago

The number of people in this comment thread defending this gargantuan energy footprint for a technology that currently in large measure is being used for a tremendous amount of dogshit things (and oceans of visual/text spam) is amusing considering the hysterics that this same energy use problem caused when it came to crypto. I guess it becomes okay when the companies guzzling the energy are some of the biggest tech employers in the world, buttering your bread in some way.

admiralrohan 1y ago
Good that you connected it with crypto boom. But problem is it's hard to rewind these kind of trends once started. Human nature.
- pjc50 1y ago
  NFTs are dying and link rotting. ICOs are out of fashion. VR is back in its niche. 3DTV .. I've not seen that marketed for quite a while now. Things that are fads will die out eventually. Crypto meanwhile is in an odd space: everyone knows blockchains are too much of a hassle and that the volatility is too high, so they're using centralized exchanges and "stablecoins" (shadow dollars). There's still a huge amount of money there but not quite as much as the stadium ads / FTX peak.
  - alnwlsn 1y ago
    I wish you could still buy 3D TVs. Why? We used them to build a truck simulator which had a driver and passenger. The view out the window is different for each person. They were perfect for that, but it's too small a niche to keep making new TVs.
    - ben_w 1y ago
      Have you considered DIY-ing it with a normal TV/monitor and a lenticular lens?
  - dragonwriter 1y ago

deleted 1y ago

[deleted]

polski-g 1y ago

Seems like a solved problem. We have an unlimited sources of clean power from nuclear and solar. China alone now has almost 1TW of solar capacity. Build more nuclear, build more solar. Tax carbon.

troyvit 1y ago

> Tech companies like Meta, Amazon, and Google have responded to this fossil fuel issue by announcing goals to use more nuclear power. Those three have joined a pledge to triple the world’s nuclear capacity by 2025. Erm ... that's a weird date considering this article came out yesterday. They actually pledge to triple the world's nuclear capacity by 2050[1] There are a couple of weird things like that in this article, including the classic reference to "experts" for some of its data points. Still ... at least somebody's trying to quantify this. [1] https://www.world-nuclear-news.org/articles/amazon-google-meta-and-dow-back-tripling-nuclear-goal https://www.world-nuclear-news.org/articles/amazon-google-me...

neves 1y ago
The weirdest thing in the article is the refusal of Big Tech to release this data. We wouldn't experts to guess. Society must have information to make decisions about what affects all the world.
WillPostForFood 1y ago
Do you think it is weird or just a typo?
- troyvit 1y ago
  Heh, when I first started typing it I thought it was weird, then I discovered it was a typo, but didn't edit my first sentence. Weird huh?

palmfacehn 1y ago

From "The Limits to Growth" to peak oil and beyond: As a rule, scarcity doom and neo-malthusianism hasn't played out the way proponents claim. I've been very critical of the AI hype cycle here and elsewhere, but this isn't it. In the long run technological advancements increase our productivity and quality of life. Yes, much of what is being promoted is slop. Yes, this bubble is driven by an overly financialized economy. That doesn't preclude the possibility of AI models precipitating meaningful advancements in the human condition. From refrigeration to transportation, cheap and abundant energy has been one of the major driving forces in human advancement. Paradoxically, consuming cheap energy doesn't reduce the amount of energy available on the market. Instead it increases the size of the market.

thedudeabides5 1y ago

long silver for solar

pizzuh 1y ago

Energy use is going to continue to climb. I'm worried about the environmental impacts of this, but from everything I've seen society values model output more. Curious to watch this over the rest of the decade.

jes5199 1y ago
solar and battery costs are falling so fast that emissions are likely to go down, even with growth in energy use
- edhelas 1y ago
  Sure sure, you did the maths ? How many batteries do you need to power-up a datacenter at night ?
  - Filligree 1y ago
    Not that many, if you pair it with offshore wind, a large grid (which covers multiple time zones), and ideally nuclear. The conditions for that don’t exist everywhere, but we don’t need these datacentres everywhere. LLMs aren’t latency-sensitive.
    - p_j_w 1y ago
      >if you pair it with offshore wind Unfortunately the current government of the US is entirely beholden to fossil fuel interests so they're actively hostile to offshore wind.
  - jes5199 1y ago
    Tony Seba did the maths: https://www.rethinkx.com/publications/understandingstellarenergy2025.en https://www.rethinkx.com/publications/understandingstellaren...
- pizzuh 1y ago
  Great point. Assuming we can adopt them at the correct pace and on a global scale, then energy doesn't have to be a limiting factor

some_random 1y ago

We shouldn't be hand wringing on energy usage, we should be generating energy too cheap to bother metering and too clean to bother thinking about. Nuclear baseline, renewables everywhere they make sense, we know how to do this we just need to do it.

airylizard 1y ago

As more and more people use brute force loops to make their AI agents more reliable, this hidden inference giant will only continue to grow. This is why I put my framework together, using just 2 passes as opposed to n+ can increase accuracy and reliability by a far greater amount than brute force loops while using significantly less resources. Supportive evidence and data can be found in my repo: https://github.com/AutomationOptimization/tsce_demo https://github.com/AutomationOptimization/tsce_demo

benlivengood 1y ago

I find it weird that on the whole there wasn't as huge a pushback against AWS and Google and Facebook (by far the biggest fleets of data centers) for power and carbon emissions over the last couple decades. Maybe they managed propaganda better about paying for future renewable resources or how low their PUEs are. Turning 4% of the U.S.'s electricity into cat videos and online shopping and advertisements and heat already sounds like a lot of use. Maybe the rapid rise of AI use is what's alarming people? Contrast the 2016 study[0] of data center energy use where use was recently flat because of efficiency improvements in 2010-2020 but historically there was a ton of growth in energy consumption since ~1990; basically we have always been on a locally exponential growth curve in data center energy use but our constant factors were being optimized by the hyperscalers in that 2010-2020 period. We also need to compare the efficiency of AI with other modes of computation/work. The article goes into detail on the supposed actual energy use but there's a simple metric; All the large companies provide costs per unit of inference which can put a hard ceiling on actual energy cost. Something like $20/1M tokens for the best models. METR used a 2M token budget. So you can currently price out N hours of work at $40 from whichever latest METR benchmarks come out and have a worst case cost for efficiency comparison. Lastly, if we're not on a trend toward having Dyson swarms of compute in the long run then what are we even doing as a species? Of course energy spent on compute is going to grow quadratically into the distant future as we expand. People are complaining about compute for AI but compute is how we figure things out and get things done. AI is the latest tool. [0] https://eta.lbl.gov/publications/united-states-data-center-energy https://eta.lbl.gov/publications/united-states-data-center-e...

insane_dreamer 1y ago

It's hard to believe that as a society we are prioritizing AI queries (which are fine, nothing wrong with them in principle, and they do increase productivity in some cases) over the future wellbeing of our species (and planet). Winter is Coming, and these big companies and their investors couldn't care less. I can't decide whether they think it won't be that bad, or the scientific forecasts are wrong, or that they just don't care because whatever turmoil results from serious climate change, they'll be able to rise above it, and well, fuck the rest of humanity. I'm disgusted by our sense of priorities. (Though maybe I shouldn't be since we live in a country that values subsidizing the industrial-military complex over the health and education of its citizens.)