5 ms·

Yet we also see that hyperscale cloud emissions targets have been reversed due to AI investment, Datacenter growth is hitting grid capacity limits in many regio

Yet we also see that hyperscale cloud emissions targets have been reversed due to AI investment, Datacenter growth is hitting grid capacity limits in many regions, and peaker plant and other non-renewable resources on the grid are being deployed more to handle this specific growth from AI. I think the author, by qualifying on "chatgpt" maybe can make the claims they are making but I don't believe the larger argument would hold for AI as a whole or when you convert the electricity use to emissions. I'm personally on the side that the ROI will probably work out in the long run but not by minimizing the potential impact and keeping the focus on how we can make this technology (currently in its infancy) more efficient. [edit wording]

seanmcdirmid 2y ago
Is that true though? Data centers can be placed anywhere in the USA, they could be placed near a bunch of hydro or wind farm resources in the western grid which has little coal anyways outside of one line from Utah to socal. The AI doesn’t have to be located anywhere near to where it is used since fiber is probably easier to run than a high voltage power line.
- wmf 2y ago
  That was already done years ago and people are predicting that the grid will be maxed out soon.
  - seanmcdirmid 2y ago
    Build new data centers near sources of power, and grid capacity isn’t going to be a problem. Heck, American industry used to follow that (building garment factories on fast moving rivers before electricity was much of a thing, Boeing grew up in the northwest due to cheap aluminum helped out by hydro). Why is AI somehow different from an airplane?
    - wmf 2y ago
      They'll have to build new power generation and build the data centers next to it.
    - sitkack 2y ago
      You are massively conflating what is possible with what is done.
      - seanmcdirmid 2y ago
        I know data centers are built next to wind farms for these reasons already. We have an abundance of those projects out here in the PNW.
- scottcha 2y ago
  There are a large number of reasons the AI datacenters are geographically distributed--just to list a few off the top of my head which come up as top drivers: latency, data sovereignty, resilience, grid capacity, renewable energy availability.
  - Karrot_Kream 2y ago
    Why does latency matter for a model that responds in 10s of seconds? Latency to a datacenter is measured in 10s or 100s of milliseconds, which is 3-4 orders of magnitude less.
    - scottcha 2y ago
      Two reasons that I understand 1. not all these AIs are LLMs and many have much lower latency SLAs than chat and 2. These are just one part of a service architecture and when you have multiple latencies across the stack they tend to have multiplicative effects.
    - recursivecaveat 2y ago
      If you look at a model with a diverse competitive provider set like llama 3 the latency is 1/4 second, and it will definitely improve at a minimum incrementally if quality is held constant: https://artificialanalysis.ai/models/llama-3-3-instruct-70b/providers#latency https://artificialanalysis.ai/models/llama-3-3-instruct-70b/... Remember that as long as you experience the response linearly (very much the case for audio output for eg) then the first-chunk latency is your actual latency, not to stream the entire response.
- fulafel 2y ago
  The root problem there is that fossil energy is very cheap and the state sponsors production of fossil fuels. Consequently the energy incentives are weak and other concerns take priority. This is coupled with low public awareness, many people don't understand the moral problem in using fossils so the PR penalty from a fossil powered data center is low.
wmf 2y ago
Voluntary conservation was only working by accident and guilt tripping never works. The grid needs to become clean so that we can have new industries.
- fmbb 2y ago
  The grid being clean means not having any fossil power. We can only get there by shutting down all fossil fuel power plants. We can not get there by adding new power generation.
  - celticninja 2y ago
    Well you need the latter to replace the former. So you need to add new power generation to allow you to shut down fossil fuel plants. And to be honest what we need to do is replace them with nuclear power stations to manages the base load of nations power requirements. Either that or much better power storage is required
  - caseyy 2y ago
    > We can only get there by shutting down all fossil fuel power plants. We can not get there by adding new power generation. https://knowyourmeme.com/photos/1433498-no-take-only-throw https://knowyourmeme.com/photos/1433498-no-take-only-throw "No add new power plants, only transform our grid to greener".
- XorNot 2y ago
  Yep, this is the real answer. It's also the only answer. The big fiction was everyone getting hopped on the idea that "karma" was going to be real, and people's virtue would be correctly identified by overt environmentalism rather then action. Fossil fuel companies won, and they won in about 1980s when BP paid an advertising firm to come up with "personal carbon footprint" as a meaningful metric. Basically destroyed environmentalism since...well I'll let you know when it stops.
  - andymasley 2y ago
    I made a point in the post to say that it's better to mostly ignore your personal carbon footprint and focus on systematic change, but that I was writing the post for people who still wanted to reduce their consumption anyway
YetAnotherNick 2y ago
Why do you belive this? Datacenter uses just a 1-1.3 percent of electricity from grid and even if you suppose AI increased the usage by 2x(which I really doubt), the number would still be tiny. Also AI training is easiest workload to regulate, as you can only train when you have cheaper green energy.
- kolinko 2y ago
  I also had doubts, but asked chat and it confirms it’s an issue - including sources. https://chatgpt.com/share/678b6b3e-9708-8009-bcad-8ba84a51456f https://chatgpt.com/share/678b6b3e-9708-8009-bcad-8ba84a5145... The issue is that they are often localised, so even if it’s just 1% of power, it can cause issues. Still, by itself, grid issues don’t mean climate issues. And any argument complaining about a co2 cost should also consider alternative cost to be reliable. Even if ai was causing 1% or 2% or 10% of energy use, the real question is how much it saves by making society more efficient. And even if it wasn’t, it’s again more of a question about energy companies polluting with co2. Microsoft, which hosts OpenAI, is famously amazing in terms of their co2 emissions - so far they were going way beyond what other companies were doing.
  - YetAnotherNick 2y ago
    What do you mean by confirms the issue? What's the issue exactly?
    - oneplane 2y ago
      The issue is that when you have a high local usage your grid loses the ability to respond to peaks since that capacity is now always in use. Essentially it raises the baseline use which means your elasticity is pretty much gone. A grid isn't a magic battery that is always there, it is constantly fluctuating, regardless of the intent of producers and consumers. You need to be able to have enough elasticity to deal with that fact. Changing that is hard (and expensive), but it is the only way (such as the technical reality). The solution is not to create say, 1000 extra coal-fired generating facilities since you can't really turn them on or off at will. Same goes for gas, nuclear etc. You'd need a few of them for your baseline load (combined with other sources like solar, wind, hydro, whatever) and then make sure you have your non-renewable sources have margin and redundancy and use storage for the rest. This was always the case, and it will always be the case. But now with information technology, the degree to which you can permanently raise demand on the grid to an extreme degree is where the problem becomes much more apparent. And because it's not manufacturing (which is an extreme consumer of energy) you don't really get the "run on lower output" option. You can't have an LLM do "just a little bit of inferencing". Just like you can't have your Netflix send only half a movie to "save power". In the past we had the luxury of nighttime lower demand which means industry could up their usage, but datacenters don't sleep at night. And they also can't wait for batch processing during the day.

seanmcdirmid 2y ago

Is that true though? Data centers can be placed anywhere in the USA, they could be placed near a bunch of hydro or wind farm resources in the western grid which has little coal anyways outside of one line from Utah to socal. The AI doesn’t have to be located anywhere near to where it is used since fiber is probably easier to run than a high voltage power line.

wmf 2y ago
That was already done years ago and people are predicting that the grid will be maxed out soon.
- seanmcdirmid 2y ago
  Build new data centers near sources of power, and grid capacity isn’t going to be a problem. Heck, American industry used to follow that (building garment factories on fast moving rivers before electricity was much of a thing, Boeing grew up in the northwest due to cheap aluminum helped out by hydro). Why is AI somehow different from an airplane?
  - wmf 2y ago
    They'll have to build new power generation and build the data centers next to it.
  - sitkack 2y ago
    You are massively conflating what is possible with what is done.
    - seanmcdirmid 2y ago
      I know data centers are built next to wind farms for these reasons already. We have an abundance of those projects out here in the PNW.
scottcha 2y ago
There are a large number of reasons the AI datacenters are geographically distributed--just to list a few off the top of my head which come up as top drivers: latency, data sovereignty, resilience, grid capacity, renewable energy availability.
- Karrot_Kream 2y ago
  Why does latency matter for a model that responds in 10s of seconds? Latency to a datacenter is measured in 10s or 100s of milliseconds, which is 3-4 orders of magnitude less.
  - scottcha 2y ago
    Two reasons that I understand 1. not all these AIs are LLMs and many have much lower latency SLAs than chat and 2. These are just one part of a service architecture and when you have multiple latencies across the stack they tend to have multiplicative effects.
  - recursivecaveat 2y ago
    If you look at a model with a diverse competitive provider set like llama 3 the latency is 1/4 second, and it will definitely improve at a minimum incrementally if quality is held constant: https://artificialanalysis.ai/models/llama-3-3-instruct-70b/providers#latency https://artificialanalysis.ai/models/llama-3-3-instruct-70b/... Remember that as long as you experience the response linearly (very much the case for audio output for eg) then the first-chunk latency is your actual latency, not to stream the entire response.
fulafel 2y ago
The root problem there is that fossil energy is very cheap and the state sponsors production of fossil fuels. Consequently the energy incentives are weak and other concerns take priority. This is coupled with low public awareness, many people don't understand the moral problem in using fossils so the PR penalty from a fossil powered data center is low.

wmf 2y ago

Voluntary conservation was only working by accident and guilt tripping never works. The grid needs to become clean so that we can have new industries.

fmbb 2y ago
The grid being clean means not having any fossil power. We can only get there by shutting down all fossil fuel power plants. We can not get there by adding new power generation.
- celticninja 2y ago
  Well you need the latter to replace the former. So you need to add new power generation to allow you to shut down fossil fuel plants. And to be honest what we need to do is replace them with nuclear power stations to manages the base load of nations power requirements. Either that or much better power storage is required
- caseyy 2y ago
  > We can only get there by shutting down all fossil fuel power plants. We can not get there by adding new power generation. https://knowyourmeme.com/photos/1433498-no-take-only-throw https://knowyourmeme.com/photos/1433498-no-take-only-throw "No add new power plants, only transform our grid to greener".
XorNot 2y ago
Yep, this is the real answer. It's also the only answer. The big fiction was everyone getting hopped on the idea that "karma" was going to be real, and people's virtue would be correctly identified by overt environmentalism rather then action. Fossil fuel companies won, and they won in about 1980s when BP paid an advertising firm to come up with "personal carbon footprint" as a meaningful metric. Basically destroyed environmentalism since...well I'll let you know when it stops.
- andymasley 2y ago
  I made a point in the post to say that it's better to mostly ignore your personal carbon footprint and focus on systematic change, but that I was writing the post for people who still wanted to reduce their consumption anyway

YetAnotherNick 2y ago

Why do you belive this? Datacenter uses just a 1-1.3 percent of electricity from grid and even if you suppose AI increased the usage by 2x(which I really doubt), the number would still be tiny. Also AI training is easiest workload to regulate, as you can only train when you have cheaper green energy.

kolinko 2y ago
I also had doubts, but asked chat and it confirms it’s an issue - including sources. https://chatgpt.com/share/678b6b3e-9708-8009-bcad-8ba84a51456f https://chatgpt.com/share/678b6b3e-9708-8009-bcad-8ba84a5145... The issue is that they are often localised, so even if it’s just 1% of power, it can cause issues. Still, by itself, grid issues don’t mean climate issues. And any argument complaining about a co2 cost should also consider alternative cost to be reliable. Even if ai was causing 1% or 2% or 10% of energy use, the real question is how much it saves by making society more efficient. And even if it wasn’t, it’s again more of a question about energy companies polluting with co2. Microsoft, which hosts OpenAI, is famously amazing in terms of their co2 emissions - so far they were going way beyond what other companies were doing.
- YetAnotherNick 2y ago
  What do you mean by confirms the issue? What's the issue exactly?
  - oneplane 2y ago
    The issue is that when you have a high local usage your grid loses the ability to respond to peaks since that capacity is now always in use. Essentially it raises the baseline use which means your elasticity is pretty much gone. A grid isn't a magic battery that is always there, it is constantly fluctuating, regardless of the intent of producers and consumers. You need to be able to have enough elasticity to deal with that fact. Changing that is hard (and expensive), but it is the only way (such as the technical reality). The solution is not to create say, 1000 extra coal-fired generating facilities since you can't really turn them on or off at will. Same goes for gas, nuclear etc. You'd need a few of them for your baseline load (combined with other sources like solar, wind, hydro, whatever) and then make sure you have your non-renewable sources have margin and redundancy and use storage for the rest. This was always the case, and it will always be the case. But now with information technology, the degree to which you can permanently raise demand on the grid to an extreme degree is where the problem becomes much more apparent. And because it's not manufacturing (which is an extreme consumer of energy) you don't really get the "run on lower output" option. You can't have an LLM do "just a little bit of inferencing". Just like you can't have your Netflix send only half a movie to "save power". In the past we had the luxury of nighttime lower demand which means industry could up their usage, but datacenters don't sleep at night. And they also can't wait for batch processing during the day.