5 ms·

Generative AI's environmental costs are soaring – and mostly secret

38 points by alexzeitler 3y ago

claytongulick 3y ago
This has felt a lot like the early days of the streaming wars to me, where business model took a back seat to user acquisition. I've been deeply curious about about costs vs revenue with the "AI everywhere" push. Local models seem like the way?
- jsheard 3y ago
  Local models from where though? As you say the industry is currently in a money-burning phase where user acquisition and growing mindshare takes priority over short term profits, and that's where most of the decent quality local models are coming from, companies like Stability and Mistral giving away expensive models for free. That's obviously not a sustainable long term plan for local AI, we're already seeing Mistral pivot away from releasing their models. Where is the money to train open/local models going to come from once this initial hype cycle is over?
Havoc 3y ago
While true on this particular topic I'm inclined to ignore it in a pure accelerationism sense. [not convinced on accelerationism as a whole] If this works out everything experiences a sea change...including the environmental calculus. So I'm quite wary of "it uses a lot of energy" arguments on this one. Other stuff like aluminium smelters use a lot of energy too but to me this one is philosophically different. Even at 50% odds or whatever...if this could change everything we have to go for it, even at env impact
- dougb5 3y ago
  Can you say more about what you mean by "50% odds" here?
  - Havoc 3y ago
    There is no science behind the 50%. I used it as a proxy for uncertainty. Nothing particularly scientific about that 50% odds...it was intended purely to capture the sentiment that "AI will change everything"...or maybe that's just hot air. My point is that in that uncertain context I'd lean towards all-in on AI despite cost.
- jeremyjh 3y ago
  I'm not sure how ChatGPT being used by half a billion people to generate blog spam is going to contribute to AI going FOOM.
  - CuriouslyC 3y ago
    It's all part of a giant GAN algorithm. People generate a sea of blogspam, some of it gets selected for and condensed because it's accidentally good, the rest is used as a training set for a garbage detection algorithm, which drives the spammers to produce better spam ad nauseum. AGI will be born on the back of people trying to dodge spam filters.
    - bugbuddy 3y ago
      The logical jumps in your thinking are so wild. How could people exercise such insane mental gymnastics while keeping a straight face? Are all human intellectual work so easily subsumed into spam filtering system? This is a clear sign of mania. Just like the crypto mania, people just could not think straight to save their lives.
  - silverquiet 3y ago
    It sounds a bit like religion, doesn't it? I keep seeing people who want to free these models from the constraints that Google and OpenAI place on these things, and I wonder, what do they think they'll learn from them? Some hidden truth or meaning? I doubt it's lurking there.
    - dwaltrip 3y ago
      They’re just useful tools that are likely to become even more so.
- lm28469 3y ago
  Do you pray for every major gods every day? Because even at 50% odds or whatever it could change everything
  - Havoc 3y ago
    I would if I saw plausible indicators that they exist
throwaway22032 3y ago
Sometimes I feel as if environmentalists don't really understand the concept of entropy. You can't do anything without externality. It's literally impossible. The framing of energy usage as being environmentally costly is tautological. But there's a fundamental difference between turning on a light bulb and using some energy in the process vs. bulldozing a forest, burning a load of coal, etc etc. The method matters.
- dheera 3y ago
  However, I don't think we're anywhere even remotely within several orders of magnitude of entropy limits in the energy usage we use for AI models. I'm pretty convinced it shouldn't take several hundred megawatts to produce something like GPT-4. The human brain does it much more efficiently, and even that isn't near entropy limits.
- lm28469 3y ago
  Do you see a difference between burning a lake of petrol vs using it to power a billion car for a month? It's not just about using energy, it's about what you do with it
- ripjaygn 3y ago
  > You can't do anything without externality. It's literally impossible. The framing of energy usage as being environmentally costly is tautological. You're completely missing the concept of efficiency. Take the light bulb you mentioned, a filament bulb turns a very small fraction of energy used to light and the rest to waste heat. An LED is the opposite. That's why environmentalists successfully campaigned against filament bulbs instead of just shrugging saying 'entropy', 'energy usage', and 'tautological'. It also matters if the process releases greenhouse gases or not, because those have multiplicative effects on climate change.
deleted 3y ago
[deleted]
jncfhnb 3y ago
Looks like a play by Altman to regulate away the ability of others to train models
CrypticShift 3y ago
I keep hearing of AI photonic chips and how energy efficient (and faster) they are. This could be THE solution ? How far away are we?
nealabq 3y ago
A typical fission nuclear power plant produces about a gigawatt of power. About 8.76 terawatt hours (TWh) every year. According to https://www.eia.gov/todayinenergy/detail.php?id=61364 https://www.eia.gov/todayinenergy/detail.php?id=61364 humanity used about 27400 TWh in 2023, with bitcoin mining using in the range of 67 to 240 TWh. So we're using roughly half-a-percent of global electricity (~140 TWh) on bitcoin. That's about the 16 fission reactors. If we decide that AI is as worthwhile as bitcoin then we should be willing to invest that much electricity. Maybe the U.A.E. is thinking they could provide those power plants.
mmoskal 3y ago
Training GPT-4 is estimated at 50GWh [0]. Driving a pickup truck consumes about 30MWh per year in gasoline (13500 miles/year, 15 mpg, 33kWh/gallon). So, training GPT-4 is equivalent to driving 1000-2000 trucks for a year, or in other words gasoline consumption of small American town. Doesn't seem very high for a product used by 180m people. [0] https://www.ri.se/en/news/blog/generative-ai-does-not-run-on-thin-air https://www.ri.se/en/news/blog/generative-ai-does-not-run-on...
- malphite 3y ago
  Wow, I didn’t realize GPT-4 usage was so high. Where did you read the DAU numbers?
  - mmoskal 3y ago
    First hit on google [0] - no idea if it's true but seems plausible... (half a year ago sama said 100m) [0] https://explodingtopics.com/blog/chatgpt-users https://explodingtopics.com/blog/chatgpt-users
qgin 3y ago
The environmental costs of rejecting nuclear power are soaring -- and mostly getting blamed on end users.

38 points by alexzeitler 3y ago

This has felt a lot like the early days of the streaming wars to me, where business model took a back seat to user acquisition. I've been deeply curious about about costs vs revenue with the "AI everywhere" push. Local models seem like the way?

jsheard 3y ago
Local models from where though? As you say the industry is currently in a money-burning phase where user acquisition and growing mindshare takes priority over short term profits, and that's where most of the decent quality local models are coming from, companies like Stability and Mistral giving away expensive models for free. That's obviously not a sustainable long term plan for local AI, we're already seeing Mistral pivot away from releasing their models. Where is the money to train open/local models going to come from once this initial hype cycle is over?

Havoc 3y ago

While true on this particular topic I'm inclined to ignore it in a pure accelerationism sense. [not convinced on accelerationism as a whole] If this works out everything experiences a sea change...including the environmental calculus. So I'm quite wary of "it uses a lot of energy" arguments on this one. Other stuff like aluminium smelters use a lot of energy too but to me this one is philosophically different. Even at 50% odds or whatever...if this could change everything we have to go for it, even at env impact

dougb5 3y ago
Can you say more about what you mean by "50% odds" here?
- Havoc 3y ago
  There is no science behind the 50%. I used it as a proxy for uncertainty. Nothing particularly scientific about that 50% odds...it was intended purely to capture the sentiment that "AI will change everything"...or maybe that's just hot air. My point is that in that uncertain context I'd lean towards all-in on AI despite cost.
jeremyjh 3y ago
I'm not sure how ChatGPT being used by half a billion people to generate blog spam is going to contribute to AI going FOOM.
- CuriouslyC 3y ago
  It's all part of a giant GAN algorithm. People generate a sea of blogspam, some of it gets selected for and condensed because it's accidentally good, the rest is used as a training set for a garbage detection algorithm, which drives the spammers to produce better spam ad nauseum. AGI will be born on the back of people trying to dodge spam filters.
  - bugbuddy 3y ago
    The logical jumps in your thinking are so wild. How could people exercise such insane mental gymnastics while keeping a straight face? Are all human intellectual work so easily subsumed into spam filtering system? This is a clear sign of mania. Just like the crypto mania, people just could not think straight to save their lives.
- silverquiet 3y ago
  It sounds a bit like religion, doesn't it? I keep seeing people who want to free these models from the constraints that Google and OpenAI place on these things, and I wonder, what do they think they'll learn from them? Some hidden truth or meaning? I doubt it's lurking there.
  - dwaltrip 3y ago
    They’re just useful tools that are likely to become even more so.
lm28469 3y ago
Do you pray for every major gods every day? Because even at 50% odds or whatever it could change everything
- Havoc 3y ago
  I would if I saw plausible indicators that they exist

throwaway22032 3y ago

Sometimes I feel as if environmentalists don't really understand the concept of entropy. You can't do anything without externality. It's literally impossible. The framing of energy usage as being environmentally costly is tautological. But there's a fundamental difference between turning on a light bulb and using some energy in the process vs. bulldozing a forest, burning a load of coal, etc etc. The method matters.

dheera 3y ago
However, I don't think we're anywhere even remotely within several orders of magnitude of entropy limits in the energy usage we use for AI models. I'm pretty convinced it shouldn't take several hundred megawatts to produce something like GPT-4. The human brain does it much more efficiently, and even that isn't near entropy limits.
lm28469 3y ago
Do you see a difference between burning a lake of petrol vs using it to power a billion car for a month? It's not just about using energy, it's about what you do with it
ripjaygn 3y ago
> You can't do anything without externality. It's literally impossible. The framing of energy usage as being environmentally costly is tautological. You're completely missing the concept of efficiency. Take the light bulb you mentioned, a filament bulb turns a very small fraction of energy used to light and the rest to waste heat. An LED is the opposite. That's why environmentalists successfully campaigned against filament bulbs instead of just shrugging saying 'entropy', 'energy usage', and 'tautological'. It also matters if the process releases greenhouse gases or not, because those have multiplicative effects on climate change.

deleted 3y ago

[deleted]

jncfhnb 3y ago

Looks like a play by Altman to regulate away the ability of others to train models

CrypticShift 3y ago

I keep hearing of AI photonic chips and how energy efficient (and faster) they are. This could be THE solution ? How far away are we?

nealabq 3y ago

A typical fission nuclear power plant produces about a gigawatt of power. About 8.76 terawatt hours (TWh) every year. According to https://www.eia.gov/todayinenergy/detail.php?id=61364 https://www.eia.gov/todayinenergy/detail.php?id=61364 humanity used about 27400 TWh in 2023, with bitcoin mining using in the range of 67 to 240 TWh. So we're using roughly half-a-percent of global electricity (~140 TWh) on bitcoin. That's about the 16 fission reactors. If we decide that AI is as worthwhile as bitcoin then we should be willing to invest that much electricity. Maybe the U.A.E. is thinking they could provide those power plants.

mmoskal 3y ago

Training GPT-4 is estimated at 50GWh [0]. Driving a pickup truck consumes about 30MWh per year in gasoline (13500 miles/year, 15 mpg, 33kWh/gallon). So, training GPT-4 is equivalent to driving 1000-2000 trucks for a year, or in other words gasoline consumption of small American town. Doesn't seem very high for a product used by 180m people. [0] https://www.ri.se/en/news/blog/generative-ai-does-not-run-on-thin-air https://www.ri.se/en/news/blog/generative-ai-does-not-run-on...

malphite 3y ago
Wow, I didn’t realize GPT-4 usage was so high. Where did you read the DAU numbers?
- mmoskal 3y ago
  First hit on google [0] - no idea if it's true but seems plausible... (half a year ago sama said 100m) [0] https://explodingtopics.com/blog/chatgpt-users https://explodingtopics.com/blog/chatgpt-users

qgin 3y ago

The environmental costs of rejecting nuclear power are soaring -- and mostly getting blamed on end users.