4 ms·

> creating this garbage consumes staggering amounts of water and electricity, contributing to emissions that harm the planet This is highly dependent on which

by Dwedit 1y ago

> creating this garbage consumes staggering amounts of water and electricity, contributing to emissions that harm the planet This is highly dependent on which model is being used and what hardware it's running on. In particular, some older article claimed that the energy used to generate an image was equivalent to charging a mobile phone, but the actual energy required for a single image generation (SDXL, 25 steps) is about 35 seconds of running a 80W GPU.

samplatt 1y ago
35 seconds @ 80W is ~210 mAh, so definitely a lot less than the ~4000+ mAh in today's phone batteries.
- serial_dev 1y ago
  Don’t you ignore the energy used to train the models? I don’t know how much is that “per image”, but it should be included (and if it shouldn’t, we should know why it is negligible). I’m not sure it will be as high as a full charge of a phone, but it’s incomplete without the resources needed for collecting data and training the model.
- red369 1y ago
  I'm going to expose my ignorance here, but I thought mAh/Ah was not a good measure for comparing storage of quite different devices, because it doesn't take into account voltage. This is fine for comparing Li-ion devices, because they use the same voltage, but I understood that using watt-hours was therefore more appropriate for apples-to-apples comparisons for devices with different wattages. Am I missing something? Does the CPU/GPU/APU doing this calculation on servers/PCs run the same wattage as mobile devices?
  - Gigachad 1y ago
    No you are completely right. mAh is a unit of current over time. Not power. The proper unit is watt hours.
- quickthrowman 1y ago
  Current over time (ampere hours) and power over time (watt hours) are two different things. You should be using KWh.
th0ma5 1y ago
An ideal measurement would be to calculate your utility's water usage to kWh then to this, perhaps a token per gram measurement. Of course it would be small, but it should be calculable and then directly comparable to these models if we try to go by something like token per gram of water. I suspect due to DC power distribution they may be more efficient in the data center. You could get more specific about the water too, recycled vs evaporated etc etc
a2128 1y ago
Nobody's running SDXL on an 80W GPU when they're talking about generating images, and you also have to take into account training and developing SDXL or the relevant model. AI companies are spending a lot of resources on training, trying various experiments, and lately they've become a lot more secretive when it comes to reporting climate impact or even any details about their models (how big is ChatGPT's image generation model compared to SDXL? how many image models do they even have?)
numpad0 1y ago
IIRC some of those researches you're taking estimates from not only used cherry picked figures for AI image generators, but also massively underestimated man-hour costs of human artists by using commission prices and market labor rates without requisite corroboration works before choosing those values. Their napkin math went like, human artists take $50 or so per art, which is let's say $200/hr skill, which means each art cannot take longer than 10 minutes, therefore the values used for AI must add up to less than 10 workstation minutes, or something like that. And that math is equally broken for both sides: SDXL users easily spend hours rolling dice a hundred times without usable images, and likewise, artists just easily spend a day or two for an interesting request that may or may not come with free chocolates. So those estimates are not only biased, but basically entirely useless.
visarga 1y ago
I did a little investigation. Turns out that GPT-4's training consumes as much energy as 300 cars in their lifetime, which comes about 50 GWh. Not really that much, could be just families on a short street burning that kind of energy. As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour. If you compare datacenter energy usage to the rest, it amounts to 5%. Making great economies on LLMs won't save the planet.
- lelanthran 1y ago
  > As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour. This can't be correct, I'd like to see how this was measured. Running a GPU at full throttle for one hour uses less power than serving data for one hour? I'm very sceptical.
  - visarga 1y ago
    An hour of Netflix streaming consumes approximately 77 Wh according to IEA analysis showing streaming a Netflix video in 2019 typically consumed around 0.077 kWh of electricity per hour [1], while an hour of active GPT-4 chatting (assuming 20 queries at 0.3 Wh each) consumes roughly 6 Wh based on Epoch AI's estimate that a single query to GPT-4o consumes approximately 0.3 watt-hours per query [2]. That makes Netflix about 13 times more energy-intensive than LLM usage. [1] https://www.iea.org/commentaries/the-carbon-footprint-of-streaming-video-fact-checking-the-headlines https://www.iea.org/commentaries/the-carbon-footprint-of-str... [2] https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use https://epoch.ai/gradient-updates/how-much-energy-does-chatg...
    - lelanthran 1y ago
      Jesus Christ, what a poor take on those numbers! It's possible to have a more wrong interpretation, but not by much. The Netflix consumption takes into account everything[1], the numbers for AI are only the GPU power consumption, not including the user's phone/laptop. IOW, you are comparing the power cost of using a datacenter + global network + 55" TV to the cost of a single 1shot query (i.e. a tiny prompt) on the GPU only Once again, I am going to say that the power cost of serving up a stored chunk of data is going to be less than the power cost of first running a GPU and then serving up that chunk. ================== [1] Which (in addition to the consumption by netflix data centers) includes the network equipment in between, the computer/TV on the user's end. Consider that the user is watching netflix on a TV (min 100w, but more for a 60" large screen).
lelanthran 1y ago
> the actual energy required for a single image generation (SDXL, 25 steps) is about 35 seconds of running a 80W GPU. And just how many people manage to 1shot the image? There are maybe 5 to 20 images generated before the user is happy.
- blharr 1y ago
  I mean, 80 W is not a ton even if you're generating images constantly. How many people leave the lights on at home?
  - lelanthran 1y ago
    > I mean, 80 W is not a ton even if you're generating images constantly. Compared to what? > How many people leave the lights on at home? What does that have to do with this?

by Dwedit 1y ago

samplatt 1y ago

35 seconds @ 80W is ~210 mAh, so definitely a lot less than the ~4000+ mAh in today's phone batteries.

serial_dev 1y ago
Don’t you ignore the energy used to train the models? I don’t know how much is that “per image”, but it should be included (and if it shouldn’t, we should know why it is negligible). I’m not sure it will be as high as a full charge of a phone, but it’s incomplete without the resources needed for collecting data and training the model.
red369 1y ago
I'm going to expose my ignorance here, but I thought mAh/Ah was not a good measure for comparing storage of quite different devices, because it doesn't take into account voltage. This is fine for comparing Li-ion devices, because they use the same voltage, but I understood that using watt-hours was therefore more appropriate for apples-to-apples comparisons for devices with different wattages. Am I missing something? Does the CPU/GPU/APU doing this calculation on servers/PCs run the same wattage as mobile devices?
- Gigachad 1y ago
  No you are completely right. mAh is a unit of current over time. Not power. The proper unit is watt hours.
quickthrowman 1y ago
Current over time (ampere hours) and power over time (watt hours) are two different things. You should be using KWh.

th0ma5 1y ago

An ideal measurement would be to calculate your utility's water usage to kWh then to this, perhaps a token per gram measurement. Of course it would be small, but it should be calculable and then directly comparable to these models if we try to go by something like token per gram of water. I suspect due to DC power distribution they may be more efficient in the data center. You could get more specific about the water too, recycled vs evaporated etc etc

a2128 1y ago

Nobody's running SDXL on an 80W GPU when they're talking about generating images, and you also have to take into account training and developing SDXL or the relevant model. AI companies are spending a lot of resources on training, trying various experiments, and lately they've become a lot more secretive when it comes to reporting climate impact or even any details about their models (how big is ChatGPT's image generation model compared to SDXL? how many image models do they even have?)

numpad0 1y ago

IIRC some of those researches you're taking estimates from not only used cherry picked figures for AI image generators, but also massively underestimated man-hour costs of human artists by using commission prices and market labor rates without requisite corroboration works before choosing those values. Their napkin math went like, human artists take $50 or so per art, which is let's say $200/hr skill, which means each art cannot take longer than 10 minutes, therefore the values used for AI must add up to less than 10 workstation minutes, or something like that. And that math is equally broken for both sides: SDXL users easily spend hours rolling dice a hundred times without usable images, and likewise, artists just easily spend a day or two for an interesting request that may or may not come with free chocolates. So those estimates are not only biased, but basically entirely useless.

visarga 1y ago

I did a little investigation. Turns out that GPT-4's training consumes as much energy as 300 cars in their lifetime, which comes about 50 GWh. Not really that much, could be just families on a short street burning that kind of energy. As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour. If you compare datacenter energy usage to the rest, it amounts to 5%. Making great economies on LLMs won't save the planet.

lelanthran 1y ago
> As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour. This can't be correct, I'd like to see how this was measured. Running a GPU at full throttle for one hour uses less power than serving data for one hour? I'm very sceptical.
- visarga 1y ago
  An hour of Netflix streaming consumes approximately 77 Wh according to IEA analysis showing streaming a Netflix video in 2019 typically consumed around 0.077 kWh of electricity per hour [1], while an hour of active GPT-4 chatting (assuming 20 queries at 0.3 Wh each) consumes roughly 6 Wh based on Epoch AI's estimate that a single query to GPT-4o consumes approximately 0.3 watt-hours per query [2]. That makes Netflix about 13 times more energy-intensive than LLM usage. [1] https://www.iea.org/commentaries/the-carbon-footprint-of-streaming-video-fact-checking-the-headlines https://www.iea.org/commentaries/the-carbon-footprint-of-str... [2] https://epoch.ai/gradient-updates/how-much-energy-does-chatgpt-use https://epoch.ai/gradient-updates/how-much-energy-does-chatg...
  - lelanthran 1y ago
    Jesus Christ, what a poor take on those numbers! It's possible to have a more wrong interpretation, but not by much. The Netflix consumption takes into account everything[1], the numbers for AI are only the GPU power consumption, not including the user's phone/laptop. IOW, you are comparing the power cost of using a datacenter + global network + 55" TV to the cost of a single 1shot query (i.e. a tiny prompt) on the GPU only Once again, I am going to say that the power cost of serving up a stored chunk of data is going to be less than the power cost of first running a GPU and then serving up that chunk. ================== [1] Which (in addition to the consumption by netflix data centers) includes the network equipment in between, the computer/TV on the user's end. Consider that the user is watching netflix on a TV (min 100w, but more for a 60" large screen).

lelanthran 1y ago

> the actual energy required for a single image generation (SDXL, 25 steps) is about 35 seconds of running a 80W GPU. And just how many people manage to 1shot the image? There are maybe 5 to 20 images generated before the user is happy.

blharr 1y ago
I mean, 80 W is not a ton even if you're generating images constantly. How many people leave the lights on at home?
- lelanthran 1y ago
  > I mean, 80 W is not a ton even if you're generating images constantly. Compared to what? > How many people leave the lights on at home? What does that have to do with this?