4 ms·

> environmental impact of a single average prompt (generating 400 tokens' worth of text, or about a page's worth) was relatively minimal: just 1.14 grams of CO2

by M4v3R 1y ago

> environmental impact of a single average prompt (generating 400 tokens' worth of text, or about a page's worth) was relatively minimal: just 1.14 grams of CO2 emitted and 45 milliliters of water consumed While it’s non-zero, it doesn’t strike me to be “hurting the planet” as some people would want me to believe I’m doing when I decide to use LLMs. Yes, the training has a much bigger impact but the benefits of training are shared will all users and it’s a one-time cost per model. I did the math and if I’m right the environmental footprint of a single LLM training, emitting 13,600 metric tons of CO2 and consuming 187,333 cubic meters of water annually, represents 0.000026% of global greenhouse gas emissions and 0.0000047% of freshwater use.

zekrioca 1y ago
How many trainings and retraining are happening as we speak? How many more when the transition from millions of jobs replaced by AI do you expect? What about inference across all of that?
- M4v3R 1y ago
  > How many trainings and retraining are happening as we speak? Quite a lot. Let's assume a hundred different LLMs of this scale are being trained at the same time. If you multiply the global use percentages by a hundred you'll get: 0.0026% of global greenhouse gas emissions and 0.00047% of freshwater use. Still a literal drop in the bucket. > How many more when the transition from millions of jobs replaced by AI do you expect? Dunno, but the argument is that I should feel bad about my current impact on the environment as I use my LLM to autocomplete my code or answer my questions. We have no idea what the future will hold. We can and of course should do everything to minimize the environmental impact of everything we do, but that's a different discussion. For example switching to clean energy sources will make a big positive impact on these numbers. > What about inference across all of that? The report speaks about that, the inference cost in marginal when compared to the training cost (~15% for CO2 and ~9% for water consumption).
  - zekrioca 1y ago
    > Quite a lot. Let's assume a hundred different LLMs of this scale are being trained at the same time. It won’t be 100, you are underestimating it to make the number be small, ignoring the fact that people are talking about GW worth of continuous power, not counting the refresh rate of GPUs (every 3-5 years the whole infrastructure is renewed). > Dunno, but the argument is that I should feel bad about my current impact on the environment as I use my LLM to autocomplete my code or answer my questions. That’s not the argument. The argument is that you should be aware of your consumption and therefore the impact it has. Right now people use everything as a ‘’dumb’’ magical API that just spits things out from nowhere with no impacts. > The report speaks about that, the inference cost in marginal when compared… Don’t ignore how many of these are happening as we speak. ChatGPT went from 0 to 100mi users within months, all submitting hundreds of queries.
    - M4v3R 1y ago
      > It won’t be 100, you are underestimating it to make the number be small Make it a 1000 (I seriously doubt there are one thousand simultaneous training runs of Mistral Large 2 scale models going on every second) and it's still a drop in the bucket. > not counting the refresh rate of GPUs (every 3-5 years the whole infrastructure is renewed I am accounting for this by citing annual usage instead of one-time cost.
      - zekrioca 1y ago
        Not sure what you think demand is, but operators are building 10 GW AI datacenters. Assuming a GPU consumes ~1 kW, the number is potentially (upper bound) 10 GW / 1 kW, way larger than ‘1000’. For one company.
        lostmsu 1y ago
        Still drop in the bucket considering world total electricity production is about 10 TW. Where did you read one company? I found 10 GW new capacity next year for the entire industry.

by M4v3R 1y ago

zekrioca 1y ago

How many trainings and retraining are happening as we speak? How many more when the transition from millions of jobs replaced by AI do you expect? What about inference across all of that?

M4v3R 1y ago
> How many trainings and retraining are happening as we speak? Quite a lot. Let's assume a hundred different LLMs of this scale are being trained at the same time. If you multiply the global use percentages by a hundred you'll get: 0.0026% of global greenhouse gas emissions and 0.00047% of freshwater use. Still a literal drop in the bucket. > How many more when the transition from millions of jobs replaced by AI do you expect? Dunno, but the argument is that I should feel bad about my current impact on the environment as I use my LLM to autocomplete my code or answer my questions. We have no idea what the future will hold. We can and of course should do everything to minimize the environmental impact of everything we do, but that's a different discussion. For example switching to clean energy sources will make a big positive impact on these numbers. > What about inference across all of that? The report speaks about that, the inference cost in marginal when compared to the training cost (~15% for CO2 and ~9% for water consumption).
- zekrioca 1y ago
  > Quite a lot. Let's assume a hundred different LLMs of this scale are being trained at the same time. It won’t be 100, you are underestimating it to make the number be small, ignoring the fact that people are talking about GW worth of continuous power, not counting the refresh rate of GPUs (every 3-5 years the whole infrastructure is renewed). > Dunno, but the argument is that I should feel bad about my current impact on the environment as I use my LLM to autocomplete my code or answer my questions. That’s not the argument. The argument is that you should be aware of your consumption and therefore the impact it has. Right now people use everything as a ‘’dumb’’ magical API that just spits things out from nowhere with no impacts. > The report speaks about that, the inference cost in marginal when compared… Don’t ignore how many of these are happening as we speak. ChatGPT went from 0 to 100mi users within months, all submitting hundreds of queries.
  - M4v3R 1y ago
    > It won’t be 100, you are underestimating it to make the number be small Make it a 1000 (I seriously doubt there are one thousand simultaneous training runs of Mistral Large 2 scale models going on every second) and it's still a drop in the bucket. > not counting the refresh rate of GPUs (every 3-5 years the whole infrastructure is renewed I am accounting for this by citing annual usage instead of one-time cost.
    - zekrioca 1y ago
      Not sure what you think demand is, but operators are building 10 GW AI datacenters. Assuming a GPU consumes ~1 kW, the number is potentially (upper bound) 10 GW / 1 kW, way larger than ‘1000’. For one company.
      - lostmsu 1y ago
        Still drop in the bucket considering world total electricity production is about 10 TW. Where did you read one company? I found 10 GW new capacity next year for the entire industry.