13 ms·

Of course. At current trend its going to consume ridiculous amount of electricity for no good reason (and make some more CO2). Funny that 60W old school light b

by hexo 2y ago

Of course. At current trend its going to consume ridiculous amount of electricity for no good reason (and make some more CO2). Funny that 60W old school light bulb is bad bad bad, but 400W gfx card is ok and armies of 3kW servers to produce bs responses to queries are also ok. Consuming ridiculous kilowatts to produce even more bs video is also "no problemo". Hm. Poor artists, I can see why they're furious about this.

Ukv 2y ago
> At current trend its going to consume ridiculous amount of electricity for no good reason (and make some more CO2). 5 joules per token[0] * 200 tokens per query * 10 queries per day * 365 days per year = 1.014kWh Which, going by the US's energy mix[1], means a year of someone's LLM usage would be about 0.4kg of CO2 - the same as a single cup of coffee[2]. Worth double-checking, since these are just napkin calculations and I could have made a large mistake. If it is correct (within an order of magnitude or two), that seems to me a relatively small amount of energy. Obviously still expensive to provide for free to hundreds of millions of users, as almost anything would be. [0]: https://arxiv.org/pdf/2310.03003 https://arxiv.org/pdf/2310.03003 [1]: https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 [2]: https://www.co2everything.com/co2e-of/coffee https://www.co2everything.com/co2e-of/coffee
- hexo 2y ago
  I've read somewhere that half of googles datacenter energy consumption is ai. So I think the real numbers are probably somewhere else. Its not just about one query - its about everything that come with it, including production of hw including price manipulation (that means we [better say companies] have to produce more money [ie. make some more CO2] to even afford the hw in the first place) and soon also energy plants construction - and we can only hope thats not a coal one. Also, there's that pesky training that is hugely expensive on electricity. Oh dont get me wrong - there are good uses of neural networks, but the stuff that pushed so much is not that. An example of good one was nvidias sw (rtx voice) to clear up voice during calls, dunno if it exists or not.
  - Ukv 2y ago
    > I've read somewhere that half of googles datacenter energy consumption is ai I can't find that figure, or what's being included in it: major Google products like Search and Translate are AI (since at least ~2018, depending on your definition of AI) and have huge numbers of users - I think it'd be hard to deny their utility. > Its not just about one query [...] To be clear, I didn't calculate that "one query" was equivalent to a cup of coffee - it was a year's worth of someone's LLM usage. > Also, there's that pesky training that is hugely expensive on electricity. I don't believe the coffee comparison took into consideration the one-off costs either, like manufacturing of your coffee machine, but let's consider the amortized emissions of training anyway: 500 tons of CO2[0] over 200,000,000 users[1] means 0.0025kg of CO2 per user, barely shifting from the figure of inference alone. Again, rough calculation - would be lower per-user if taking into account API/3rd-party users, but then higher taking into account more model versions. Realistically, I don't think someone's CO2 emissions on the order of 0.4kg, whether that's coffee, video streaming/rendering, LLM usage, or so on, should take so much of the focus while we have, for instance, celebrities emitting 8,000,000kg a year[2] from private jets. [0]: https://foundation.mozilla.org/en/blog/ai-internet-carbon-footprint/ https://foundation.mozilla.org/en/blog/ai-internet-carbon-fo... [1]: https://a16z.com/how-are-consumers-using-generative-ai/ https://a16z.com/how-are-consumers-using-generative-ai/ [2]: https://i.imgur.com/EP0Gbtk.jpeg https://i.imgur.com/EP0Gbtk.jpeg
- leereeves 2y ago
  Both 200 tokens per query and 10 queries per day seem far too low.
  - Ukv 2y ago
    This entire article is under 500 tokens, and I think over the course of a year most people will balance out occasional long conversations with days where they don't touch it at all. Can tweak the numbers, but even if you think - say - that the average response length is 2000 tokens (~8000 characters), a year of usage would still just be on the level of a few days' worth of coffee for me. Unless I've made some major error in the calculations, ~0.4kg just doesn't seem like all that much when in the same timeframe a celebrity will be emitting 8,000,000kg from their private jet.
    - mysterydip 2y ago
      Sounds like the solution is to replace the celebrity with an AI :)
    - deleted 2y ago
      [deleted]
    - leereeves 2y ago
      I see that the paper that reported 5J per token used LLaMA 65B, a relatively small model. ChatGPT 4 reportedly has trillions of parameters They also didn't measure power use vs context window size, and used single prompts that would be much smaller than conversations (because in a conversation with an LLM, every prompt entered sends the entire conversation to the LLM again.) Or the much larger context windows of RAG, or programmers sending entire source code archives with every prompt. As for the number of interactions, a professional using the tool forty hours a week to support their work could easily have a hundred conversations per week (hundreds of prompts total). There are enough factors here with enough uncertainty that the result could be off by a lot. That said, I agree that it's unlikely to be worse than a celebrity with a private jet. I can't take anyone who discusses climate change seriously if they don't demand banning private jets.
karmakurtisaani 2y ago
> Funny that 60W old school light bulb is bad bad bad, but 400W gfx card is ok Could be because there actually exists a 10x more efficient alternative for the light bulb.

by hexo 2y ago

Ukv 2y ago

> At current trend its going to consume ridiculous amount of electricity for no good reason (and make some more CO2). 5 joules per token[0] * 200 tokens per query * 10 queries per day * 365 days per year = 1.014kWh Which, going by the US's energy mix[1], means a year of someone's LLM usage would be about 0.4kg of CO2 - the same as a single cup of coffee[2]. Worth double-checking, since these are just napkin calculations and I could have made a large mistake. If it is correct (within an order of magnitude or two), that seems to me a relatively small amount of energy. Obviously still expensive to provide for free to hundreds of millions of users, as almost anything would be. [0]: https://arxiv.org/pdf/2310.03003 https://arxiv.org/pdf/2310.03003 [1]: https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 https://www.eia.gov/tools/faqs/faq.php?id=74&t=11 [2]: https://www.co2everything.com/co2e-of/coffee https://www.co2everything.com/co2e-of/coffee

hexo 2y ago
I've read somewhere that half of googles datacenter energy consumption is ai. So I think the real numbers are probably somewhere else. Its not just about one query - its about everything that come with it, including production of hw including price manipulation (that means we [better say companies] have to produce more money [ie. make some more CO2] to even afford the hw in the first place) and soon also energy plants construction - and we can only hope thats not a coal one. Also, there's that pesky training that is hugely expensive on electricity. Oh dont get me wrong - there are good uses of neural networks, but the stuff that pushed so much is not that. An example of good one was nvidias sw (rtx voice) to clear up voice during calls, dunno if it exists or not.
- Ukv 2y ago
  > I've read somewhere that half of googles datacenter energy consumption is ai I can't find that figure, or what's being included in it: major Google products like Search and Translate are AI (since at least ~2018, depending on your definition of AI) and have huge numbers of users - I think it'd be hard to deny their utility. > Its not just about one query [...] To be clear, I didn't calculate that "one query" was equivalent to a cup of coffee - it was a year's worth of someone's LLM usage. > Also, there's that pesky training that is hugely expensive on electricity. I don't believe the coffee comparison took into consideration the one-off costs either, like manufacturing of your coffee machine, but let's consider the amortized emissions of training anyway: 500 tons of CO2[0] over 200,000,000 users[1] means 0.0025kg of CO2 per user, barely shifting from the figure of inference alone. Again, rough calculation - would be lower per-user if taking into account API/3rd-party users, but then higher taking into account more model versions. Realistically, I don't think someone's CO2 emissions on the order of 0.4kg, whether that's coffee, video streaming/rendering, LLM usage, or so on, should take so much of the focus while we have, for instance, celebrities emitting 8,000,000kg a year[2] from private jets. [0]: https://foundation.mozilla.org/en/blog/ai-internet-carbon-footprint/ https://foundation.mozilla.org/en/blog/ai-internet-carbon-fo... [1]: https://a16z.com/how-are-consumers-using-generative-ai/ https://a16z.com/how-are-consumers-using-generative-ai/ [2]: https://i.imgur.com/EP0Gbtk.jpeg https://i.imgur.com/EP0Gbtk.jpeg
leereeves 2y ago
Both 200 tokens per query and 10 queries per day seem far too low.
- Ukv 2y ago
  This entire article is under 500 tokens, and I think over the course of a year most people will balance out occasional long conversations with days where they don't touch it at all. Can tweak the numbers, but even if you think - say - that the average response length is 2000 tokens (~8000 characters), a year of usage would still just be on the level of a few days' worth of coffee for me. Unless I've made some major error in the calculations, ~0.4kg just doesn't seem like all that much when in the same timeframe a celebrity will be emitting 8,000,000kg from their private jet.
  - mysterydip 2y ago
    Sounds like the solution is to replace the celebrity with an AI :)
  - deleted 2y ago
    [deleted]
  - leereeves 2y ago
    I see that the paper that reported 5J per token used LLaMA 65B, a relatively small model. ChatGPT 4 reportedly has trillions of parameters They also didn't measure power use vs context window size, and used single prompts that would be much smaller than conversations (because in a conversation with an LLM, every prompt entered sends the entire conversation to the LLM again.) Or the much larger context windows of RAG, or programmers sending entire source code archives with every prompt. As for the number of interactions, a professional using the tool forty hours a week to support their work could easily have a hundred conversations per week (hundreds of prompts total). There are enough factors here with enough uncertainty that the result could be off by a lot. That said, I agree that it's unlikely to be worse than a celebrity with a private jet. I can't take anyone who discusses climate change seriously if they don't demand banning private jets.

karmakurtisaani 2y ago

> Funny that 60W old school light bulb is bad bad bad, but 400W gfx card is ok Could be because there actually exists a 10x more efficient alternative for the light bulb.