3 ms·

I think the comparison that should be made here is the actual alternative - what is the cost in kw/hr of me sitting in front of my powered-on and not-sleeping l

by luma 2y ago

I think the comparison that should be made here is the actual alternative - what is the cost in kw/hr of me sitting in front of my powered-on and not-sleeping laptop while I spend the next 10 minutes penning an email? If the machine can make that happen in a second, and it takes me 600x longer to do that while consuming power that entire time, I suspect that the AI carries an overall advantage when you actually include the full story. I'll file this under "lazy tech journalists didn't bother to do the math past the first, most obvious step".

Yoric 2y ago
Of course, if it's streamlined to be 600x faster, it's just going to be done 600x more often.
- fhd2 2y ago
  You said what I wanted to say 600x more efficiently :D
- knodi123 2y ago
  see related: "By the 2000s, increases in worker efficiency will mean that most people only need to work two or three days a week!"
EraYaN 2y ago
The biggest problem there is that that laptop is pretty darn efficient. And multiple GPUs and other servers running 100% to run that query for you are not quite as efficient. It does so much more math to get the LLM to do anything useful, the scale is really staggering compared to registering your keystrokes and updating the screen, even if that all happens in a bloated Javascript code base. You can run that laptop, which is not using more than 40 watts in spikes when just typing in a text editor for a long time compared to the full inference and processing load of that LLM. And that ignores the cost of training and getting the training data amortized over all queries.
- rainsford 2y ago
  Per the article, the energy used by AI to write a short email (140 watt hours) is roughly equivalent to running a higher end desktop CPU flat out for an hour. That really puts the amount of computation going on behind the scenes of LLMs in perspective.
fhd2 2y ago
That seems to get a bit philosophical. Are you not gonna use your laptop ~8 hours a day thanks to generative AI? If you still do, that part is constant. Are you gonna get less done in those 8 hours? Probably. Is it good for the environment to get more done in less time? Can't really answer that one generically. We humans do a lot of stuff simply because we can. If unemployment goes up 20% from generative AI (just a made up number for narrative purposes), are those 20% not gonna sit in front of a screen all day? Are they perhaps even going to play games that max out their machines instead? Of course another way to see all this is to argue that we need 20% less humans now, so we save any environmental costs those people induce. Seems a bit gruesome to me.
- jedoch 2y ago
  You writing remids me, what the heck had us so damn distracted 20y ago, cos in front of google (if that may be a starting point ~20y ago) we had heard much of it, not? ...OT...but what was with the radiation levels caesium an uran that past day measured ...what is in norway...or...um...realy off-topic... (to downvote till deleted, please)^^
- schiffern 2y ago
  >Of course another way to see all this is to argue that we need 20% less humans now, so we save any environmental costs those people induce. Seems a bit gruesome to me. Seems a bit of a straw man to me. Perhaps there's some light between ecosystem literacy ("carrying capacity is real") and mass murder? :-\ A third way (which differs only in timing) is thinking we can support 100% of the people who exist right now, but acknowledge that if you iteratively "add 20% more people" enough times eventually we will exceed the environment's capacity. If you think any solution that might be proposed to that is "gruesome" (AKA the usual anti-population modulation trope) just look at the cruelties people inflict on each-other when humans blindly exceed the environment's capacity. Though I suppose such "law of the jungle" ad hoc population control could be preferred by many, since it effects mostly the poor and powerless, whereas intentional population policy would effect all people equally.
  - fhd2 2y ago
rainsford 2y ago
It would have been nice if the article did the comparison for us, but the the premise of the article seems correct. According to the article, ChatGPT uses 140 watt hours to compose a short email. That's considerably more than the battery capacity of many laptops, so even if you spent all day staring at your laptop trying to write the email, you're probably using less energy. Now there's certainly a time advantage to using AI, but LLM based AI is pretty inefficient from an energy perspective.

by luma 2y ago

Yoric 2y ago

Of course, if it's streamlined to be 600x faster, it's just going to be done 600x more often.

fhd2 2y ago
You said what I wanted to say 600x more efficiently :D
knodi123 2y ago
see related: "By the 2000s, increases in worker efficiency will mean that most people only need to work two or three days a week!"

EraYaN 2y ago

The biggest problem there is that that laptop is pretty darn efficient. And multiple GPUs and other servers running 100% to run that query for you are not quite as efficient. It does so much more math to get the LLM to do anything useful, the scale is really staggering compared to registering your keystrokes and updating the screen, even if that all happens in a bloated Javascript code base. You can run that laptop, which is not using more than 40 watts in spikes when just typing in a text editor for a long time compared to the full inference and processing load of that LLM. And that ignores the cost of training and getting the training data amortized over all queries.

rainsford 2y ago
Per the article, the energy used by AI to write a short email (140 watt hours) is roughly equivalent to running a higher end desktop CPU flat out for an hour. That really puts the amount of computation going on behind the scenes of LLMs in perspective.

fhd2 2y ago

That seems to get a bit philosophical. Are you not gonna use your laptop ~8 hours a day thanks to generative AI? If you still do, that part is constant. Are you gonna get less done in those 8 hours? Probably. Is it good for the environment to get more done in less time? Can't really answer that one generically. We humans do a lot of stuff simply because we can. If unemployment goes up 20% from generative AI (just a made up number for narrative purposes), are those 20% not gonna sit in front of a screen all day? Are they perhaps even going to play games that max out their machines instead? Of course another way to see all this is to argue that we need 20% less humans now, so we save any environmental costs those people induce. Seems a bit gruesome to me.

jedoch 2y ago
You writing remids me, what the heck had us so damn distracted 20y ago, cos in front of google (if that may be a starting point ~20y ago) we had heard much of it, not? ...OT...but what was with the radiation levels caesium an uran that past day measured ...what is in norway...or...um...realy off-topic... (to downvote till deleted, please)^^
schiffern 2y ago
>Of course another way to see all this is to argue that we need 20% less humans now, so we save any environmental costs those people induce. Seems a bit gruesome to me. Seems a bit of a straw man to me. Perhaps there's some light between ecosystem literacy ("carrying capacity is real") and mass murder? :-\ A third way (which differs only in timing) is thinking we can support 100% of the people who exist right now, but acknowledge that if you iteratively "add 20% more people" enough times eventually we will exceed the environment's capacity. If you think any solution that might be proposed to that is "gruesome" (AKA the usual anti-population modulation trope) just look at the cruelties people inflict on each-other when humans blindly exceed the environment's capacity. Though I suppose such "law of the jungle" ad hoc population control could be preferred by many, since it effects mostly the poor and powerless, whereas intentional population policy would effect all people equally.
- fhd2 2y ago

rainsford 2y ago

It would have been nice if the article did the comparison for us, but the the premise of the article seems correct. According to the article, ChatGPT uses 140 watt hours to compose a short email. That's considerably more than the battery capacity of many laptops, so even if you spent all day staring at your laptop trying to write the email, you're probably using less energy. Now there's certainly a time advantage to using AI, but LLM based AI is pretty inefficient from an energy perspective.