5 ms·

Watt-hours seem good right now, and rebut the current arguments that LLMs are wasteful. They might soon stop being the right lens, if your agents can go off and

Watt-hours seem good right now, and rebut the current arguments that LLMs are wasteful. They might soon stop being the right lens, if your agents can go off and (in parallel) expend some arbitrary and borderline-unknowable budget of energy on a single request. At some point you’ll need to measure in terms of $ earned per W-H, or just $out/$in.

actuallyalys 1y ago
To be clear, I'm only saying the unit is correct, not whether the number of watt-hours is accurate or (assuming it is correct) that it's a good amount to spend on each query.
- theptip 1y ago
  Fair, I was trying to make a broader point without being clear about it :)
  - actuallyalys 1y ago
    Gotcha, thanks for clarifying!
tsimionescu 1y ago
(average) Watt-hours would still be the right unit to measure that, if the question is about energy efficiency / environmental impact. You still care how much energy those agents consume to achieve a given task, and the time it takes for tasks is still variable, so you need to multiply by time. Money is irrelevant to energy efficiency.
- theptip 1y ago
  > People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours My point was twofold, “the average query” becomes less meaningful as the variance increases. Sure one can _in principle_ report W-h spent on your account or query, but I think it will get more opaque and hard to predict. Average becomes less useful when agents can do an unpredictable amount of work with increasingly large bounds. Second, and this was perhaps worthy of expanding in my post - I think in the radical abundance world that Altman is describing, energy and efficiency stops being something that people talk about. Fusion, space based solar farms, whatever, it’s easy to imagine solving this stuff. And I think you can even imagine this happening sooner if for example one provider stamps a “100% renewable datacenter” option on their AI product. Then you might not care about energy efficiency at all; in which case you just care about profit.

actuallyalys 1y ago

To be clear, I'm only saying the unit is correct, not whether the number of watt-hours is accurate or (assuming it is correct) that it's a good amount to spend on each query.

theptip 1y ago
Fair, I was trying to make a broader point without being clear about it :)
- actuallyalys 1y ago
  Gotcha, thanks for clarifying!

tsimionescu 1y ago

(average) Watt-hours would still be the right unit to measure that, if the question is about energy efficiency / environmental impact. You still care how much energy those agents consume to achieve a given task, and the time it takes for tasks is still variable, so you need to multiply by time. Money is irrelevant to energy efficiency.

theptip 1y ago
> People are often curious about how much energy a ChatGPT query uses; the average query uses about 0.34 watt-hours My point was twofold, “the average query” becomes less meaningful as the variance increases. Sure one can _in principle_ report W-h spent on your account or query, but I think it will get more opaque and hard to predict. Average becomes less useful when agents can do an unpredictable amount of work with increasingly large bounds. Second, and this was perhaps worthy of expanding in my post - I think in the radical abundance world that Altman is describing, energy and efficiency stops being something that people talk about. Fusion, space based solar farms, whatever, it’s easy to imagine solving this stuff. And I think you can even imagine this happening sooner if for example one provider stamps a “100% renewable datacenter” option on their AI product. Then you might not care about energy efficiency at all; in which case you just care about profit.