3 ms·

The positive side is supercomputers help us analyze the climate, the Atlantic current, and so forth. The negative side is they also help locate oil. Is it a win

by ip26 2y ago

The positive side is supercomputers help us analyze the climate, the Atlantic current, and so forth. The negative side is they also help locate oil. Is it a win or loss on net? Hard to know. Thinking of entertainment, it probably IS a climate win if you spend an hour at home watching Netflix or chatting with GPT, as opposed to driving around town or jetting across the world. Supposedly a GPT-4 query costs 0.01kWh - meanwhile, a Tesla consumes 0.35kWh a minute at freeway speed.

defrost 2y ago
> it probably IS a climate win if you spend an hour at home watching Netflix or chatting with GPT, as opposed to driving around town or jetting across the world. Nope. Look back to the GP's paradox and the energy consumed watching even just the single most popular youtube video . . . That isn't "energy saved" from "otherwise people would be flocking miles in cars to watch Despacito and Baby Shark Dance in theatres. The AI training loads are over and above the already existing supercomputer modelling of land|sea|air fluid flows vie regular means - it's questionable whether LLM's et al even add anything of values in that domain (despite a plethora of papers asserting it to be so).
Syonyk 2y ago
I would be very interested in the breakdown of how they conclude it's 0.01kWh (10Wh) per request, and what that does and doesn't include. I expect that if you were to calculate "incremental energy per request" - how much "extra CPU compute" each request adds, you could probably get to that sort of value. But odds are good that figure ignored all the training data collection, all the processing on that, storage of that digested information, retrieval of it, etc, and that sort of number tends to also skip things like "storage systems running to have the information available." If I've got an entire datacenter running to provide services, and the request consumes, say, 3 GPU-minutes of time across all the nodes, sure. This is a sane value. It just ignores a lot of the other resources dedicated to the task at various points. Microsoft isn't using 30% more energy than their current footprint on 10Wh/request AI answers. Math like this is very much a "Tell me what answer you'd like, and I'll make it work out!" sort of scenario. An increase in data center use by 30% is harder to fudge.
- bruce511 2y ago
  To put your follow upquestion differently, 'Is 10w/h per request the marginal cost of a request? Or does it factor in the fixed energy cost of the whole facility? Does it include training costs' I'm inclined to lean towards it including at least some fixed costs. It seems rather high to be marginal cost. I have no gut feel for training costs though. So, assuming it does include at least some fixed cost, using it more will reduce 'cost per use' while at the same time driving up actual consumption.

by ip26 2y ago

defrost 2y ago

> it probably IS a climate win if you spend an hour at home watching Netflix or chatting with GPT, as opposed to driving around town or jetting across the world. Nope. Look back to the GP's paradox and the energy consumed watching even just the single most popular youtube video . . . That isn't "energy saved" from "otherwise people would be flocking miles in cars to watch Despacito and Baby Shark Dance in theatres. The AI training loads are over and above the already existing supercomputer modelling of land|sea|air fluid flows vie regular means - it's questionable whether LLM's et al even add anything of values in that domain (despite a plethora of papers asserting it to be so).

Syonyk 2y ago

I would be very interested in the breakdown of how they conclude it's 0.01kWh (10Wh) per request, and what that does and doesn't include. I expect that if you were to calculate "incremental energy per request" - how much "extra CPU compute" each request adds, you could probably get to that sort of value. But odds are good that figure ignored all the training data collection, all the processing on that, storage of that digested information, retrieval of it, etc, and that sort of number tends to also skip things like "storage systems running to have the information available." If I've got an entire datacenter running to provide services, and the request consumes, say, 3 GPU-minutes of time across all the nodes, sure. This is a sane value. It just ignores a lot of the other resources dedicated to the task at various points. Microsoft isn't using 30% more energy than their current footprint on 10Wh/request AI answers. Math like this is very much a "Tell me what answer you'd like, and I'll make it work out!" sort of scenario. An increase in data center use by 30% is harder to fudge.

bruce511 2y ago
To put your follow upquestion differently, 'Is 10w/h per request the marginal cost of a request? Or does it factor in the fixed energy cost of the whole facility? Does it include training costs' I'm inclined to lean towards it including at least some fixed costs. It seems rather high to be marginal cost. I have no gut feel for training costs though. So, assuming it does include at least some fixed cost, using it more will reduce 'cost per use' while at the same time driving up actual consumption.