4 ms·
> "Thought for 80m 17s" Is there any good rule of thumb for how many kWh of electricity this is?
by jgalt212 6mo ago
> "Thought for 80m 17s"
Is there any good rule of thumb for how many kWh of electricity this is?
- WarmWash 6mo agoMany orders of magnitude less than the energy needed to sustain a human while they work through the problem.
- bijowo1676 6mo agothe electricity was going to be consumed regardless whether you ask chatGPT or not. It would have been either idle, or serving other users' requests. so the incremental kWh consumption is zero, since costs are fixed and sunk. as a rule of thumb you can lookup the power consumption of the latest nVidia chip, multiply by factor of two or three (to account for cpu/storage/cooling/network/infra)
- tredre3 6mo agoAn idle GPU consumes almost nothing, a loaded (server-class) GPU can consume over 2kW. Admittedly a single request isn't a full load, but claiming that a request makes no difference vs idle is misguided, in my opinion.
- bijowo1676 6mo agoOpenAI GPU wont be idle for long because they have all other requests to serve. Over time there will be a certain % of idle GPUs, amortized across all hundreds of millions of requests they receive.
- aoeusnth1 6mo agoAnd idle% is causally connected to whether you make a request or not, surely? I don't understand how your mental model works.