4 ms·
I'd love to see credible numbers on the energy usage of thousands of people running models on their own devices compared to sharing data center resources to run
by simonw 4mo ago
I'd love to see credible numbers on the energy usage of thousands of people running models on their own devices compared to sharing data center resources to run big models that serve many different people at the same time.
My hunch is that the energy/water usage of the data centers is a whole lot more efficient than everyone running at home, but I'd be interested in seeing real data on that.
- cold_harbor 4mo agothe comparison misses that local LLM usage covers tasks you'd never send to an API — private code, offline work, medical notes. the baseline is 'local vs not-doing-it', not 'local vs cloud'
- simonw 4mo agoI have bad news about my private code and medical notes...
- verdverm 4mo agoWith hardware like the Spark and Strix, the water usage is known to be zero, yea? On the energy front, I assume less efficient, but I also think there is a tradeoff in efficiency versus freedom, that's why I have my own hardware.
- fluoridation 4mo agoAll consumer hardware (not counting XOC) uses either air cooling or closed-loop liquid cooling, so the water usage is zero, always. Power is a little trickier. I'd assume it's less efficient, but also the total usage is less, because the user sometimes turns the machine off, and the hardware idles to a deeper sleep state than server hardware.
- simonw 4mo agoThe electricity used by hardware itself consumes water. When people talk about data center water usage they're often also including the water used in electricity generation.
- verdverm 4mo agoIronically, 77% of my electricity comes from flowing water rather than the boiling it, ~90% renewable overall Once we get to fully renewable (as a country), there should be no water involved in electricity generation as well.
- Windchaser 4mo agoWater usage goes up with data centers because more cooling is needed when you run the hardware harder. So: if you're running the models on your own machine, presumably you're not running them as often, and air cooling is sufficient. But, at the same time, this is less efficient in terms of hardware use; the data centers need water cooling specifically because they're getting more bang from their buck from their hardware, by running their hardware harder. So that's the tradeoff: more hardware-use efficiency means more water usage.
- nok22kon 4mo agothat water is recirculated. nobody sends it down the drain after one loop
- jazzyjackson 4mo agoEvaporative cooling is cheaper > [Google’s] largest data center, located in Council Bluffs, IA, withdrew an average of 3.9 million gallons of water and consumed 2.8 million gallons per day. https://mostpolicyinitiative.org/science-note/data-center-water-use/ https://mostpolicyinitiative.org/science-note/data-center-wa...
- echelon 4mo agoAlmonds and avocados use two orders of magnitude more water. You can start adjusting your consumption of those products.