4 ms·

What matters most is training, post-training, fine-tuning and data scraping, not inference IMO. The "prompting destroys the environment" deal always seemed sens

What matters most is training, post-training, fine-tuning and data scraping, not inference IMO. The "prompting destroys the environment" deal always seemed sensationalistic to me, I'm glad it's starting to be debunked, but at the same time we can see IRL the heavy toll the new data centers being built are taking on the energy grid. If the tech was lean at it's heart in terms of energy consumption, we wouldn't be seeing that or even the private nuclear reactor arms race by big tech.

a3w 1y ago
Yupp, plus the fact that I unwillingly send prompts out by visiting a website that now has a chatbot nag me; Among other annoyances in the economy of scale that say "if it is cheap, just use more of it".
- mvieira38 1y ago
  This is a good point. If we are marching towards some sort of LLM-driven world, where every company wants to have automated AI agents for customer service, internal development, marketing and prospecting, the energy usage will compound. A simple RAG system can be expected to make 5 or more prompts per query, and multi-agent systems like Roo Code power through tens of prompts for any given task.
pixl97 1y ago
> If the tech was lean at it's heart in terms of energy consumption, we wouldn't be seeing that or even the private nuclear reactor arms race by big tech. Eh, I somewhat disagree with this. The US energy grid has had almost no extra capacity for a long time. A lot of this was due to efficiency (not a bad thing) and little industrial growth (not a great thing) in the country. Data centers themselves, I don't think are the biggest cause of the issues, but the distribution grid. We've had tons of problems around distribution with new energy sources coming online, but issues distributing the power to where we need to. We see things like private power plants, not because we can't generate power, but because we absolutely suck at distribution of power.
nerevarthelame 1y ago
To set appropriate goal posts, it shouldn't just the total water/energy usage from start to finish: it's that, relative to the local regions they're being extracted from. Are those local regions at risk of being overused? Focusing on water, Google does reports on their total water withdrawal at all their data centers. It was around 10 billion gallons per year in 2024. That's around 0.03% of water usage in the US (that's just for rough context - not all of Google's data centers are in the US). I don't think that's an eye-popping amount. But when you consider that more than 1 billion gallons came from Council Bluffs, IA, you have to make sure that's being done responsibly, and that the local ecology can support it. Google admits that 28% of their water use comes from sources with "medium to high risk of depletion or scarcity." [0] [0]: https://www.gstatic.com/gumdrop/sustainability/google-2025-environmental-report.pdf https://www.gstatic.com/gumdrop/sustainability/google-2025-e...

a3w 1y ago

Yupp, plus the fact that I unwillingly send prompts out by visiting a website that now has a chatbot nag me; Among other annoyances in the economy of scale that say "if it is cheap, just use more of it".

mvieira38 1y ago
This is a good point. If we are marching towards some sort of LLM-driven world, where every company wants to have automated AI agents for customer service, internal development, marketing and prospecting, the energy usage will compound. A simple RAG system can be expected to make 5 or more prompts per query, and multi-agent systems like Roo Code power through tens of prompts for any given task.

pixl97 1y ago

> If the tech was lean at it's heart in terms of energy consumption, we wouldn't be seeing that or even the private nuclear reactor arms race by big tech. Eh, I somewhat disagree with this. The US energy grid has had almost no extra capacity for a long time. A lot of this was due to efficiency (not a bad thing) and little industrial growth (not a great thing) in the country. Data centers themselves, I don't think are the biggest cause of the issues, but the distribution grid. We've had tons of problems around distribution with new energy sources coming online, but issues distributing the power to where we need to. We see things like private power plants, not because we can't generate power, but because we absolutely suck at distribution of power.

nerevarthelame 1y ago

To set appropriate goal posts, it shouldn't just the total water/energy usage from start to finish: it's that, relative to the local regions they're being extracted from. Are those local regions at risk of being overused? Focusing on water, Google does reports on their total water withdrawal at all their data centers. It was around 10 billion gallons per year in 2024. That's around 0.03% of water usage in the US (that's just for rough context - not all of Google's data centers are in the US). I don't think that's an eye-popping amount. But when you consider that more than 1 billion gallons came from Council Bluffs, IA, you have to make sure that's being done responsibly, and that the local ecology can support it. Google admits that 28% of their water use comes from sources with "medium to high risk of depletion or scarcity." [0] [0]: https://www.gstatic.com/gumdrop/sustainability/google-2025-environmental-report.pdf https://www.gstatic.com/gumdrop/sustainability/google-2025-e...