2 ms·
> it'll be outgunned by something that just leverages raw computation better. The issue for me is that the raw computation is coming at the cost of the planet.
by kennywinker 2mo ago
> it'll be outgunned by something that just leverages raw computation better.
The issue for me is that the raw computation is coming at the cost of the planet. Throwing an aircraft carrier at a problem that needs a bicycle is dumb, but because the damage to the environment required to scale up computation isn't included in the price of that computation - it's easier to just toss the aircraft carrier at every little problem.
So when I say I want to pick and choose, and use smaller models, it's because I like technology and I don't want to hate LLMs, but I also like the planet and don't want LLMs to continue to accelerate environmental collapse.
- c7b 2mo agoIt's a noble cause, but there are probably bigger levers to pull than the model size if you care about environmental impact. If you're running Qwen3.8-27B on energy-efficient hardware like a Mac or a DGX Spark instead of an API (likely running on H100s), I'm sure you're having much more of an impact than you would by switching to, say, a 9B coding-only model on the same hardware. The thing is, I think you won't be able to go orders of magnitude smaller, because a lot of the usefulness of LLMs comes from emergent smartness, and you typically need a minimum amount of complexity to see such emergent phenomena (and I think we're pretty far from understanding this kind of emergence, much further than from the next model generation that annihilates the current one on benchmarks yet again).