3 ms·
I wouldn't call it "game over." That's too harsh. The truth is, we don't know. Sure, there's a ridiculous amount of hype, fueled by greed and FOMO, often justi
by cs702 1y ago
I wouldn't call it "game over." That's too harsh. The truth is, we don't know.
Sure, there's a ridiculous amount of hype, fueled by greed and FOMO, often justified by cargo-cultish science, but... progress in AI seems inevitable, because we have the human brain as physical proof that it's possible to build an intelligent machine with 100's of trillions of interconnections between neurons that consumes only about as much energy as an incandescent light bulb.
Today's largest AI models are still tiny in comparison, with only 1-2 trillion interconnections between neurons, each interconnection's weight specified by a parameter value. And these tiny AI models we have today consume many orders of magnitude more energy than a human brain. We have a long way to go, but we have proof that a human-brain equivalent is physically possible.
The timing of progress is of course unpredictable. Maybe we will need new breakthroughs. Maybe not. No one knows for sure. In any case, breakthroughs don't come along on a predictable schedule. It's possible we will go through a period of stagnation that lasts months, or even years. We cannot rule out another "AI winter" just because we don't want one.
Even if progress is punctuated by high peaks and deep valleys, I believe we'll get there, eventually.
- ufmace 1y agoI mostly agree, though I actually wonder if the energy difference is as big as you say. Yeah the big LLM company datacenters consume tremendous amounts of power, but that's mostly for either training or serving ~millions of requests at once. I wonder what the actual net power consumption is for a single machine doing about as much work as a single ordinary person could do with just enough hardware to run the necessary models. Or what the average amount of energy for a single set of interactions is at one of the big shared datacenters - they reportedly have a lot of optimizations to fit more requests on the same hardware. I think it might be only one order of magnitude greater than a human brain. Maybe actually pretty close to equal if you compare total energy used against work being done, since the human brain needs to be kept alive all the time, but can only effectively do work for a limited part of the day.
- cs702 1y agoGreat points. I was thinking only about the noggin inside our heads, without considering all the infrastructure it relies on for support!