3 ms·
The view that LLMs alone are insufficient for AGI is based on their fundamental mathematical architecture
by fnord77 1y ago
The view that LLMs alone are insufficient for AGI is based on their fundamental mathematical architecture
- ACCount37 1y agoWhat is the exact limitation imposed by "their fundamental mathematical architecture"? I'm not aware of any such thing.
- Jensson 1y agoThey don't update themselves as they work on a problem. Humans solve hard problems by forming new neural connections, LLM cannot do that. If you make a model that does that then its no longer the LLM architecture we use today and it would be called something else.
- ACCount37 1y agoEven if we assume that "update themselves as they work on a problem" is vital for AGI (not proven, it's just an assumption), then what stops us from just giving an LLM an "update itself" tool call? As the most naive implementation: the LLM just collects the relevant training data, and then runs PEFT on itself. Then it tests the tuned version of itself to see whether the tune was any good and is worth retaining. Sure, that naive approach would require an LLM to be good at assembling training sets, and validating performance gains, and it would be very computationally expensive. But none of those things would somehow make it not-an-LLM-anymore.