3 ms·
They don't update themselves as they work on a problem. Humans solve hard problems by forming new neural connections, LLM cannot do that. If you make a model th
by Jensson 1y ago
They don't update themselves as they work on a problem. Humans solve hard problems by forming new neural connections, LLM cannot do that. If you make a model that does that then its no longer the LLM architecture we use today and it would be called something else.
- ACCount37 1y agoEven if we assume that "update themselves as they work on a problem" is vital for AGI (not proven, it's just an assumption), then what stops us from just giving an LLM an "update itself" tool call? As the most naive implementation: the LLM just collects the relevant training data, and then runs PEFT on itself. Then it tests the tuned version of itself to see whether the tune was any good and is worth retaining. Sure, that naive approach would require an LLM to be good at assembling training sets, and validating performance gains, and it would be very computationally expensive. But none of those things would somehow make it not-an-LLM-anymore.