3 ms·
I understand you're assuming that NNs won't be useful for this purpose, until they can be continuously on-line trained - that is, learning while working, like a
by TeMPOraL 3y ago
I understand you're assuming that NNs won't be useful for this purpose, until they can be continuously on-line trained - that is, learning while working, like animal and human brains. Everyone else on this subthread is arguing that this is not necessary. Instead, you can use pre-trained NNs as fixed components, use a different system (like a vector database, in my example) to provide memory, and the whole thing will already be capable of learning.
Sure, it won't be able to fine-tune its instincts, at least not at first[0], but this doesn't matter - it makes no sense to compare robots and squirrels across their entire life cycle, as where every animal needs to learn most things from experience, an AI-powered robot rolls out of factory with all that experience preloaded, and experience itself is something developed separately, at scale, synthesizing much more any individual animal or human could ever learn in their lifetime.
In short: the limitation you're pointing out is a limitation of LARPing biological life. It's not a limitation for beating its performance across the board.
> For the record, what I'm talking about is called "catastrophic forgetting"
As I understand it, this is exactly the thing the robot-controlling AI will be immune to, if you build it around current pre-trained NN models and external memory. No NN fine-tuning on the fly -> no possibility for "catastrophic forgetting".
EDIT: Re launching a robot with an untrained NN - such a thing doesn't make practical sense unless you're also making the robot autonomously self-replicating. If you're going to build the hardware for the robot in a factory, you may just as well preload it with firmware; there's hardly a case where one would be possible and the other not.
--
[0] - Two avenues for future development: 1) building the robot AI around models that can be fine-tuned on the fly, or 2) training an NN that can emulate an NN inside; the internal NN would then become a function of variable inputs to fixed NN. I think option 1) is actually promising - note that those NNs don't have to be continuously fine-tuneable - you can e.g. put two NNs for each component, use one during "wake time", and fine-tune it during "sleep time" based on recorded memory, while other serves as backup. Or fine-tune one while using the other, then swap pointers. There's solid prior art here with systems designed for space missions - the design I proposed is just a more complex version of how software updates are implemented on Martian rovers, and other hardware that can't be reflashed in the lab if you brick it remotely.