4 ms·

Are you aware of the Neural Turing Machine (https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401) or Differentiable Neural Computer (https://deepmind

Are you aware of the Neural Turing Machine (https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401) or Differentiable Neural Computer (https://deepmind.com/blog/differentiable-neural-computers/ https://deepmind.com/blog/differentiable-neural-computers/)? They're not exactly what you describe, which I suppose is a truly online model that knows how/when to update its own parameters. But those two models do incorporate a similar concept of external memory, whereby the controller is trained via BP to read/write to a tensor (essentially a form of soft-addressable memory available at inference time). As far as I recall, these were never applied beyond toy problems, and it seems this line of research hasn't been very active (at least since the Transformer's "memorize all the things" approach started performing exceptionally well on all the benchmarks). I haven't read the paper you linked just yet - it may well be relevant.

albertzeyer 6y ago
For updating its own weights, there is the idea of fast weights (https://arxiv.org/abs/1610.06258 https://arxiv.org/abs/1610.06258). This idea of updating its own weights is not new. Schmidhuber has done some work on this (http://people.idsia.ch/~juergen/deep-learning-miraculous-year-1990-1991.html http://people.idsia.ch/~juergen/deep-learning-miraculous-yea...). The main idea is also that the model can modify even itself (so not having two separate nets) (see Schmidhuber, "Steps towards `self-referential' learning"). Online learning / continual learning is yet another (orthogonal) topic. This is the setting where new (training) data becomes available all the time, and the model should use that data, i.e. all input (from inference) is used to update and train the model further. This can be done by standard backpropagation. The problem is usually to overcome catastrophic forgetting in this case. See for example: https://deepmind.com/blog/article/enabling-continual-learning-in-neural-networks https://deepmind.com/blog/article/enabling-continual-learnin...

albertzeyer 6y ago

For updating its own weights, there is the idea of fast weights (https://arxiv.org/abs/1610.06258 https://arxiv.org/abs/1610.06258). This idea of updating its own weights is not new. Schmidhuber has done some work on this (http://people.idsia.ch/~juergen/deep-learning-miraculous-year-1990-1991.html http://people.idsia.ch/~juergen/deep-learning-miraculous-yea...). The main idea is also that the model can modify even itself (so not having two separate nets) (see Schmidhuber, "Steps towards `self-referential' learning"). Online learning / continual learning is yet another (orthogonal) topic. This is the setting where new (training) data becomes available all the time, and the model should use that data, i.e. all input (from inference) is used to update and train the model further. This can be done by standard backpropagation. The problem is usually to overcome catastrophic forgetting in this case. See for example: https://deepmind.com/blog/article/enabling-continual-learning-in-neural-networks https://deepmind.com/blog/article/enabling-continual-learnin...