Very cool. It's interesting how powerful the recurrent network becomes with the addition of the learned hebbian term. For context, even without the Hebbian term

Very cool. It's interesting how powerful the recurrent network becomes with the addition of the learned hebbian term. For context, even without the Hebbian term, recurrent networks can learn to learn to do quite interesting things (Hochreiter et al. 2001). Shameless plug -- our lab recently ported LSTMs to spiking networks without a significant loss in performance, and showed that learning to learn works quite well even with spiking networks (Bellec et al. 2018). So it seems like this method of learning to learn could provide a extremely biologically realistic and fundamental paradigm for fast learning. The addition of the Hebbian term neatly fits in with this paradigm too. Hochreiter et al. 2001: http://link.springer.com/chapter/10.1007/3-540-44668-0_13 http://link.springer.com/chapter/10.1007/3-540-44668-0_13 Bellec et al. 2018: https://arxiv.org/abs/1803.09574 https://arxiv.org/abs/1803.09574