5 ms·

The Neural Turing Machine has been expanded upon considerably since that paper, and inspired a number of other memory augmented neural network architectures, in

The Neural Turing Machine has been expanded upon considerably since that paper, and inspired a number of other memory augmented neural network architectures, including: https://www.gwern.net/docs/rl/2016-graves.pdf https://www.gwern.net/docs/rl/2016-graves.pdf , https://arxiv.org/abs/1605.06065 https://arxiv.org/abs/1605.06065 and https://arxiv.org/abs/1803.10760 https://arxiv.org/abs/1803.10760 . If you don't want to read a bunch of papers this video also has a good discussion of that last paper, describing how a memory system based on the NTM has been applied to reinforcement learning to achieve very impressive results that seem to me to be a very significant step towards human-like general purpose intelligence: https://www.youtube.com/watch?v=9z3_tJAu7MQ https://www.youtube.com/watch?v=9z3_tJAu7MQ

sterlind 8y ago
Exciting to see a video referencing the MERLIN paper! I hope Deepmind decides to release the code. The paper outlines the architecture well, but reproducing RL results is very tricky. With so many interlinked neural networks in the closed loop, it'll be slow going to isolate failures due to bugs from unfortunate hyperparameter selection or starting seed.. I'd still like to try, though!
- goolulusaurs 8y ago
  Yeah, it is super interesting. I've been gradually working on reproducing it too, mostly as a way to challenge myself and to try to keep on top of some of the cool RL research that has been coming out lately, but I've still got a bit left to do. I started out by working on World Models (https://worldmodels.github.io/ https://worldmodels.github.io/) since it is conceptually similar, but without the memory system and the components are more isolated and easier to test. It has been a lot of fun though, and all the background reading has been very educational! In the Merlin paper I do appreciate how thorough the description of the architecture is, especially compared to some of the earlier deep RL papers. I am hoping since its just a preprint we may get code released when/if it gets officially published, although maybe its not too likely given their history.

sterlind 8y ago

Exciting to see a video referencing the MERLIN paper! I hope Deepmind decides to release the code. The paper outlines the architecture well, but reproducing RL results is very tricky. With so many interlinked neural networks in the closed loop, it'll be slow going to isolate failures due to bugs from unfortunate hyperparameter selection or starting seed.. I'd still like to try, though!

goolulusaurs 8y ago
Yeah, it is super interesting. I've been gradually working on reproducing it too, mostly as a way to challenge myself and to try to keep on top of some of the cool RL research that has been coming out lately, but I've still got a bit left to do. I started out by working on World Models (https://worldmodels.github.io/ https://worldmodels.github.io/) since it is conceptually similar, but without the memory system and the components are more isolated and easier to test. It has been a lot of fun though, and all the background reading has been very educational! In the Merlin paper I do appreciate how thorough the description of the architecture is, especially compared to some of the earlier deep RL papers. I am hoping since its just a preprint we may get code released when/if it gets officially published, although maybe its not too likely given their history.