It is interesting how they are using various biological models to develop their own model. They gave their model a reward system and a memory. It will be intere

It is interesting how they are using various biological models to develop their own model. They gave their model a reward system and a memory. It will be interesting to see how far deep Q-networks can be extended and at what point they hit the wall of diminishing returns. |Nevertheless, games demanding more temporally extended planning strategies still constitute a major challenge for all existing agents including DQN. |Notably, the succesfsful integration of reinforcement learning with deep network architectures was critically dependent on our incorporation of a replay algorithm involving the storage and representations of recently experienced transitions. I am not for sure what data the replay algorithm has access to, but I wonder what happens if you extend the amount of data it has. This might be the brick wall this algorithm hits of diminishing returns. It would be interesting to hear what the authors think could help help improve how their model deals with temporally extended planning strategies. As someone who grew up on Atari, Nintendo and Sony this is pretty cool work.