3 ms·
Well, basically all the author did was rediscover reinforcement learning, but with a bit of additional heuristics in how to do epsilon-greedy exploration. I bet
by cyber_kinetist 5y ago
Well, basically all the author did was rediscover reinforcement learning, but with a bit of additional heuristics in how to do epsilon-greedy exploration. I bet 50/50 that the algorithm will work with some tuning and training time, but I doubt what the author said can be extended to anything much more complex than those these simple OpenAI Gym experiments.
The problem I have with their argument is a lot more philosophical: it all boils down to the validity of the reward hypothesis (http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/rewardhypothesis.html http://incompleteideas.net/rlai.cs.ualberta.ca/RLAI/rewardhy...), and how we even define “intelligent” behavior. I’ve written another comment somewhere in this thread for more detail.
- abeppu 5y agoI don't think the author rediscovered reinforcement learning -- they just didn't use that phrase or cite much of the prior work there. But they do mention a Bellman-style decayed sum of rewards, and they use the terms 'state' and 'action' and associated notation which follow the conventions in in RL, so I'm pretty sure they're aware of that field.
- iroh2727 5y agoCorrect. I'm very familiar with RL, but I didn't see how referring to RL literature fit in to my particular argument (and I'm not intimately familiar with that literature). That said, I'd suspect there are fruitful connections to make there, especially considering that this model completely glosses over learning, which is a pretty crucial element... The K&A paper does mention the connection with RL: "Here, we consider the function of the circuit from the point of view of navigation, a connection anticipated by early work on TD learning (Montague et al, 1995)" (specifically, Montague et al found this connection when building a model of bee foraging).