3 ms·
Another issue that this article sort of touches on but doesn't make explicit: the real world is not a Markov decision process. There are complex, variable-order
by hyperbovine 12y ago
Another issue that this article sort of touches on but doesn't make explicit: the real world is not a Markov decision process. There are complex, variable-order time dependencies which we are barely aware of but which influence our thinking every second of every day. Trying to model this in software leads to an exponential increase storage and time complexity. The curse of dimensionality has been with us ever since Bellman coined the phrase almost 60 years ago; it's not going away anytime soon. Thus, it's difficult for me to see how deep Q-learning (or any other MDP-based algorithm) gets us any closer to human-level understanding.
- Houshalter 12y agoRecurrent neural networks with e.g. Long Short Term Memory (https://en.wikipedia.org/wiki/Long_short_term_memory https://en.wikipedia.org/wiki/Long_short_term_memory) can model very long term dependencies effectively, and there has been some work lately that get good results with simpler models. They don't use it because it's computationally expensive and totally unnecessary for Atari games, but it's certainly possible.