4 ms·
Based on a quick skim, what they're describing already exists in the form of a feature extractor + traditional RL algorithm. State -> features/symbols -> action
by alexbeloi 10y ago
Based on a quick skim, what they're describing already exists in the form of a feature extractor + traditional RL algorithm. State -> features/symbols -> actions as opposed to state -> actions.
So, sure if you split up your decision making into more pieces, then you can specialize those pieces better to your specific task, then it will probably perform better at that task. But the whole point (and success) of deep learning is the feature extractor and decision model should are one network, where the early layers are acting as feature extractors (symbolic component) and the later layers are acting as the decision makers or value learners.
In my mind, this partitioning strategy runs counter to the greater goals of machine learning, which to me is a generic learner that can 'take raw data in' and 'get smart stuff out'.
- Eridrus 10y agoYou're definitely right that we've found joint training to be highly useful and having a neural net simply act as a feature extractor is unsatisfying, but I think that success at unifying neural nets with symbolic or even non-learning systems would be a big step forward. This isn't it, but even the alphago work relied on Monte Carlo tree search, so I wouldn't dismiss this out of hand.
- hacker42 10y agoBut MCTS works in AlphaGo because the 'world' can be fully represented by a tiny state vector which fits a billion times into primary memory these days. The real difficultly of real world problems is the ill-defined, mutable, high-dimensional and partially observable state space.
- Eridrus 10y agoI'm merely pointing out that "all neural everything" is not always the best approach despite joint training being very appealing.