3 ms·
> What? Exactly They used the game database to learn the value network, then reinforcement learning of the policy network was performed on self-play games. I.
by cgearhart 11y ago
> What?
Exactly
They used the game database to learn the value network, then reinforcement learning of the policy network was performed on self-play games. I.e., the machine learned to play from existing data, then played against itself to learn the search heuristics (the policy network) without the need for expert data.
- j2kun 11y agoYour claim still doesn't make sense. They either used expert data or they didn't. If the algorithm would lose when they remove the expert data, then they really do need expert data. The tree search wasn't even the novel part of the algorithm... the authors even cite others who had used the identical technique in previous Go algorithms.
- cgearhart 11y agoIt seems that my original comment is unclear. My apologies for the ambiguity. I did not mean that they do not need any expert data, but that part of the training did not require a training data set. They definitely need training data to learn the value function, but training the policy network is based on self-play. While MCTS is not new, I believe bootstrapping reinforcement learning with self-play to train a policy network that guides the MCTS is novel.