3 ms·

I was really hoping to see a more technical discussion than what I found here in the comments. It's too bad that such a cool accomplishment gets reduced to argu

I was really hoping to see a more technical discussion than what I found here in the comments. It's too bad that such a cool accomplishment gets reduced to arguments about the implications for an AI apocalypse and "moving the goalposts". This isn't strong AI, and it was at least believed to be possible (albeit incredibly difficult), but it is still a remarkable achievement. To my mind, this is a really significant achievement not because a computer was able to beat a person at Go, but because the DeepMind team was able to show that deep learning could be used successfully on a complex task that requires more than an effective feature detector, and that it could be done without having all of the training data in advance. Learning how to search the board as part of the training is brilliant. The next step is extending the technique to domains that are not easily searchable (fortunately for DeepMind, Google might know a thing or two about that), and to extend it to problems where the domain of optimal solutions is less continuous.

j2kun 11y ago
> without having all of the training data in advance What? They certainly trained the algorithm on a huge database of professional go games. It's even in the abstract. [1] [1]: http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html http://www.nature.com/nature/journal/v529/n7587/full/nature1...
- spot 11y ago
  they were amateur expert games from the KGS server.
  - hyperpape 11y ago
    I believe the October match used amateur games, but for this match, they added a professional database.
- cgearhart 11y ago
  > What? Exactly They used the game database to learn the value network, then reinforcement learning of the policy network was performed on self-play games. I.e., the machine learned to play from existing data, then played against itself to learn the search heuristics (the policy network) without the need for expert data.
  - j2kun 11y ago
    Your claim still doesn't make sense. They either used expert data or they didn't. If the algorithm would lose when they remove the expert data, then they really do need expert data. The tree search wasn't even the novel part of the algorithm... the authors even cite others who had used the identical technique in previous Go algorithms.
    - cgearhart 11y ago
      It seems that my original comment is unclear. My apologies for the ambiguity. I did not mean that they do not need any expert data, but that part of the training did not require a training data set. They definitely need training data to learn the value function, but training the policy network is based on self-play. While MCTS is not new, I believe bootstrapping reinforcement learning with self-play to train a policy network that guides the MCTS is novel.

j2kun 11y ago

> without having all of the training data in advance What? They certainly trained the algorithm on a huge database of professional go games. It's even in the abstract. [1] [1]: http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html http://www.nature.com/nature/journal/v529/n7587/full/nature1...

spot 11y ago
they were amateur expert games from the KGS server.
- hyperpape 11y ago
  I believe the October match used amateur games, but for this match, they added a professional database.
cgearhart 11y ago
> What? Exactly They used the game database to learn the value network, then reinforcement learning of the policy network was performed on self-play games. I.e., the machine learned to play from existing data, then played against itself to learn the search heuristics (the policy network) without the need for expert data.
- j2kun 11y ago
  Your claim still doesn't make sense. They either used expert data or they didn't. If the algorithm would lose when they remove the expert data, then they really do need expert data. The tree search wasn't even the novel part of the algorithm... the authors even cite others who had used the identical technique in previous Go algorithms.
  - cgearhart 11y ago
    It seems that my original comment is unclear. My apologies for the ambiguity. I did not mean that they do not need any expert data, but that part of the training did not require a training data set. They definitely need training data to learn the value function, but training the policy network is based on self-play. While MCTS is not new, I believe bootstrapping reinforcement learning with self-play to train a policy network that guides the MCTS is novel.