3 ms·

>Based on our data, AlphaGo’s bold move 37 in Game 2 had a 1 in 10,000 chance of being played by a human. This is meaningless without context. What algorithm,

by ikeboy 11y ago

>Based on our data, AlphaGo’s bold move 37 in Game 2 had a 1 in 10,000 chance of being played by a human. This is meaningless without context. What algorithm, has it been tested for calibration, etc. I honestly don't know what I'm supposed to take away from this number.

thisisdave 11y ago
My understanding is that this is based on the policy network output. If I understand correctly, the policy network is designed to estimate probabilities accurately, although I don't know how accurate it is for low-probability moves like this one.
Houshalter 11y ago
AlphaGo was initially trained to predict the moves that humans would make, using a deep neural network, and given tons of game records. Then it was trained again with reinforcement learning by playing against itself by taking moves based on that probability, and increasing the probability of moves that lead to wins. So when they say it predicted 1 in 10,000 chance, it means it thinks it's really unlikely a human would play that move. Just playing random moves means each move has only 1 361 chance at worst, so that move must strongly violate normal human play patterns.
- ikeboy 11y ago
  It's also odd that both numbers are 10,000. Maybe their probability function bottoms out there?
  - fullshark 11y ago
    More likely just rounded for the purpose of easy digestion in a blogpost
  - sanxiyn 11y ago
    In the Nature paper, move prediction was trained on 160,000 games. So naively, I expect probability can't go below 1 in 160,000.
    - ant6n 11y ago
      0 is also a thing.
    - ikeboy 11y ago
      But those have many more moves.
gcr 11y ago
They describe their algorithm at length in their nature paper, "Mastering the game of Go with deep neural networks and tree search", D Silver at al 2016. http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html http://www.nature.com/nature/journal/v529/n7587/full/nature1...

by ikeboy 11y ago

thisisdave 11y ago

My understanding is that this is based on the policy network output. If I understand correctly, the policy network is designed to estimate probabilities accurately, although I don't know how accurate it is for low-probability moves like this one.

Houshalter 11y ago

AlphaGo was initially trained to predict the moves that humans would make, using a deep neural network, and given tons of game records. Then it was trained again with reinforcement learning by playing against itself by taking moves based on that probability, and increasing the probability of moves that lead to wins. So when they say it predicted 1 in 10,000 chance, it means it thinks it's really unlikely a human would play that move. Just playing random moves means each move has only 1 361 chance at worst, so that move must strongly violate normal human play patterns.

ikeboy 11y ago
It's also odd that both numbers are 10,000. Maybe their probability function bottoms out there?
- fullshark 11y ago
  More likely just rounded for the purpose of easy digestion in a blogpost
- sanxiyn 11y ago
  In the Nature paper, move prediction was trained on 160,000 games. So naively, I expect probability can't go below 1 in 160,000.
  - ant6n 11y ago
    0 is also a thing.
  - ikeboy 11y ago
    But those have many more moves.

gcr 11y ago

They describe their algorithm at length in their nature paper, "Mastering the game of Go with deep neural networks and tree search", D Silver at al 2016. http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html http://www.nature.com/nature/journal/v529/n7587/full/nature1...