4 ms·

> "DeepMind has found a way around this by employing vast amounts of computer power. AlphaGo takes thousands of years of human game-playing time to learn anythi

> "DeepMind has found a way around this by employing vast amounts of computer power. AlphaGo takes thousands of years of human game-playing time to learn anything." It seems the author may not have been familiar with AlphaGo Zero, which used substantially less processing power. https://deepmind.com/blog/alphago-zero-learning-scratch/ https://deepmind.com/blog/alphago-zero-learning-scratch/

Ajedi32 8y ago
Less power doesn't necessarily mean fewer games. According to the paper on AlphaGo Zero, they trained it on ~4.9 million games. > Over the course of training, 4.9 million games of self-play were generated, using 1,600 simulations for each MCTS, which corresponds to approximately 0.4 s thinking time per move.
- est31 8y ago
  Assuming a go game takes 30 minutes on average, and you are never sleeping, resting, etc, you can do approx 18k games per year. In order to reach 4.9 million games you'd have to play for approx 280 years. So yeah, definitely not thousands of years :). Still, we are maybe one or two orders of magnitude away from the amount of games that humans need to play to become world class players. That being said, the AlphaGo zero paper ends with the words: > Humankind has accumulated Go knowledge from millions of games played over thousands of years, collectively distilled into patterns, proverbs and books. In the space of a few days, starting tabula rasa, AlphaGo Zero was able to rediscover much of this Go knowledge, as well as novel strategies that provide new insights into the oldest of games.
  - deep_etcetera 8y ago
    I doubt a human could learn to become even remotely competitive with only self-play within a human lifetime. Go has improved via a distributed effort, so we should try to estimate the number of go games played by humanity (as an upper bound).
    - est31 8y ago
      Good point. I guess this ability to condense knowledge into language and pass it on has brought us where we are today. Genetically, we aren't that different from cavemen who lived tens of thousands of years ago.
  - visarga 8y ago
    > In order to reach 4.9 million games you'd have to play for approx 280 years. Humans also benefit from millions of years of evolution which shaped our brain architecture in a specific way, and from a rich environment to learn from - nature and society. AG Zero was doing just self play.
  - EGreg 8y ago
    The real question is, how do we train bots for environments which the responses cannot be well simulated, unlike in turn based games? They can’t play against themselves, then. They have to just sort of fit recorded data like backtesting, or do experiments that take time (“is this joke funny?”) or lots of time (“is this diet going to lead to an improvement in metric X?”) How do we solve that?
cgearhart 8y ago
AlphaGo Zero uses substantially less power to play, but it used astronomically more compute power to train & learn. According to OpenAI, AlphaGo Zero used more than 1,000 Petaflop/s-day, or about 4x higher than Alpha Zero and 100x more than Dota 1v1 or 10,000x more than VGG/ResNet. [1] The combination of better algorithms + more efficient hardware has significantly reduced the energy waste of that additional compute power. [1] https://openai.com/blog/ai-and-compute/ https://openai.com/blog/ai-and-compute/

Ajedi32 8y ago

Less power doesn't necessarily mean fewer games. According to the paper on AlphaGo Zero, they trained it on ~4.9 million games. > Over the course of training, 4.9 million games of self-play were generated, using 1,600 simulations for each MCTS, which corresponds to approximately 0.4 s thinking time per move.

est31 8y ago
Assuming a go game takes 30 minutes on average, and you are never sleeping, resting, etc, you can do approx 18k games per year. In order to reach 4.9 million games you'd have to play for approx 280 years. So yeah, definitely not thousands of years :). Still, we are maybe one or two orders of magnitude away from the amount of games that humans need to play to become world class players. That being said, the AlphaGo zero paper ends with the words: > Humankind has accumulated Go knowledge from millions of games played over thousands of years, collectively distilled into patterns, proverbs and books. In the space of a few days, starting tabula rasa, AlphaGo Zero was able to rediscover much of this Go knowledge, as well as novel strategies that provide new insights into the oldest of games.
- deep_etcetera 8y ago
  I doubt a human could learn to become even remotely competitive with only self-play within a human lifetime. Go has improved via a distributed effort, so we should try to estimate the number of go games played by humanity (as an upper bound).
  - est31 8y ago
    Good point. I guess this ability to condense knowledge into language and pass it on has brought us where we are today. Genetically, we aren't that different from cavemen who lived tens of thousands of years ago.
- visarga 8y ago
  > In order to reach 4.9 million games you'd have to play for approx 280 years. Humans also benefit from millions of years of evolution which shaped our brain architecture in a specific way, and from a rich environment to learn from - nature and society. AG Zero was doing just self play.
- EGreg 8y ago
  The real question is, how do we train bots for environments which the responses cannot be well simulated, unlike in turn based games? They can’t play against themselves, then. They have to just sort of fit recorded data like backtesting, or do experiments that take time (“is this joke funny?”) or lots of time (“is this diet going to lead to an improvement in metric X?”) How do we solve that?

cgearhart 8y ago

AlphaGo Zero uses substantially less power to play, but it used astronomically more compute power to train & learn. According to OpenAI, AlphaGo Zero used more than 1,000 Petaflop/s-day, or about 4x higher than Alpha Zero and 100x more than Dota 1v1 or 10,000x more than VGG/ResNet. [1] The combination of better algorithms + more efficient hardware has significantly reduced the energy waste of that additional compute power. [1] https://openai.com/blog/ai-and-compute/ https://openai.com/blog/ai-and-compute/