3 ms·
>> AlphaGo's core is built around a similar machine learning technology to Deepmind's Atari-playing system - the single, untweaked program that was able to lear
by YeGoblynQueenne 11y ago
>> AlphaGo's core is built around a similar machine learning technology to Deepmind's Atari-playing system - the single, untweaked program that was able to learn superhuman play on dozens of different Atari games just by looking at the pixels, without specialization for each particular game.
Woa there. The Deepmind Atari-playing AI was too specialised for each particular
game. It had a reward function that translated the score for it. It couldn't
learn the importance of the game score on its own, just from "looking at the
pixels" and it couldn't learn the significance of the score display and how it
changed as a result of its actions on its own. All this had to be hand-coded.
And if I've seen a hint that this means it couldn't really-really generalise to
other games, like Deepmind claimed, then that's this bit that you report
yourself:
>> Deepmind ... did reuse the widely
known core insight of Monte Carlo Tree Search.
That is no mere detail. That is the crux of the matter, right there. Deepmind
used their architecture to improve MCTS far enough that it could beat Lee
Se-Dol. They didn't just add MCTS to their already general-game playing system.
Because the original Atari-playing AI was completely useless for playing Go, a
game that doesn't have a score and looks nothing like an Atari game. So it
wasn't very general at all, despite Google's and DeepMind's claims to the
contrary.
- moyix 11y agoI think you're wrong here. From the Nature paper describing AlphaGo: "We also tested against the strongest open-source Go program, Pachi, a sophisticated Monte Carlo search program, ranked at 2 amateur dan on KGS, that executes 100,000 simulations per move. Using no search at all, the RL policy network won 85% of games against Pachi." AlphaGo does use MCTS, but it seems that most of its improvements are actually coming from the deep reinforcement learning approach.
- YeGoblynQueenne 11y agoI don't know whether this is true. If AlphaGo could do just as well without search, then why did it use search at all? But in any case, I'm not necessarily disputing that. I'm particularly refuting the claim that the AlphaGo architecture is identical to the one that learned to play Atari games and that Deepmind have advertised as a general game-playing agent. My comment here is specifically in reply to the GP who repeated this claim, but I'll dig up the relevant link if you're interested.
- moyix 11y agoOh, fair enough. There are certainly differences; it's definitely not exact same architecture. They are both using Deep Reinforcement Learning, but e.g. AlphaGo benefits from getting an explicit representation of the board state and game rules rather than having to learn them. Hassabis has said that in the next few months they want to try and get up to current AlphaGo performance without using any MCTS at all.
- suryabhupa 11y agoThe idea is that even there's a policy network that is able to decide at some point what the best possible move is, the tree search is done to refine this choice and to "evaluate" it. This is why a value network is derived from policy network and is used in conjunction with MCTS to make sure that the moves AlphaGo picks are good ones.
- hacker42 11y agoIt is necessary to make multiple alternatives in the tree comparable in an easy way (and nothing is better comparable than scalars). They could also go about training a network that compares two positions to decide which one is superior, but that would require much more computation. Or another alternative would possibly to learn the value somehow jointly with the action selection, but that would possibly also be harder both to train and evaluate.
- hacker42 11y agoWhere did he say "identical"?
- YeGoblynQueenne 11y agoNoone really said that AlphaGo is identical to the Atari playing AI from a while ago. What Deepmind did say is that that agent was a general game-playing agent. If it was, then why was it not employed against Lee Sedol? Well- because it wasn't.
- deleted 11y ago[deleted]