5 ms·
It's not an improved tree search. Deepmind was almost pro level purely using the deep learning network before doing any Monte Carlo search. One way of looking
by Eliezer 11y ago
It's not an improved tree search. Deepmind was almost pro level purely using the deep learning network before doing any Monte Carlo search.
One way of looking at the significance of this is that it might tell us that relatively simple machine learning algorithms can capture key aspects of the versatile human cortical capacity to learn things like Go using sheer pattern recognition. (It's amazing that human visual cortex can do that.) If the human brain were more mystical in its power, then human-level ability to recognize Go patterns wouldn't have been penetrable at all to a comparatively simple neural algorithm like deep learning.
From another standpoint, this could show that we're reaching the point where, when an AI algorithm starts to reach interesting levels of performance, a much-encouraged Google dumps 100,000 tons of GPU time and 5 months of a few dozen researchers' time to improve the algorithm right past the human performance level. In N years from now when it's a more interesting AI system doing more interesting cognition, we could see a more interesting result materialize when a big organization, encouraged by some key milestone, invests 5 months of time and massively more computing power to further improve an AI system.
- panic 11y agoRegardless of how good the deep learning network is on its own, the algorithm described in the DeepMind paper is an improved tree search.
- Eliezer 11y agoI don't think it's accurate to call AlphaGo 'improved tree search' the way that Deep Blue was improved tree search. You could with equal justice call it an improved neural net.
- argonaut 11y agoThe correct characterization is that it's a hybrid of deep learning and tree search. It's also a hybrid in the feature representation for the algorithm. There are raw features (board state), but there are also handcrafted features for the tree search.
- panic 11y agoI'd say both characterizations are accurate. Neither the tree search nor the neural net could have accomplished this on their own. But the essential interface to the algorithm is the tree search: it's picking the best move from a set of legal moves determined by formal game rules. The real world doesn't follow formal game rules. I find it difficult to see the progression from this victory to some kind of real-world AI takeover.
- igravious 11y ago> The real world doesn't follow formal game rules. Really? Why not?
- argonaut 11y agoInfinite state/belief/world space. Infinite action space. It's not so much that there aren't rules (there are - physics), it's that the complexity of the full set of rules is exponential or super-exponential.
- ogrisel 11y agoYou can do math with continuous and infinite dimensional spaces.
- argonaut 11y agoAnd? This does not address my argument that the complexity is beyond-combinatorially explosive (infinite spaces). I'm not talking about the space of possible board states. I'm talking about merely the set of all possible actions. EDIT: clarified my language to address below reply.
- eru 11y agoGo is combinatorially explosive, too.
- brian_cloutier 11y agoForest for the trees. Monte Carlo Tree Search was necessary and itself a massive improvement over minimax but not sufficient for creating a Go program to challenge professional players. The true innovation here is the neural networks. Without those networks to guide it AlphaGo plays far worse than existing programs. The fact that those networks are sufficient is pretty incredible. We already knew that by inventing them we had created a very pure form of pattern recognition, but it's surprising that the pattern recognition coupled with some tree search seems to be all you need to play Go as well as humans. It's not impressive to you that we've now reproduced a piece of human intelligence, "intuition", which was previously considered out of reach?
- panic 11y agoIt's totally impressive! I'm excited for the future of DeepMind and to see what other kinds of things we can build from neural networks. I'm just saying that we have to be careful with what kind of intelligence we ascribe to an algorithm like this. An AI is not going to take over the world by impressing us with its game-playing skills.
- nl 11y agoDeepMind is the company/lab Google bought. The algorithm is more properly called AlphaGo.
- JabavuAdams 11y ago> An AI is not going to take over the world by impressing us with its game-playing skills. What if you train an AI to play an RTS where matter, energy, and time are the resources and the goal is to take over the world?
- argonaut 11y agoI'm not going to fear an AI whose idea of "the world" is a computer game, an AI that isn't even aware of the existence of the real world, and isn't even aware of the existence of the set of real world actions.
- zodiac 11y agoWell, by that logic, why can't you describe Lee Sedol as applying an "improved tree search" that happens to have really good move selection heuristics and static board evaluation function as well? After all, all good human Go players read ahead a few moves.