7 ms·

Actually, I think alpha-go would be more in the "looked cool" category. It was an applied engineering task, not so much a fundamental approach change like adver

by habitue 10y ago

Actually, I think alpha-go would be more in the "looked cool" category. It was an applied engineering task, not so much a fundamental approach change like adversarial networks. That's not to take away from the monumental nature of the feat, but it seems like this blog post was more about higher level developments. Similarly, pixelcnn and wavenet were "what can you do when computing power is no object". I would have liked to see something on RL^2 / "learning to reinforcement learn" which do seem like huge developments to me, but maybe are too new to see the impact of yet.

gwern 10y ago
I don't know. I see a lot of papers which are getting great results on tough domains like program writing using the idea of tree search guided by NNs which may be very old but AlphaGo brought to everyone's attention by demonstrating what modern NNs could do in it. Any domain which is tree or DAG structured and you have a simulator for can be tackled much more effectively now.
- nickpsecurity 10y ago
  Heuristic search is a classic technique for getting things done in AI. Just wasn't good on recognizing the value of some moves. Aka scoring. Today's AI people have ANN's that are great at scoring. Their tunnel vision just made them focus almost exclusively using such tech. Then, a team combines two proven methods to do what each are good at. Big results follow. These fads and tunnel vision happen all the time. The most interesting stuff often happens when a talented person inside the bubble opens their mind to bring in something outside it.
- feral 10y ago
  I agree with this; I've seen people writing off AlphaGo as if it was very little new conceptually over MCTS (but just with more hardware). But the power of the NN was surprising. There's a paragraph from the AlphoGo paper that I think speaks to this: "We also tested against the strongest open-source Go program, Pachi, a sophisticated Monte-Carlo search program, ranked at 2 amateur dan on KGS, that executes 100,000 simulations per move. Using no search at all, the RL policy network won 85% of games against Pachi."
deepnotderp 10y ago
I think it's more a demonstration than anything, but unlike these demonstrations, no one expected just how powerful deep nets could be. No one expected they could win Go, by contrast, everyone (in DL) expected that they could do these things. I do agree though that meta-learning is the future and very interesting, especially interested in the fact that DRL can now construct nets better than humans (and you can actually speed it up with an in-house technique I'm not allowed to share :( ). IMO, wavenet and pixelcnn/rnn demonstrate the power of convolutions as a general purpose AI and broke the reign of RNNs/LSTMs for many tasks.
- deepnotderp 10y ago
  (first paragraph referring to AlphaGo)
Florin_Andrei 10y ago
Meh. It's the old feud between fundamental research and applied engineering. You need both to keep moving forward. AlphaGo was a bit more in the applied camp.
inglor 10y ago
Hi, sorry for the bother but the terms "RL^2" and "learning to reinforcement learn" aren't very googleable - would you mind sharing a link to the paper you're discussing?
- s4chin 10y ago
  Actually, googling 'learning to reinforcement learn' links to this paper - https://arxiv.org/abs/1611.05763 https://arxiv.org/abs/1611.05763
kowdermeister 10y ago
AlphaGo was a serious milestone and message to the public that AI has broken another barrier. http://www.nytimes.com/1997/07/29/science/to-test-a-powerful-computer-play-an-ancient-game.html?pagewanted=all http://www.nytimes.com/1997/07/29/science/to-test-a-powerful... In 1997 this was the general thinking about it: ''It may be a hundred years before a computer beats humans at Go -- maybe even longer,'' said Dr. Piet Hut, an astrophysicist at the Institute for Advanced Study in Princeton
- nl 10y ago
  I agree it was an impressive feat. I don't agree that it was clear it was as far off as that: I believe that a world-champion-level Go machine can be built within 10 years, based on the same method of intensive analysis—brute force, basically—that Deep Blue employed for chess -- Feng-Hsiung Hsu, 2007, http://spectrum.ieee.org/computing/software/cracking-go http://spectrum.ieee.org/computing/software/cracking-go

by habitue 10y ago

gwern 10y ago

I don't know. I see a lot of papers which are getting great results on tough domains like program writing using the idea of tree search guided by NNs which may be very old but AlphaGo brought to everyone's attention by demonstrating what modern NNs could do in it. Any domain which is tree or DAG structured and you have a simulator for can be tackled much more effectively now.

nickpsecurity 10y ago
Heuristic search is a classic technique for getting things done in AI. Just wasn't good on recognizing the value of some moves. Aka scoring. Today's AI people have ANN's that are great at scoring. Their tunnel vision just made them focus almost exclusively using such tech. Then, a team combines two proven methods to do what each are good at. Big results follow. These fads and tunnel vision happen all the time. The most interesting stuff often happens when a talented person inside the bubble opens their mind to bring in something outside it.
feral 10y ago
I agree with this; I've seen people writing off AlphaGo as if it was very little new conceptually over MCTS (but just with more hardware). But the power of the NN was surprising. There's a paragraph from the AlphoGo paper that I think speaks to this: "We also tested against the strongest open-source Go program, Pachi, a sophisticated Monte-Carlo search program, ranked at 2 amateur dan on KGS, that executes 100,000 simulations per move. Using no search at all, the RL policy network won 85% of games against Pachi."

deepnotderp 10y ago

I think it's more a demonstration than anything, but unlike these demonstrations, no one expected just how powerful deep nets could be. No one expected they could win Go, by contrast, everyone (in DL) expected that they could do these things. I do agree though that meta-learning is the future and very interesting, especially interested in the fact that DRL can now construct nets better than humans (and you can actually speed it up with an in-house technique I'm not allowed to share :( ). IMO, wavenet and pixelcnn/rnn demonstrate the power of convolutions as a general purpose AI and broke the reign of RNNs/LSTMs for many tasks.

deepnotderp 10y ago
(first paragraph referring to AlphaGo)

Florin_Andrei 10y ago

Meh. It's the old feud between fundamental research and applied engineering. You need both to keep moving forward. AlphaGo was a bit more in the applied camp.

inglor 10y ago

Hi, sorry for the bother but the terms "RL^2" and "learning to reinforcement learn" aren't very googleable - would you mind sharing a link to the paper you're discussing?

s4chin 10y ago
Actually, googling 'learning to reinforcement learn' links to this paper - https://arxiv.org/abs/1611.05763 https://arxiv.org/abs/1611.05763

kowdermeister 10y ago

AlphaGo was a serious milestone and message to the public that AI has broken another barrier. http://www.nytimes.com/1997/07/29/science/to-test-a-powerful-computer-play-an-ancient-game.html?pagewanted=all http://www.nytimes.com/1997/07/29/science/to-test-a-powerful... In 1997 this was the general thinking about it: ''It may be a hundred years before a computer beats humans at Go -- maybe even longer,'' said Dr. Piet Hut, an astrophysicist at the Institute for Advanced Study in Princeton

nl 10y ago
I agree it was an impressive feat. I don't agree that it was clear it was as far off as that: I believe that a world-champion-level Go machine can be built within 10 years, based on the same method of intensive analysis—brute force, basically—that Deep Blue employed for chess -- Feng-Hsiung Hsu, 2007, http://spectrum.ieee.org/computing/software/cracking-go http://spectrum.ieee.org/computing/software/cracking-go