20 ms·
Lee Sedol Beats AlphaGo in Game 4
- makoz 11y agoSome pretty questionable moves from AlphaGo in that game, but I'm glad LSD managed to close it out.
- _snydly 11y agoWas it AlphaGo losing the game, or Lee Sedol winning it?
- shadowmint 11y agoGiven the number of errors by the AlphaGo in the last 10 minutes, probably the former.
- nanofortnight 11y agoThose moves look like AlphaGo had calculated a loss way ahead of time.
- calvins 11y agoErrors when already lost don't mean much.
- greenyoda 11y agoI think Lee Sedol won the game earlier by destroying AlphaGo's territory in the center. The commentator (Michael Redmond) was quite impressed with what he did there.
- hoonah 11y agoGiven the genius of LSD, probably the latter.
- techdragon 11y agoSometimes its just not possible to stop the snowball rolling down hill. You cant always turn a retreat into an advance or flanking move, sometimes the first step backwards just turns into a full on rout. I got the impression (possibly incorrectly) that AlphaGo was trying to throw curve-balls and be 'unexpected' in a way that might have 'forced' a mistake it could exploit.
- Crazywater 11y agoAccording to the head of DeepMind, AlphaGo made a mistake in evaluating move 79: https://twitter.com/demishassabis/status/708928006400581632 https://twitter.com/demishassabis/status/708928006400581632
- HardyLeung 11y agoLee Sedol winning, and keeping his cool and not make any mistakes. AlphaGo, on the other hand, went bonker especially towards the end but it got into bad territory not because of silly mistakes but brilliant play by Lee Sedol. Could it possibly be that both of the mistakes were bugs? Perhaps it suggested a non-sensical position such as (25.23, 13.15), and it was snapped to (19, 13) :D
- emcq 11y agoI dont think they were bugs in a traditional sense. I think AlphaGo picked moves to try and maximize the probability of winning, and at some point that was only by the opponent making a suboptimal response. I remember reading somewhere most of it's training data is from amateur games. The model doesn't have a prior that AlphaGo is playing a professional who won't make a bad response. It probably would have resigned a lot earlier with that prior :) Another thing to keep in mind is that AlphaGo has no "memory", so every turn it looks at the board fresh. This means if the probabilities are very close you could have it jump around a bit either due to numerical noise from floating point calculations, model errors, or just tiny differences in probability making the behavior appear erratic and quick to change "strategy".
- eru 11y ago> Perhaps it suggested a non-sensical position such as (25.23, 13.15), and it was snapped to (19, 13) :D Alphago doesn't work like that..
- moistgorilla 11y agoHonestly, I think that is a meaningless distinction unless AlphaGo actually broke down in the middle of the match
- minimaxir 11y agoThere were a few jokes made during the round about how AlphaGo resigns. Turns out it's just a popup window! http://i.imgur.com/WKWMHLv.png http://i.imgur.com/WKWMHLv.png
- ljk 11y agoAnother angle - https://i.imgur.com/Pl7xPfk.jpg https://i.imgur.com/Pl7xPfk.jpg
- agildehaus 11y agoAlphaGo resigns The result "W:Resign" was added to the game information. Edit: Tinyyy is right.
- Tinyyy 11y agoI think it reads W:Resign
- killerna 11y agoNo, it's "The result “W+Resign” was added to the game information" https://www.dropbox.com/s/c5730ibejbv4wle/AlphaGo.jpg?dl=0 https://www.dropbox.com/s/c5730ibejbv4wle/AlphaGo.jpg?dl=0
- tel 11y agoUsually the nomenclature is "W+Resign"
- partycoder 11y ago"White wins by resignation".
- barosl 11y agoAccording to this picture[1], it is more likely "W+Resign". I'm curious why a plus sign is used instead of a colon! [1] http://gall.dcinside.com/board/view/?id=baduk&no=109200&page=1&exception_mode=recommend http://gall.dcinside.com/board/view/?id=baduk&no=109200&page...
- adnzzzzZ 11y agoAccording to the commentary of both streams I was watching, after losing an important exchange in the middle (apparently move 79 https://twitter.com/demishassabis/status/708928006400581632 https://twitter.com/demishassabis/status/708928006400581632) it seems AlphaGo sort of bugged out and started making wrong moves on an already dead group on the right side of the board. After that it kept repeating similar mistakes until it resigned a lot of moves after. But the game was already won for Lee Sedol after that middle exchange. It was really interesting seeing everyone's reactions to AlphaGo's bad moves though.
- ljk 11y ago> it seems AlphaGo sort of bugged out and started making wrong moves Didn't they say that it's not considered a "bug" but rather how AlphaGo "thinks"? "when it's winning it doesn't care about how much it's winning, and when it's losing it doesn't care how bad it's losing"
- pbang 11y agoit does seem like it is suboptimal behavior to keep making obviously dumb plays instead of attempting a comeback like a professional human would.
- titzer 11y agoSpeculating that the reinforcement learning phase reinforced all the best winning strategies but had few examples of weak positions out of which the AI had to fight.
- versteegen 11y agoAlphaGo lost half the games it played against itself, so it's not like it doesn't have millions of training examples. However maybe it didn't learn very well how to recover once it's losing, but rather concentrated on learning how to avoid that in the first place.
- Zarel 11y ago
- esturk 11y agoLSD maybe the only human to ever win against AlphaGo.
- jerrickhoang 11y agoKe Jie won 8 out of 10 when went against Lee though. Lee is probably not the strongest in the world right now.
- dba7dba 11y agoLee is 3rd ranked as of now. He was 1st for almost 10 years before that. Ke Jie is 19 years old. Lee has been a pro for nearly 20 years. Lee is not old but certainly not young. Go game prodigies seem to peak when early 20's, much like mathematicians.
- Noge_Sako 11y agohttp://lesswrong.com/lw/4gi/age_fluid_intelligence_and_intelligent_posts/ http://lesswrong.com/lw/4gi/age_fluid_intelligence_and_intel... My personal hypothesis for the reason why Magnus Carlsen is the youngest chess champion of all time is that with the rise of online battles and computer battles, the age where the blend of crystalized intelligence and working intelligence combines lowers. This has obvious implications for the recent rise of machines immediately correcting human mistakes in mathematics and physics.
- jkn 11y agoI think esturk meant that AlphaGo is barely beatable now, so by the time anyone else gets a chance, it will have improved in the meantime and even a stronger human player won't be able to beat it.
- versteegen 11y agoBy that measure, Fan Hui beat AlphaGo before Lee Sedol did, since he was playing an earlier and bit weaker version of more or less the same complete, distributed, system.
- Bytes 11y agoI was not expecting Lee Sedol to come back and win a game after his first three losses. AlphaGo seemed to be struggling at the end of the match.
- mikeyouse 11y agoThe commentary from Redmond was illuminating for me, in his estimation having played against monte carlo simulations before, at that point the computer just plays very low probability moves with extremely high outcomes if the other player makes a grievous mistake. So the moves looked terrible since the 'retort' is obvious, but if for some reason LSD didn't play the obvious counter, it would catapult Alphago back into the lead. Of course since LSD is an expert, he never missed the retort, but very interesting behavior nonetheless.
- techdragon 11y agoI was hoping Lee Sedol would be able to at least win one, humans can 'learn fast'. We cant learn 24/7 in parallel like a computer, but we do seem to have quite the talent at learning 'situational' things very quickly, quite likely because our very survival depended on it in the deep distant past. :-) I don't believe AlphaGo had the time to do any additional training between matches. So effectively Lee has the ability to 'learn his opponent' while AlphaGo cannot until the entire match set is over because of how long it would take do do additional training.
- bsimpson 11y agoIt's been said elsewhere that AlphaGo studying such a corpus of matches that even every match Lee ever recorded wouldn't be enough to bias it. I presume this also means that adding the last 3 games to that corpus would still not be enough to affect AlphaGo meaningfully.
- mikeyouse 11y agoRelevant tweets from Demis; Lee Sedol is playing brilliantly! #AlphaGo thought it was doing well, but got confused on move 87. We are in trouble now... Mistake was on move 79, but #AlphaGo only came to that realisation on around move 87 When I say 'thought' and 'realisation' I just mean the output of #AlphaGo value net. It was around 70% at move 79 and then dived on move 87 Lee Sedol wins game 4!!! Congratulations! He was too good for us today and pressured #AlphaGo into a mistake that it couldn’t recover from From: https://twitter.com/demishassabis https://twitter.com/demishassabis
- comboy 11y agoI'll risk assumption that somebody from Deepmind team is reading this. Guys, please, publish charts of win prob estimated by alpha go in time during these games. Some heatmap telling which moves did it consider as best for both sides during the games would also be cool, but that's surely more time consuming to prepare. It would be great to be able to have such things for top pro tournaments in the future.
- magoghm 11y agoThat would be awesome.
- lifthrasiir 11y agoI.e. Something like [1] for every move. I was a bit disappointed to learn that such table is only available for that particluar move at the first reading. [1] http://www.nature.com/nature/journal/v529/n7587/fig_tab/nature16961_F5.html http://www.nature.com/nature/journal/v529/n7587/fig_tab/natu...
- fdej 11y agoOff-topic: we (the human species) have built an AI that can master the game of Go, but we don't yet have the intelligence to publish charts correctly on the web. That blur of a JPG is half a megabyte!
- deleted 11y ago[deleted]
- deleted 11y ago[deleted]
- awwducks 11y agoLee Sedol definitely did not look like he was in top form there. I would say (as an amateur) his play in Game 2 was far better. It was the funky clamp position that perhaps forced AlphaGo to start falling apart this game. [0] I wonder if Lee Sedol can find a way to replicate that in Game 5. [0]: https://twitter.com/demishassabis/status/708928006400581632 https://twitter.com/demishassabis/status/708928006400581632
- deleted 11y ago[deleted]
- vc98mvco 11y agoI hope it won't turn out they let him win.
- ljk 11y agoHopefully they're not that disrespectful
- hasenj 11y agoThe game seemed to be going in AlphaGo's favour when it was half way through. Black (AG) had secured a large area on the top that seemed nearly impossible to invade. It was amazing to see how Lee Sedol found the right moves to make the invasion work. This makes me think that if the time for match was three hours instead of two, maybe a professional player will have enough time to read the board deeply enough to find the right moves.
- apetresc 11y agoThe thing is, Lee Sedol didn't "find the right moves"; the wedge at L11 shouldn't have worked. If black's 79th move had been at L10 instead of K10, Sedol would likely have resigned on the spot.
- hasenj 11y agoAs far as I can tell from watching Redmond's commentary at the time, there were other options for white in the area.
- apetresc 11y agoI was on the other stream; Myungwan Kim and Hajin Lee went way deeper than Redmond typically goes (since they have access to an SGF editor instead of a clumsy demo board, and they aren't performing for the camera as much). They seemed pretty confident in their conclusion that L10 killed white.
- joe_the_user 11y agoJust from Redmond's commentary, killing white in the center is just one of the outcomes that Alphago was impelled to achieve. [for just an arbitrary example] If Alphago killed white in the center but lost the bottom and the top right, it could have still lost. (and Sedol had to acheive multiple objectives also, of course). Sure, maybe Alphago missed a winning move. But the situation was fluid both strategically and tactically, which might have been why that machine chose it's losing move and moreover, why I don't think it was just a matter of the machine failing to find a kill - especially since I think computers have been able to beat humans on pure tesuji for a while now.
- spatulan 11y agoI wonder what the chances are of a cosmic ray or some stray radiation causing AlphaGo to have problems. It's quite a rare event, but when you have 1920 CPUs and 280 GPUs, it might up the probability enough to be something you have to worry about.
- teling 11y agoI would expect them to be running on Google's distributed infrastructure which has error corrections through end to end checksums
- tomlock 11y agoActually they were running on a single PC! https://twitter.com/demishassabis/status/708489093676568576 https://twitter.com/demishassabis/status/708489093676568576 ed: oops I misread!
- ljk 11y agoDoes this mean Lee found AlphaGo's weakness, and AlphaGo wasn't player at a out-of-reach level?
- techdragon 11y agoI was hoping to see how AlphaGo would play in overtime. Now I'm curious, does it know how to play in overtime? Can the system handle evaluating how much time it can give itself to 'think' about each move, or does it fall into the halting problem territory and it was programmed to evaluate its probability of winning given the 'fixed' time it had left.
- KPLauritzen 11y agoThey've been in overtime before. In game 2 I think. AlphaGo spent about 30 seconds on each move
- apetresc 11y agoIt was in overtime in game 2, I believe. It did fine.
- grogenaut 11y agoIt's called scorboarding. You start coming up with solutions and ranking them and putting the best one up on the scoreboard. When you run out of time, you just go with what you got. Pretty much what humans have. I'm sure there are many levels of watchdogs in this program.
- emcq 11y agoThat was really cool! It seemed after the brilliant play in the middle the most probable moves for winning required Lee Sedol to make impossibly bad mistakes for a professional, which would be a prior that AlphaGo doesn't incorporate. I've heard the training data was mostly amateur games so perhaps the value/policy networks were overfit? Or maybe greedily picking the highest probability, common with tree search approaches, is just suboptimal?
- tel 11y agoI think it's more that the value network includes moves which look plausible but won't concentrate around the answer to a forcing move as having >99% probability. A human has a heuristic: "I must play here else I lose" but AG assumes its opponent might play anywhere that the ANN calls reasonable.
- argonaut 11y agoThat's not the definition of overfitting.
- emcq 11y agoIf the value/policy model is predictive with a dataset containing only amateur games, but fails to generalize to unseen data with professional games, that seems like a case of overfitting to a dataset only containing amateur games. In this case the expected value network may be different for amateur games than professional games. Is there something I'm missing?
- argonaut 11y agoSorry, I'm being a little academic. Overfitting is when the model fits to noise or error. Overfitting is not synonymous with "inability to generalize beyond the train and test distribution." For all we know AlphaGo has perfectly fit amateur games, but professional games are on a whole different level
- tel 11y ago
- keypusher 11y agoThe crucial play here seems to have been Lee Seedol's "tesuji" at White 78. From what I understand this phrase in Go means something like "clever play" but is something like sneaking up on your opponent with something that they did not see coming. Deepmind CEO confirmed that the machine actually missed the implications of this move as the calculated win percentage did not shift until later. https://twitter.com/demishassabis/status/708928006400581632 https://twitter.com/demishassabis/status/708928006400581632 Another interesting thing I noticed while catching endgame is that AlphaGo actually used up almost all of its time. In professional Go, once each player uses their original (2 hour?) time block, they have 1 minute left for each move. Lee Sedol had gone into "overtime" in some of the earlier games, and here as well, but previously AlphaGo still had time left from its original 2 hours. In this game, it came down quite close to using overtime before resigning, which is does when the calculated win percentage falls below a certain percentage.
- nitrobeast 11y agoAlphaGo was in over time in game 2.
- alanfalcon 11y agoTo add to this, it seemed to have no problem makkng moves wothin the 60 second window (at that lte tage of the game).
- CydeWeys 11y agoI suspect that in almost all cases it has already converged on a move well before sixty seconds, and so if forced by time pressure to choose a move before it has achieved the confidence it's looking for, it will probably still do the right thing most of the time (especially in the endgame, when the branching factor is cut down enormously). It just uses additional time because it has it. I would love to see a graph of AlphaGo's thinking showing certainty of time, and how long it was taking to come up with the move it ultimately ended up playing, and how that certainty evolved over its thinking period.
- 11y ago
- yoavm 11y agookay human race, let's sit back and enjoy our last moments of glory!
- kronion 11y agoAfter AlphaGo won the first three games, I wondered not if the computer had reached and surpassed human mastery, but instead how many orders of magnitude better it was. Given today's result, it may be only one order, or even less. Perhaps the best human players are relatively close to the maximum skill level for go, and that the pros of the future will not be categorically better than Lee Sedol is today.
- praptak 11y agoPros themselves estimate their strength 3-4 stones handicap below God: http://senseis.xmp.net/?KamiNoItte http://senseis.xmp.net/?KamiNoItte:
- alphydan 11y agoExactly what heuristics would they use to know how an omniscient being would play? Unless there are some strong arguments behind it, it sounds like arrogant BS.
- saulrh 11y agoMaybe they're extrapolating from relaxed instances that they've solved? If you know that humans can play pretty much optimally on, say, 11x11, and you know how much performance drops off with each expansion of the board?
- nathan_f77 11y agoYeah, I was waiting for AlphaGo to connect up 10 seemingly unrelated moves into some amazing unpredictable shape that wins the game. But now I think that maybe some humans actually have the ability to play a near-perfect game of Go. Maybe the most skilled human players have already nearly reached the peak of what is possible in a Go game.
- greeneggs 11y agoYudkowski argues that on the scale of intelligence, Einstein and a village idiot are basically right next to each other [1]. So once artificial intelligence gets close to matching the village idiot, it is not far from completely thrashing Einstein. Now if that same picture held for Go, then a situation like this would seem to be impossible. Either the computer should be much worse than a human player, or much better. It would be an incredible coincidence that, at the end of six months of training, the computer happened to be of comparable skill to humans. For the game of Go, at least, Yudkowski is wrong. What other aspects of intelligence are this way? Yudkowski's picture seems appealing, but perhaps it is wrong for many areas of intelligence. [1] http://lesswrong.com/lw/ql/my_childhood_role_model/ http://lesswrong.com/lw/ql/my_childhood_role_model/
- herrvogel- 11y agoAm I right by asumming, that if they would play another game (AlphaGo black and Lee Sedol white), that Lee Sedol could pressure AlphaGo into makeing the same mistake again?
- Buge 11y agoCould the AlphaGo developers make modifications before the next game to prevent this happening again?
- NolF 11y agoThey stated before game 3 that AG had been locked down a while before these games to test the code for bugs and issues. I also think they don't want some of the implications associated with DeepBlue where people modified code even during the games.
- emcq 11y agoThis is an interesting question - if Lee Sedol simply replays the game exactly, does he repeat a win? I think the answer would be most likely not - the monte carlo tree search is randomized so AlphaGo's responses to Sedol may not be exactly the same, requiring Sedol to not be able to repeat the exact same play.
- the_mitsuhiko 11y agoAssuming it's a parallel monte carlo computation i doubt it would make exactly the same moves.
- argonaut 11y agoIf it's true that AlphaGo started making a series of bad moves after its mistake on move 79, this might tie into a classic problem with agents trained using reinforcement learning, which is that after making an initial mistake (whether by accident or due to noise, etc.), the agent gets taken into a state it's not familiar with, so it makes another mistake, digging an even deeper hole for itself - the mistakes then continue to compound. This is one of the biggest challenges with RL agents in the real, physical world, where you have noise and imperfect information to confront. Of course, a plausible alternate explanation is that AlphaGo felt like it needed to make risky moves to catch up.
- kotach 11y agoWhat you are talking about here is called "label bias". [2] It is present only if training is done badly. When you have a game of Go, or Super Mario level. You don't want to make your decisions by just checking the local features and doing them, because it can be the case that by compounding errors you end up in a state you never saw, and all of the future decisions won't be good. One can avoid these situations by training jointly over the whole game. For example, maximum entropy models can work for decision making problems but their training leaves them in a "label bias" state because the training is trying to minimize loss of local decisions, instead of trying to minimize the future regret of current local decision. The solution to these label bias problems are Conditional Random Fields, or Hidden Markov Models. You could accomplish the same with Recursive Neural Networks if you trained them properly. For example, there is no search part (monte carlo tree search, or dynamic programming [viterbi] like it is in CRFs or HMMs) in RNNs but they are completely adequate for decision based problems (sequence labeling etc.). Why is that the case? Because search results are present in the data, there's no need to search if you can just learn to search from the data. If DeepMind open-sourced the hundreds of millions of games that AlphaGo played, it is quite possible to train a model that wouldn't need a Monte Carlo search and would work quite well, because you would learn the model to make local decisions to minimize future regret, not to minimize its local loss. [1] The only reason why reinforcement learning is used is because there are too few human games of Go available for the model to generalize well. Reinforcement learning can be used in the setting of joint learning because you play out the whole game before you do the learning. This means that you can try to learn a classifier that will minimize the regret by making a proper local decision. Although, as far as I know, and can see from the paper, they didn't train AlphaGo jointly over the game sequence. But! Now they have a lot of data and they can repeat the process. [1]: http://arxiv.org/abs/1502.02206 http://arxiv.org/abs/1502.02206 [2]: http://repository.upenn.edu/cgi/viewcontent.cgi?article=1162&context=cis_papers http://repository.upenn.edu/cgi/viewcontent.cgi?article=1162...
- jballanc 11y agoSo AlphaGo is just a bot after all... Toward the end AlphaGo was making moves that even I (as a double-digit kyu player) could recognize as really bad. However, one of the commentators made the observation that each time it did, the moves forced a highly-predictable move by Lee Sedol in response. From the point of view of a Go player, they were non-sensical because they only removed points from the board and didn't advance AlphaGo's position at all. From the point of view of a programmer, on the other hand, considering that predicting how your opponent will move has got to be one of the most challenging aspects of a Go algorithm, making a move that easily narrows and deepens the search tree makes complete sense.
- ljk 11y agoDo you think Lee could use this as a way to crack AlphaGo?
- deleted 11y ago[deleted]
- jballanc 11y agoFrom my limited knowledge of the game, a few of the moves that Lee made before AlphaGo "lost its mind" were a tad on the aggressive side. The conventional wisdom in Go is to prefer more conservative moves (increasingly so as the game progresses). Usually, if your opponent is being overly aggressive then you want to play more conservatively and wait for them to make a mistake, but in AlphaGo's case, it attempted to match Lee's aggressiveness move-for-move, and Lee was able to capitalize. If I had to guess (and this is pure speculation), AlphaGo has no concept of waiting for its opponent to make a mistake. Instead, it assumes its opponent will continue to make the best possible follow-ups, and so AlphaGo feels overly compelled to "keep up". In this case, that did it in. If this is what happened, then yes, I would expect Lee to be able to capitalize.
- rorykoehler 11y agoIn that case, theoretically at least, we could train AlphaGo by getting top Go players to play many games against each other where one or both players is making very aggressive moves when reasonable?
- yk 11y agoApparently AlphaGo made two rather stupid moves on the sidelines, judging from the commentary. Which incidentally is the kind of edgecase one would expect machine learning against itself is bad at learning, since there is a possibility that AlphaGo just tries to avoid such situations. It will be interesting to see if top players are able to exploit such weaknesses once AlphaGo is better understood by high level Go players.
- Jach 11y agoFrom my perspective as a weak player, those stupid moves are the sorts of moves I would make if I already knew I was losing (but not by a huge margin, otherwise resign) but I was in byo-yomi overtime. It settles already dead stones and gives me a bit more time to keep searching for a better move that could maybe turn the game around, and if the opponent fails to make the obvious response then I can turn the game around from their blunder. The only odd thing is AlphaGo makes these moves without having entered byo-yomi time.
- yk 11y agoThanks, this just raises further questions. I have a lot to learn before I can call myself a weak player, but it does sound like AlphaGo regressed to the level of an amateur, which needs a explanaition by itself.
- konart 11y agohttps://twitter.com/StatsmanBruno/status/708941065877737472/photo/1 https://twitter.com/StatsmanBruno/status/708941065877737472/...
- deleted 11y ago[deleted]
- kibaekr 11y agoThat got me for a second hehe (it's a fake)
- partycoder 11y agoMontecarlo bots behave weirdly when losing.
- magoghm 11y agoRight now I don't know if I'm more impressed by AlphaGo's artificial intelligence or its artificial stupidity. Lee Sedol won because he played extremely well. But when AlphaGo was already losing it made some very bad moves. One of them was so bad that it's the kind of mistake you would only expect from someone who's starting to learn how to play Go.
- meric 11y agoAlphaGo kept making bad moves in such a way the rest of the game becomes more and more predictable - each of the moves Lee Sedol makes could be described as the only obvious one. On the surface, as an analogy, it sounds like investors in financial markets, capitulating, selling at a loss for a more risky outcome. In hindsight almost always bad moves, but at the time of making them it feels right because it's removing risk. Investors are losing, and then when capitulating they make even worse moves, like selling at market bottoms. http://www.investopedia.com/terms/c/capitulation.asp?layout=infini&v=4B&adtest=4B http://www.investopedia.com/terms/c/capitulation.asp?layout=...
- meric 11y agoCorrection: for a more risky outcome should be for a less risky outcome
- Tempest1981 11y agoMaybe it has never been in a losing position before?
- chx 11y agoThe overwhelming majority of games it played were against itself. So yes, it has been almost the same number as it has been winning.
- versteegen 11y agoI found this comment on that thread quite insightful: https://gogameguru.com/alphago-4/#comment-13410 https://gogameguru.com/alphago-4/#comment-13410 Edit: here's another great one on MCTS: https://gogameguru.com/alphago-4/#comment-13479 https://gogameguru.com/alphago-4/#comment-13479
- creamyhorror 11y agoHere's the post-game conference livestream: https://www.youtube.com/watch?v=yCALyQRN3hw https://www.youtube.com/watch?v=yCALyQRN3hw At the end, Lee asked to play white in the last match, and the Deepmind guys agreed. He feels that AlphaGo is stronger as white, so he views it as more worthwhile to play as black and beat AlphaGo. Conference over, see you all tomorrow.
- nemonemo 11y agoLee asked to be black, because there's 7.5 points advantage for the white who follows the black, and Lee won as a white this time.
- alanfalcon 11y agoGiven that Lee was black in games one (due to luck of the draw) and three I expected he would be in game five as well. Perhaps the system permits loser to choose?
- hatsunearu 11y agonormally the color of the 5th game is chosen by the result of the 4th game, but Lee asked to change that rule and make him black, so he can have a shot at beating AlphaGo as black.
- CydeWeys 11y agoAre you sure? I thought I heard the commentators saying that it was randomized again for game 5, but maybe I was mistaken?
- aprescott 11y agoIs there a source of the rules for colour choice? I had thought it would be another nigiri as in 1st game. Choosing it based on the 4th game's result would imply Lee Sedol would have been given white. Since he won with white, isn't it an unfair advantage in general? For instance, if it were 2-2 after the 4th game.
- h43k3r 11y agoThe post match conference analysis with Lee Sedol and the CEO of deepmind about the different aspects of the game is beautiful to watch. There seems to be a sense of sincerity rather than the greed to win from each of the side.
- kibaekr 11y agoSo where can we see this "move 78" that everyone is talking about, without having to go through the entire match counting?
- yulunli 11y agoAlphaGo obviously made mistakes in game 4 under the pressure from LSD's brilliant play. I'd like to know if the "dumb moves" are caused by the lack of pro data or some more fundamental flaws with the algorithm/methodology. AlphaGo was trained on millions of amateur games, but if Google/Deepmind builds a website where people (including prop players) can play with AlphaGo, it would be interesting to see who improves faster.
- thangalin 11y agoAlphaGo doesn't feel pressure. http://i.imgur.com/ny3RhD4.png http://i.imgur.com/ny3RhD4.png My guess is that Sedol won because he introduced sufficient complexity through cutting points and numerous black groups (see the image). Since AlphaGo uses Value and Policy networks to determine the hot spots to analyse using Monte Carlo tree searches, by making a game rife with lots of simultaneous fights, Sedol dodged the one-two punch of Value and Policy networks combined with MCTS. In other words, if Sedol can make over a dozen points of interest on the board, AlphaGo cannot deeply assess them all. In the image, there are at least 13 interesting moves and cuts plus up to 15 groups (depending if lone stones are considered groups by AlphaGo). I suspect that this position was far more complex than at any point during any of the three previous games. It might also explain the meltdown of playing out an unfavourable ladder (the P10 group, as P8 is another possible move).
- fma 11y agoCould this be overcome by throwing more hardware?
- thangalin 11y agoYes, to a certain extent and certain complexity. https://en.wikipedia.org/wiki/Go_and_mathematics#Game_tree_complexity https://en.wikipedia.org/wiki/Go_and_mathematics#Game_tree_c... Eventually, math wins. There will come a point where humans cannot make the game sufficiently complex to beat a domain-specific machine intelligence (such as AlphaGo).
- repulsive 11y agonegativist paranoid skeptic could say that it would be a good move for the team to intentionally make αgo lose single battle in the moment it has already won the war..
- dannysu 11y agoIn the post-game press conference I think Lee Sedol said something like "Before the matches I was thinking the result would be 5-0 or 4-1 in my favor, but then I lost 3 straight... I would not exchange this win for anything in the world." Demis Hassabis said of Lee Sedol: "Incredible fighting spirit after 3 defeats" I can definitely relate to what Lee Sedol might be feeling. Very happy for both sides. The fact that people designed the algorithms to beat top pros and the human strength displayed by Lee Sedol. Congrats to all!
- dba7dba 11y agoI've read some articles in Korean press that suggested AlphaGo team picked Lee as their match and not Ke Jie (currently #1 ranked, 19 years young) because there's a lot more public records of Lee's plays over his much longer pro career (nearly 20 years now). Thus more material for AlphaGo to train with and against. So it's not totally arrogant of Ke Jie to suggest he could beat AlphaGo. AlphaGo has not much 'experience' in dealing with Ke Jie. And Lee winning of game 4 shows a human is still indeed more capable than any AI. He basically reprogrammed his game on his own. Sorta.
- kybernetikos 11y agoThat was specifically discussed in the after match conference today. Demis said that it wasn't trained specifically against Lee but was trained against strong amateurs on the Internet initially before playing against itself. He said that even if they'd wanted to, alpha go requires a much larger number of games than are available by Lee Seoul to train so it wouldn't have been possible anyway.
- phatfish 11y agoIt was discussed pre-match for game 2 or 3 as well, and Lee's games were described as "a drop in the ocean" I believe. Came up because of an assertion from the interviewer that chess AI had been trained against it's opponent specifically.
- Angostura 11y ago
- antonioalegria 11y agoDon't want to sound all Conspiracy Theory but somehow this feels planned.. It plays into DeepMind's hand to not have the machine completely trouncing the human. It's less scary and keeps people engaged further into the future. Also seems in-line with the way Demis was "rooting" for the human this time – they already won so now they focus on PR.
- runesoerensen 11y agoDon't post unfounded conspiracy theories if you don't want to sound like a conspiracy theorist.
- antonioalegria 11y agoA lot of comments here are unfounded. It was a thought I had that imo warrants some discussion (which you may disagree)
- hendekagon 11y agoI agree, this point does warrant discussion. How could DeepMind prove that AlphaGo played its best in match 4 ?
- throw_xyz111 11y agoAgreed. 99% of the comments are just speculation - Yours is as good as any.
- wdw129831 11y agoTHIS MUST BE FLAGGED!!!!
- throw121221 11y agoShush! You cannot say anything like this on HN! People here want to believe that the world is fair, Google and Apple do not do evil and pg is the greatest philosopher who has ever lived.
- uzyn 11y agoIt seems Lee Sedol fares better at late to end game than AlphaGo. Makes one wonder if Lee might have won the earlier games had Lee pushed on until the late game stages.
- versteegen 11y agoI don't think so. Everyone seems to agree that MCTS only gets stronger in the endgame, and that AlphaGo started making weird moves towards the end because it thought it was already losing, which by definition means there's no strategy leading to a winning position. ...However if it's true that it couldn't find a way out, shouldn't its probability of winning have hit ~0% much earlier? There was still a long time between when it started acting strangely and when it resigned.
- hatsunearu 11y agomaybe the probability was calculated by playing against other alphagos?
- gcatalfamo 11y agoI believe that after winning 3 out of 5, AlphaGo team started experimenting with variables now that they can relax. Which will in turn be even more helpful for future AlphaGo version than the previous 3 wins.
- Filligree 11y agoThey've declared AlphaGo's code and network frozen for the purposes of this competition, in part to avoid the PR issues that Deep Blue got for doing exactly that.
- Dowwie 11y agoBut, did he pull a Kirk vs Kobayashi Maru? :) (yes, I went there)
- rubiquity 11y agoThis is a great day for humans. Glad to see all those years of human research finally pay off.
- conanbatt 11y agoThis game is a great example for the people that said that AlphaGo didnt play mistakes when it had a better position because it lowered the margin, because it only looks at winning probability. AlphaGo made a mistake and realized it was behind, and crumbled because all moves are "mistakes"(they all lead to loss) so any of them is as good as any other. Im very suprrised and glad to see Humans still have something against AlphaGo, but ultimately, these kind of errors might dissapear if AlphaGo trains 6 more months. It made a tactical mistake, not a theory one.
- edraferi 11y agoThat doesn't make sense to me. Even if the objective function is win probability, it's used to order all potential moves. Thus given a menu of bad options, it should choose the least-bad one, not start choosing at random. I think there's something more subtle going on.
- eru 11y agoAlphago only has an approximation of win probability, and to be more precise of `win probability playing against itself'. That works well in an even match against humans---but when far to the losing side, Alphago's win probability estimate is not very good.
- mizzao 11y agoAnother way to look at this is just how efficient the human brain is for the same amount of computation. On one hand, we have racks of servers (1920 CPUs and 280 GPUs) [1] using megawatts (gigawatts?) of power, and on the other hand we have a person eating food and using about 100W of power (when physically at rest), of which about 20W is used by the brain. [1] http://www.economist.com/news/science-and-technology/21694540-win-or-lose-best-five-battle-contest-another-milestone http://www.economist.com/news/science-and-technology/2169454...
- Drdrdrq 11y agoSure, but AlphaGo will probably evolve much faster. In a few years it will run on much smaller devices, as happened to chess programs.
- WoodenChair 11y agoExactly - the watts comparison is a bad one. Stockfish running on an iPhone (~5 watts?) can play world class chess.
- hajile 11y agoA single A15 core at around 1Ghz has more Gflops of power than deep blue had across it's whole system (11.38Gflops). 1920 CPUs (a 4-core haswell from 2013 is around 170Gflops). 280 GPUs (previous gen Nvidia K series peaks at around 5200GFLOPS). That's 1,782,400Gflops or around 150,000x more processing power. If they were running latest-gen hardware, then the would be closer to 200,000x faster. Given that Moore's law is slowing down and the size of the system, we're a long way from considering putting that in a smartphone.
- Bud 11y agoDefinitely not anything close to gigawatts: ALL of Google (not DeepMind, but the entire company) uses only about 260 megawatts. Probably on the order of one megawatt or so. http://inhabitat.com/infographic-how-much-energy-does-google-use/ http://inhabitat.com/infographic-how-much-energy-does-google...
- Angostura 11y agoBizarre. I felt a palpable sense of relief when I read this. Silly meat-brain that I am.
- hendekagon 11y agoLiving flesh is not meat. Meat is what we call dead flesh. A living brain is made of neural nano-processors.
- jessaustin 11y agoThis is a matter of perspective. If you were a tiger, you would consider anything that walks or crawls to be "meat".
- hendekagon 11y agoWhen the tiger has killed it, it's dead & is then meat, yes
- danharaj 11y agoWhat special insight do you have into the perspective of a tiger? I think this is anthropomorphization.
- eru 11y agoAre you familiar with different levels of precision in language, and word play?
- ehsanu1 11y ago"They're made out of meat": http://www.terrybisson.com/page6/page6.html http://www.terrybisson.com/page6/page6.html
- wnevets 11y agonegative, I am a meat popsicle.
- piyush_soni 11y agoI am super excited and all about the Progress AI has made in AlphaGo, but a part of me feels kind of relieved that humans won at least one match. :). Sure, won't last for long.
- fhe 11y agoMy friends and I (many of us are enthusiastic Go lovers/players) have been following all of the games closely. AlphaGo's mid game today was really strange. Many experts have praised Lee's move 78 as a "divine-inspired" move. While it was a complex setup, in terms of number of searches I can't see it be any more complex than the games before. Indeed because it was a very much a local fight, the number of possible moves were rather limited. As Lee said in the post-game conference, it was the only move that made any sense at all, as any other move would quickly prove to be fatal after half a dozen or so exchanges. Of course, what's obvious to a human might not be so at all to a computer. And this is the interesting point that I hope the DeepMind researchers would shed some light on for all of us after they dig out what was going on inside AlphaGo at the time. And we'd also love to learn why did AlphaGo seem to go off the rails after this initial stumble and made a string of indecipherable moves thereafter. Congrats to Lee and the DeepMind team! It was an exciting and I hope informative match to both sides. As a final note: I started following the match thinking I am watching a competition of intelligence (loosely defined) between man and machine. What I ended up witnessing was incredible human drama, of Lee bearing incredible pressure, being hit hard repeatedly while the world is watching, sinking to the lowest of the lows, and soaring back up winning one game for the human race.. Just incredible up and down in a course of a week. Many of my friends were crying as the computer resigned.
- hatsunearu 11y ago>"divine-inspired" move Just letting others know, this expression is a rather common way of saying a single move that changed the course of the game. There were "divine-inspired moves" that AlphaGo made in the first three games too.
- omphalos 11y agoWell it's not exactly common. It's used to describe the very best moves at the highest level of play.
- hatsunearu 11y agoI mean yeah, the decisive move _I_ make when I play my noob games is hardly divine, but in top level play, it happens every game.
- conanbatt 11y agoMaybe Alpha Go understood it won the 5 series, so its reading that it can lose the last 2 games and still win and hence plays suboptimal :P
- deleted 11y ago[deleted]
- overmille 11y agoNow that we have two points for interpolation, expectations are down to near best human competency in go using distributed computation. Also from move 79 to 87 the machine wasn't able to detect the weak position, that shows its weakness. Now Lee can try and aggressive strategy creating multiple hot points of attacks to defeat his enemy. The human player is showing the power of intelligence.
- deleted 11y ago[deleted]
- codecamper 11y agoWow that is awesome news. Very happy to read this this morning. It's a good day to be human.
- agumonkey 11y agoWay to go humans. (I felt that AlphaGo was unbeatable and a milestone in computing overthrowing organic brains... I gave in the buzz a bit prematurely).
- jacinda 11y ago<joke> AlphaGo let Lee Sedol win to lull us all into a false sense of security. The robot apocalypse is well underway. </joke>
- deleted 11y ago[deleted]
- another-hack 11y agoHumans strike back! :P
- asdfologist 11y agoOn a tangential note, apparently AlphaGo has been added to http://www.goratings.org/ http://www.goratings.org/, though its current rating of 3533 looks off. Shouldn't it be much higher?
- datadata 11y agoProbably not. Nine games of data is not really that strong of evidence. The algorithm probably gives each player a prior such has assigning one fake loss in order to limit the maximum ranking of players with few games and no losses. This prior is discussed in the paper on the ranking algorithm used here: http://www.remi-coulom.fr/WHR/WHR.pdf http://www.remi-coulom.fr/WHR/WHR.pdf
- ctstover 11y agoAs a human, I'm pulling for the human. As a computer programmer, I'm pulling for the human. As a romantic, I'm pulling for the human. As a fan of science fiction, I'm pulling for the human. To me it will matter even he can pull off a 3-2 loss over a 4-1 loss.
- zkhalique 11y agoWow! Incredible! Now we know that they have a chance against each other. I would say that this was a very major point... otherwise we wouldn't know whether AlphaGo's powers have progressed to the point where no one can ever beat it. Now I take what Ke Je said much more seriously: http://www.telegraph.co.uk/news/worldnews/asia/china/12190917/Google-AlphaGo-cant-beat-me-says-China-Go-grandmaster.html http://www.telegraph.co.uk/news/worldnews/asia/china/1219091...
- jonbarker 11y agoAlphaGo's weakness was stated in the press conference inadvertently: it considers only the opponent moves in the future which it deems to be the most profitable for the opponent. This leaves it with glaring blind spots when it has not prepared for lines which are surprising to it. Lee Sedol has now learned to exploit this fact in a mere 4 games, whereas the NN requires millions of games to train on in order to alter its playing style. So Lee only needs to find surprising and strong moves (no small feat but also the strong suit of Lee's playing style generally).
- conceit 11y agoYou are saying, its weakness is making the mistake of not looking at all the profitable moves. The contra argument for the sake of the argument goes, he just was lucky to find a local maximum (?) outside of the search space (?) by chance, rather than learning in a few days the universal function (?) that the NN thinks solves go, or at least one fixed point (?) [i.e. the surprisingly wrong expectation].
- jonbarker 11y agoWould it not be beneficial to the deepmind team to open at least the non-distributed version to the public to allow for training on more players? I was surprised to learn that the training set was strong amateur internet play, why not train on the database of the history of pro games?
- eru 11y agoThat's just the initial training set, to get things started. Alphago learns mostly from playing against itself. (And in the future they are planning to remove the crutch of starting with human generated data entirely.)
- pmontra 11y agoGoGameGuru just published a commentary of the game with some extra insight https://gogameguru.com/lee-sedol-defeats-alphago-masterful-comeback-game-4/ https://gogameguru.com/lee-sedol-defeats-alphago-masterful-c... The author thinks that Lee Sedol was able "to force an all or nothing battle where AlphaGo’s accurate negotiating skills were largely irrelevant." [...] "Once White 78 was on the board, Black’s territory at the top collapsed in value." [...] "This was when things got weird. From 87 to 101 AlphaGo made a series of very bad moves." "We’ve talked about AlphaGo’s ‘bad’ moves in the discussion of previous games, but this was not the same." "In previous games, AlphaGo played ‘bad’ (slack) moves when it was already ahead. Human observers criticized these moves because there seemed to be no reason to play slackly, but AlphaGo had already calculated that these moves would lead to a safe win." Which, I add, is something that human players also do: simplify the game and get home quickly with a win. We usually don't give up as much as AlphaGo (pride?), still it's not different. "The bad moves AlphaGo played in game four were not at all like that. They were simply bad, and they ruined AlphaGo’s chances of recovering." "They’re the kind of moves played by someone who forgets that their opponent also gets to respond with a move. Moves that trample over possibilities and damage one’s own position — achieving less than nothing." And those moves unfortunately resemble what beginners play when they stubbornly cling to the hope of winning, because they don't realize the game is lost or because they didn't play enough games yet not to expect the opponent to make impossible mistakes. At pro level those mistakes are more than impossible. Somebody asked an interesting question during the press conference about the effect of those kind of mistakes in the real world. You can hear it at https://youtu.be/yCALyQRN3hw?t=5h56m15s https://youtu.be/yCALyQRN3hw?t=5h56m15s It's a couple of minutes because of the translation overhead.
- pelf 11y agoNow THIS is news!
- Houshalter 11y agoWe were discussing the probability that Sedol would win this game. Everyone, including me, bet 90% that no human would ever win again, let alone this specific game: http://predictionbook.com/predictions/177592 http://predictionbook.com/predictions/177592 I tried to estimate it mathematically. Using a uniform distribution across possible win rates, then updating the probability of different win rates with bayes rule. You can do that with Laplace's law of succession. I got a 20% that Sedol would win this game. However a uniform prior doesn't seem right. Eliezer Yudkowsky often says that AI is likely to be much better than humans, or much worse than humans. The probability of it falling into the exact same skill level is pretty implausible. And that argument seems right, but I wasn't sure how to model that formally. But it seemed right, and so 90% "felt" right. Clearly I was overconfident. So for the next game, with we use Laplace's law again, we get 33% chance that Sedol will win. That's not factoring in other information, like Sedol now being familiar with AlphaGo's strategies and improving his own strategies against it. So there is some chance he is now evenly matched with AlphaGo! I look forward to many future AI-human games. Hopefully humans will be able to learn from them, and perhaps even learn their weaknesses and how to exploit them. Depending how deterministic they are, you could perhaps even play the same sequence of moves and win again. That would really embarrass the Google team. I hear they froze AlphaGo's weights to prevent it from developing new bugs after testing.
- jules 11y agoThe argument that AI is either much better or much worse does not apply here. It's not an accident that they chose this point in time to play against Lee Sedol instead of 10 years ago or 10 years in the future. They chose this point in time because they thought that they have a reasonable chance of winning. Also, he won with white but he will play with black next time, so playing the same sequence of moves can't happen. Additionally, even if the AI didn't incorporate any randomness in the opening (I think it does) it may choose different moves if it gets a different amount of time to think, so Lee Sedol would have to play his moves at exactly the same time as the last game. A couple of seconds deviation only has to lead to a different move in one of the 80 or so moves before the mistake was made to invalidate this strategy.
- 11y ago
- samstave 11y agoSo I am a completely ignorant of the game go. I mean I've heard about it my whole life but never bothered to understand it ever. But after watching the summary video of AlphaGos win... I'm fascinated. I'm sure there are thousands of resources that can teach me the rules, but HN; can you point me to a resource you recommend to get up to speed?
- GolDDranks 11y agohttps://gogameguru.com/lee-sedol-defeats-alphago-masterful-comeback-game-4/ https://gogameguru.com/lee-sedol-defeats-alphago-masterful-c... > This was when things got weird. From 87 to 101 AlphaGo made a series of very bad moves. It seems to me, that these bad moves were a direct result of AlphaGo's min-maxing tree search. According to @demishassabis' tweet, it had had the "realisation" that it had misestimated the board situation at move 87. After that, it did a series of bad moves, but it seems to me that those moves were done precisely because it couldn't come up with any other better strategy – the min-max algorithm used traversing the play tree expects that your opponent responds the best he possibly can, so the moves were optimal in that sense. But if you are an underdog, it doesn't suffice to play the "best" moves, because the best moves might be conservative. With that playing style, the only way you can do a comeback is to wait for your opponent to "make a mistake", that is, to stray from a series of the best moves you are able to find, and then capitalize that. I don't think AlphaGo has the concept of betting on the opportunity of the opponent making mistakes. It always just tries to find the "best play in game" with its neural networks and tree search – in terms of maximising the probability of winning. If it doesn't find any moves that would raise the probability, it picks one that will lower it as little as possible. That's why it picks uninteresting sente moves without any strategy. It just postpones the inevitable. If you're expecting the opponent to play the best move you can think of, expecting mistakes is simply not part of the scheme. In this situation, it would be actually profitable to exchange some "best-of-class" moves to moves that aren't that excellent, but that are confusing, hard to read and make the game longer and more convoluted. Note that this totally DOESN'T work if the opponent is better at reading than you, on average. It will make the situation worse. But I think that AlphaGo is better in reading than Lee Sedol, so it would work here. The point is to "stir" the game up, so you can unlock yourself from your suboptimal position, and enable your better-on-average reading skills to work for you. It seems to me that the way skilful humans are playing has another evaluation function in addition to the "value" of a move – how confusing, "disturbing" or "stirring up" a move is, considering the opponent's skill. Basically, that's a thing you'd need to skilfully assess your chances to perform an OVERPLAY. And overplay may be the only way to recover if you are in a losing situation.
- GolDDranks 11y agoIt would be interesting to see how AlphaGo would play with a handicap – because the skill of overplaying is required from the start in that situation. It might actually suck at handicap games, at the moment.
- sethbannon 11y agoI wouldn't be surprised if, in a month, Lee Sedol was able to beat AlphaGo in another match. This is what happened in chess. The best computers were able to beat the best humans, until the best humans learned how to play anti-computer chess. This bought them a year or so more, until computers finally dominated for good.
- deleted 11y ago[deleted]
- eslaught 11y agoIs there a place I can go to quickly flip through all the board states from the game?
- CydeWeys 11y agoSee the bottom of https://gogameguru.com/lee-sedol-defeats-alphago-masterful-comeback-game-4/ https://gogameguru.com/lee-sedol-defeats-alphago-masterful-c...
- deleted 11y ago[deleted]
- hyperpape 11y agoIt's worth mentioning that while 79 is where Black goes bad, not everyone is sure that 78 actually works for White (http://lifein19x19.com/forum/viewtopic.php?f=15&t=12826 http://lifein19x19.com/forum/viewtopic.php?f=15&t=12826). I'm sure we'll eventually get a more complete analysis.
- atrudeau 11y agoMove 78 gives us hope in the war against the machines. 78 could come to symbolize humanity. What a special moment.
- kazagistar 11y agoMachines retreat and regroup, and fix the problems. Next time there is no chance for a move 78.
- gdy 11y agoMove78 sounds like a good name for an undeground human resistance group)
- devanti 11y agoI wonder if Lee Sedol were to start as white again, and follow the exact same starting sequences, would AlphaGo's algorithms follow the exact same moves as it did before?
- conceit 11y agoI just noticed a pun in the name: All Phago, devourer of worlds. Especially funny as beating a stone could be imaged as swallowing.
- hendryau 11y agoHas anyone heard the crazy theory that alphago bugged out because of daylight savings (the cutover happened mid-game)? Anyone know the exact time at which alphago made its first wonky move?
- kazinator 11y agoAlso: Lee Sedol doesn't have RAM that can be crammed with faithfully recalled gigabytes of information, and that allow exhaustive, yet precise searching of vast information spaces. The amount of short-term information Sedol can remember perfectly is very small by comparison, and doing so requires a lot of concentration and effort. Secondly, the faculty with which Lee Sedol plays Go wasn't designed for the exclusive task of playing Go. Without having to load a different program, Sedol's brain can do many other things well.