4 ms·

> It's both exciting and eerie. It's like another intelligent species opening up a new way of looking at the world (at least for this very specific domain). and

> It's both exciting and eerie. It's like another intelligent species opening up a new way of looking at the world (at least for this very specific domain). and much to our surprise, it's a new way that's more powerful than ours. I have been watching Myungwan Kim's commentary for the games - and it seems notable that a few moves he finds very peculiar immediately when they are made, he will later point out to as achieving very good results some 20 moves later. So it also seems quite possible that AlphaGo is actually reading this far ahead, to find those peculiar moves achieve better results than from the more standard approaches. Whether these constitute a 'new way' or not I think depends highly on whether these kind of moves can fit into some general heuristics useful for considering positions, or whether the ability to make them is limited to intelligence's with extremely high computational power for reading ahead.

statsaresimple 11y ago
It seems that you are trying to create a new word that describe this new way of looking at the world. If human are able to decode the information contained in those unexpected moves, perhaps by creating a new heuristic, that could be viewed as a way of understanding the features the machine use internally, that is reading the machine brain. If human are able to decode that information creating new heuristics we could say that we are in a new state in IA in which learning among different intelligent species should be studied.
bpicolo 11y ago
I would imagine it's absolutely thinking that far ahead. That said, it can't possibly search every possible solution, just needs to find an adequate one
lpage 11y ago
> he will later point out to as achieving very good results some 20 moves later This. It's a fairly common feature of any AI that uses some form of tree search/minimax, and the effect is very pronounced in chess. Even the best human players can only think 6-8 plies into the feature versus ~18 for a computer. What we can (could?) do is apply smarter evaluation functions to the board states resulting from candidate plays and stop considering moves that look problematic earlier in the search (game tree pruning). AI tends to use very simple evaluation functions that can be computed quickly. They do so given that 1) it allows for deeper search, and a weak heuristic evaluated far in the future often beats a strong one evaluated a few plies prior and 2) for some games (like Go) it's really hard to codify the "intuitions" that human players speak of. Because search based AI considers board states __very__ far in the future, the results are often completely counterintuitive in a game with an established theory of play. Those theories are born of humans, for humans. The introduction of MCTS some years back was the first leap towards a human level Go AI (incidentally, MCTS is more human-like than exhaustive tree search in that it prunes aggressively by making early judgement calls as to what merits further consideration). AlphaGo's use of deep policy and evaluation networks to score the board is very cool, and the next step in that journey. What's interesting to me is that, unlike chess AI, AlphaGo might actually advance the human theory of Go. It's possible that these "strange moves" will lead to some very interesting insights if DeepMind traces them through the eval and policy networks and manages to back out a more general theory of play.
- bduerst 11y ago
  Wasn't the breakthrough with AlphaGo that it doesn't consider every board combination in the future? Because that there are too many combinations?
  - lpage 11y ago
    Yes, but pruning (not considering everything) is as old as game tree search. Previous Go AIs used MCTS as well. What's new in AlphaGo is a more sophisticated approach to scoring game boards - policy networks that help the AI prune even more aggressively, and a value network that's used to "guess" the winner in lieu of searching to endgame. Note that guessing the winner is just a special case of an evaluation function. For any game, if you could consistently search to the end, your evaluation function is always a -1/1 corresponding to lose/win. AlphaGo is still using MCTS - just a more sophisticated form.
- dragontamer 11y ago
  On the contrary. I think that Chess machines play perfectly for the next 8 moves, but don't necessarily sense the importance of a Knight Outpost (which may have relevance 20 moves ahead. A proper Knight Outpost will remain a fork threat for the rest of the game). It is far easier for a Human to beat a Chess Machine at positional play (ex: a backwards pawn shape will probably be a problem at endgame, 30+ moves from now) than to beat a Chess Machine at tactical play (3 moves from now, I can force a fork between two minor pieces)
  - rwill128 11y ago
    This was true 10-15 years ago. It is no longer true. Chess engines have positional evaluation algorithms that have been trained using many millions of games, and the weighting parameters for different kinds of positional features have been adjusted accordingly. Do some reading on Stockfish for example if you doubt the veracity of my statement.
    - dragontamer 11y ago
mwhitework 11y ago
It's also possible that the positional evaluation is strong enough that AlphaGo can see the value in a position before the human because of the complexity involved in determining the "value" of a given position. My experience is with Chess and Chess AI, but in my experience, the more positional knowledge built into the evaluation function, the better the search performs, even if you have to sacrifice some speed for more thorough evaluation. A significant positional weakness may never be discovered within the search horizon of a chess engine because it may take 50 moves for the weakness to create a material loss, so while it's certainly possible that a deep, but carefully pruned search is being utilized, I suspect that some of the Value Network's evaluation is helping to create some of these seemingly odd moves. For AlphaGo to recognize a position that doesn't achieve a good result for 20 moves, it would often have to search much deeper than those 20 moves (I'm not sure if you're using the term moves to mean ply or both players moving, but if it takes 20 AlphaGo moves for the advantage to materialize, that would be a minimum 40 ply search) to quiesce the search to the point that material exchanges have stopped (again, this is how chess typically does it, I don't know about Go), so the evaluation at the end of the 20 move sequence is arguably more important than a deep search. The sooner you can recognize that a position is good or bad for you, the more time you have to improve the position.

statsaresimple 11y ago

It seems that you are trying to create a new word that describe this new way of looking at the world. If human are able to decode the information contained in those unexpected moves, perhaps by creating a new heuristic, that could be viewed as a way of understanding the features the machine use internally, that is reading the machine brain. If human are able to decode that information creating new heuristics we could say that we are in a new state in IA in which learning among different intelligent species should be studied.

bpicolo 11y ago

I would imagine it's absolutely thinking that far ahead. That said, it can't possibly search every possible solution, just needs to find an adequate one

lpage 11y ago

> he will later point out to as achieving very good results some 20 moves later This. It's a fairly common feature of any AI that uses some form of tree search/minimax, and the effect is very pronounced in chess. Even the best human players can only think 6-8 plies into the feature versus ~18 for a computer. What we can (could?) do is apply smarter evaluation functions to the board states resulting from candidate plays and stop considering moves that look problematic earlier in the search (game tree pruning). AI tends to use very simple evaluation functions that can be computed quickly. They do so given that 1) it allows for deeper search, and a weak heuristic evaluated far in the future often beats a strong one evaluated a few plies prior and 2) for some games (like Go) it's really hard to codify the "intuitions" that human players speak of. Because search based AI considers board states __very__ far in the future, the results are often completely counterintuitive in a game with an established theory of play. Those theories are born of humans, for humans. The introduction of MCTS some years back was the first leap towards a human level Go AI (incidentally, MCTS is more human-like than exhaustive tree search in that it prunes aggressively by making early judgement calls as to what merits further consideration). AlphaGo's use of deep policy and evaluation networks to score the board is very cool, and the next step in that journey. What's interesting to me is that, unlike chess AI, AlphaGo might actually advance the human theory of Go. It's possible that these "strange moves" will lead to some very interesting insights if DeepMind traces them through the eval and policy networks and manages to back out a more general theory of play.

bduerst 11y ago
Wasn't the breakthrough with AlphaGo that it doesn't consider every board combination in the future? Because that there are too many combinations?
- lpage 11y ago
  Yes, but pruning (not considering everything) is as old as game tree search. Previous Go AIs used MCTS as well. What's new in AlphaGo is a more sophisticated approach to scoring game boards - policy networks that help the AI prune even more aggressively, and a value network that's used to "guess" the winner in lieu of searching to endgame. Note that guessing the winner is just a special case of an evaluation function. For any game, if you could consistently search to the end, your evaluation function is always a -1/1 corresponding to lose/win. AlphaGo is still using MCTS - just a more sophisticated form.
dragontamer 11y ago
On the contrary. I think that Chess machines play perfectly for the next 8 moves, but don't necessarily sense the importance of a Knight Outpost (which may have relevance 20 moves ahead. A proper Knight Outpost will remain a fork threat for the rest of the game). It is far easier for a Human to beat a Chess Machine at positional play (ex: a backwards pawn shape will probably be a problem at endgame, 30+ moves from now) than to beat a Chess Machine at tactical play (3 moves from now, I can force a fork between two minor pieces)
- rwill128 11y ago
  This was true 10-15 years ago. It is no longer true. Chess engines have positional evaluation algorithms that have been trained using many millions of games, and the weighting parameters for different kinds of positional features have been adjusted accordingly. Do some reading on Stockfish for example if you doubt the veracity of my statement.
  - dragontamer 11y ago

mwhitework 11y ago

It's also possible that the positional evaluation is strong enough that AlphaGo can see the value in a position before the human because of the complexity involved in determining the "value" of a given position. My experience is with Chess and Chess AI, but in my experience, the more positional knowledge built into the evaluation function, the better the search performs, even if you have to sacrifice some speed for more thorough evaluation. A significant positional weakness may never be discovered within the search horizon of a chess engine because it may take 50 moves for the weakness to create a material loss, so while it's certainly possible that a deep, but carefully pruned search is being utilized, I suspect that some of the Value Network's evaluation is helping to create some of these seemingly odd moves. For AlphaGo to recognize a position that doesn't achieve a good result for 20 moves, it would often have to search much deeper than those 20 moves (I'm not sure if you're using the term moves to mean ply or both players moving, but if it takes 20 AlphaGo moves for the advantage to materialize, that would be a minimum 40 ply search) to quiesce the search to the point that material exchanges have stopped (again, this is how chess typically does it, I don't know about Go), so the evaluation at the end of the 20 move sequence is arguably more important than a deep search. The sooner you can recognize that a position is good or bad for you, the more time you have to improve the position.