5 ms·

I don't know what you're trying to say. Monte Carlo based programs play out entire games, and evaluate them by scoring the end of the game--no heuristics needed

I don't know what you're trying to say. Monte Carlo based programs play out entire games, and evaluate them by scoring the end of the game--no heuristics needed for the evaluation, only for the playouts. And yes, they can accurately score a completed game.

ronaldx 12y ago
I'm trying to express why Go is a difficult problem to solve. I'm saying this (scoring, play-out) has been a difficult problem, which was not solved even a small number of years ago, and is still not considered straightforward. Even if it is solved correctly now, that's only a start. I would also note that Monte Carlo is by definition a heuristic method - it is statistical and not guaranteed to be optimal.
- rockdoe 12y ago
  I would also note that Monte Carlo is by definition a heuristic method - it is statistical and not guaranteed to be optimal. So is the endpoint evaluation in chess programs. This isn't a difference between Go and the other games. (In fact, pretty much nothing in your original post is)
  - ronaldx 12y ago
    Anyone who has a modest understanding of chess can codify very simple rules for identifying a winning endgame. There are extensive wikipedia articles on what constitutes a winning endgame in chess, as the board simplifies towards the end of the game: https://en.wikipedia.org/wiki/Endgame_tablebase https://en.wikipedia.org/wiki/Endgame_tablebase says "All chess positions with up to 6 pieces have been solved" So, I disagree. Even if you don't count this as a theoretical difference, it is a practical difference.
    - rockdoe 12y ago
      Anyone who has a modest understanding of chess can codify very simple rules for identifying a winning endgame. The exact same thing can be done in go, scoring a finished game is trivial. The strength of the program simply isn't determined much by correctly identifying and scoring terminal positions, but the intermediate ones. And for both chess and go, one very much uses heuristics which are often wrong. If the heuristics were never wrong, you wouldn't have to do the tree search part at all. Endgame databases only have a tiny effect on the strength of chess programs (a common misunderstanding!) just because it's not very common for the game to be still "flippable" by the time they become relevant. In all the other positions, you need the heuristics. The situation for checkers on the other hand, is very different. There the endgame databases were critical for solving the game, but due to mandatory capturing rules the search space reduces much faster. Edit: Not sure why HN won't let me reply. But anyway: the error that you're both making is assuming that the positions in which humans stop and score are "endgame" or "finished" positions. That's not at all the case! A game is finished if there are no more legal moves besides filling one's own eyes. Counting at that point is trivial because all the life & death situations are "resolved". Monte Carlo programs play until those positions, not the ones where a human would stop the game. (if you now bring up seki, you get half a cookie)
      - thinkpad20 12y ago
        It seems you've studied this more than me, so perhaps I'm just misunderstanding you, but it sounds like you are equating a finished game with endgame. Scoring a finished game is obviously trivial, but even when a small number of positions remain open on a Go board, the game tree is still combinatorically vast, and I don't think there are any "simple rules" for determining optimal play.
        thesteamboat 12y ago
        As it turns out, Go endgames are surprisingly tractable with clever mathematics. See for example http://senseis.xmp.net/?MathematicalGo http://senseis.xmp.net/?MathematicalGo
      - sehugg 12y ago
asolove 12y ago
I think the above is trying to say that the percentage of total possible trees that a human would bother considering is much lower in Go than most games, like chess. So the computational resources needed to do Monte Carlo and get results competitive with human players are much greater. But the poster is wrong in saying that this means computers need to use heuristics. Indeed, the recent breakthroughs in Go-playing programs come from removing heuristics that previously helped the programs prune the tree space and instead just using Monte Carlo to simulate all possibilities.
- hyperpape 12y ago
  Actually, computers use relatively knowledge heavy playouts these days. http://lifein19x19.com/forum/viewtopic.php?f=18&t=9371 http://lifein19x19.com/forum/viewtopic.php?f=18&t=9371. There's a tricky balance, because incorporating too much knowledge/heuristic use can make the program miss good but surprising moves. Nonetheless, the trend seems to be heavy playouts.
  - sehugg 12y ago
    Yes, the strategy for rollouts (the rest of the game after you leave the currently explored tree) are still an area of research. The successful programs usually involve some sort of local shape matching or other heuristic, sometimes trained using ML techniques. One of the surprising results is that having stronger play between opponents in the rollout phase does not always equate with a successful search. Having a balanced strategy -- that is, one that does not introduce a bias for either player -- appears more important.
rdtsc 12y ago
> Monte Carlo based programs play out entire games Would that be pretty much impossible with Go on current hardware and software?
- rockdoe 12y ago
  Not sure what you mean? That's exactly how go programs work today. They don't need to play out the games with an optimal strategy. Just an unbiased, sufficiently random one.
- hyperpape 12y ago
  No, it's just 300 or so moves. It's an entire game, not an entire game-tree (all possible moves).

ronaldx 12y ago

I'm trying to express why Go is a difficult problem to solve. I'm saying this (scoring, play-out) has been a difficult problem, which was not solved even a small number of years ago, and is still not considered straightforward. Even if it is solved correctly now, that's only a start. I would also note that Monte Carlo is by definition a heuristic method - it is statistical and not guaranteed to be optimal.

rockdoe 12y ago
I would also note that Monte Carlo is by definition a heuristic method - it is statistical and not guaranteed to be optimal. So is the endpoint evaluation in chess programs. This isn't a difference between Go and the other games. (In fact, pretty much nothing in your original post is)
- ronaldx 12y ago
  Anyone who has a modest understanding of chess can codify very simple rules for identifying a winning endgame. There are extensive wikipedia articles on what constitutes a winning endgame in chess, as the board simplifies towards the end of the game: https://en.wikipedia.org/wiki/Endgame_tablebase https://en.wikipedia.org/wiki/Endgame_tablebase says "All chess positions with up to 6 pieces have been solved" So, I disagree. Even if you don't count this as a theoretical difference, it is a practical difference.
  - rockdoe 12y ago
    Anyone who has a modest understanding of chess can codify very simple rules for identifying a winning endgame. The exact same thing can be done in go, scoring a finished game is trivial. The strength of the program simply isn't determined much by correctly identifying and scoring terminal positions, but the intermediate ones. And for both chess and go, one very much uses heuristics which are often wrong. If the heuristics were never wrong, you wouldn't have to do the tree search part at all. Endgame databases only have a tiny effect on the strength of chess programs (a common misunderstanding!) just because it's not very common for the game to be still "flippable" by the time they become relevant. In all the other positions, you need the heuristics. The situation for checkers on the other hand, is very different. There the endgame databases were critical for solving the game, but due to mandatory capturing rules the search space reduces much faster. Edit: Not sure why HN won't let me reply. But anyway: the error that you're both making is assuming that the positions in which humans stop and score are "endgame" or "finished" positions. That's not at all the case! A game is finished if there are no more legal moves besides filling one's own eyes. Counting at that point is trivial because all the life & death situations are "resolved". Monte Carlo programs play until those positions, not the ones where a human would stop the game. (if you now bring up seki, you get half a cookie)
    - thinkpad20 12y ago
      It seems you've studied this more than me, so perhaps I'm just misunderstanding you, but it sounds like you are equating a finished game with endgame. Scoring a finished game is obviously trivial, but even when a small number of positions remain open on a Go board, the game tree is still combinatorically vast, and I don't think there are any "simple rules" for determining optimal play.
      - thesteamboat 12y ago
        As it turns out, Go endgames are surprisingly tractable with clever mathematics. See for example http://senseis.xmp.net/?MathematicalGo http://senseis.xmp.net/?MathematicalGo
    - sehugg 12y ago

asolove 12y ago

I think the above is trying to say that the percentage of total possible trees that a human would bother considering is much lower in Go than most games, like chess. So the computational resources needed to do Monte Carlo and get results competitive with human players are much greater. But the poster is wrong in saying that this means computers need to use heuristics. Indeed, the recent breakthroughs in Go-playing programs come from removing heuristics that previously helped the programs prune the tree space and instead just using Monte Carlo to simulate all possibilities.

hyperpape 12y ago
Actually, computers use relatively knowledge heavy playouts these days. http://lifein19x19.com/forum/viewtopic.php?f=18&t=9371 http://lifein19x19.com/forum/viewtopic.php?f=18&t=9371. There's a tricky balance, because incorporating too much knowledge/heuristic use can make the program miss good but surprising moves. Nonetheless, the trend seems to be heavy playouts.
- sehugg 12y ago
  Yes, the strategy for rollouts (the rest of the game after you leave the currently explored tree) are still an area of research. The successful programs usually involve some sort of local shape matching or other heuristic, sometimes trained using ML techniques. One of the surprising results is that having stronger play between opponents in the rollout phase does not always equate with a successful search. Having a balanced strategy -- that is, one that does not introduce a bias for either player -- appears more important.

rdtsc 12y ago

> Monte Carlo based programs play out entire games Would that be pretty much impossible with Go on current hardware and software?

rockdoe 12y ago
Not sure what you mean? That's exactly how go programs work today. They don't need to play out the games with an optimal strategy. Just an unbiased, sufficiently random one.
hyperpape 12y ago
No, it's just 300 or so moves. It's an entire game, not an entire game-tree (all possible moves).