4 ms·

> Monte-Carlo seems more akin to how humans play games. I don't think random choices is how humans play chess, or games in general. I would say it's pattern m

> Monte-Carlo seems more akin to how humans play games. I don't think random choices is how humans play chess, or games in general. I would say it's pattern memory: one learns to play by building a hierarchy of patterns that lead to a prediction of outcome, thus making it possible to select outcome(s) that lead to success.

ur-whale 8y ago
Monte-carlo algorithm don't make random choices. The "randomness" part is simply a device used to approximate expected values (integrals) in very large spaces.
- java-man 8y ago
  true, but my point still holds: there is very little randomness in the way brain works. yes, it is present, but not as a primary device to deal with large search space. we don't have a definitive answers on how the brain works, but I think, in this discussion, sparsity and hierarchy are those devices.
dragontamer 8y ago
Hmmm, you're right in that humans don't play randomly. But Monte-Carlo in this instance meant "MCTS", and not "random play". I apologize for the imprecision. MCTS is very methodological despite the name. Especially in the AlphaZero implementation, where there's no random play during inference / during games (!!). So where did the "Monte Carlo" name come from? The "original" MCTS algorithm used random play to guestimate the strength of a position. But the "real" theory behind MCTS is the math revolving around multi-armed bandit problems. Given a large number of slot machines (in Chess or Go, the positions are conceptually seen as "slot machines" in a casino), the goal of MCTS is to find the slot-machine that gives the highest probability of winning. In effect: classic MCTS rolls out a position all the way to the final result: win, loss, or draw. Not really in AlphaZero, but this is how MCTS was originally designed. AlphaZero shortcuts this process with a neural-network to guestimate the endgame result. But still: the MATH, and the precise order which the search tree conducts is very heavily based in multi-armed bandit theory and random samples. Even if the algorithm in practice doesn't use randomization, all the math and understanding of the search tree is derived from randomized ideals. I think MCTS well-represents the human thinking of the typical expert. Every play is read all the way to the end of the game instinctively. Experts study the pawn-king-rook endgame not necessarily because they expect to see pawn-king-rook endgames... but because the pawn-king-rook endgame is of huge importance to middle-game pawn development. Learning to see how middle-game (and even early-game) moves develop into endgame is pretty much what being an expert is about. And MCTS best emulates that kind of thinking. True, chess is very tactical. But humans are generally bad at exhaustive searches and tactics. Humans win with positional play, and with better understanding of endgame positions. ---------- So rest assured, when I say "Monte Carlo", I'm not talking about a typical monte-carlo algorithm. I'm talking about Monte Carlo Tree Search, which is a very, very different theoretical basis than Alpha-Beta pruning.
- java-man 7y ago
  thank you very much for the explanation!

ur-whale 8y ago

Monte-carlo algorithm don't make random choices. The "randomness" part is simply a device used to approximate expected values (integrals) in very large spaces.

java-man 8y ago
true, but my point still holds: there is very little randomness in the way brain works. yes, it is present, but not as a primary device to deal with large search space. we don't have a definitive answers on how the brain works, but I think, in this discussion, sparsity and hierarchy are those devices.

dragontamer 8y ago

Hmmm, you're right in that humans don't play randomly. But Monte-Carlo in this instance meant "MCTS", and not "random play". I apologize for the imprecision. MCTS is very methodological despite the name. Especially in the AlphaZero implementation, where there's no random play during inference / during games (!!). So where did the "Monte Carlo" name come from? The "original" MCTS algorithm used random play to guestimate the strength of a position. But the "real" theory behind MCTS is the math revolving around multi-armed bandit problems. Given a large number of slot machines (in Chess or Go, the positions are conceptually seen as "slot machines" in a casino), the goal of MCTS is to find the slot-machine that gives the highest probability of winning. In effect: classic MCTS rolls out a position all the way to the final result: win, loss, or draw. Not really in AlphaZero, but this is how MCTS was originally designed. AlphaZero shortcuts this process with a neural-network to guestimate the endgame result. But still: the MATH, and the precise order which the search tree conducts is very heavily based in multi-armed bandit theory and random samples. Even if the algorithm in practice doesn't use randomization, all the math and understanding of the search tree is derived from randomized ideals. I think MCTS well-represents the human thinking of the typical expert. Every play is read all the way to the end of the game instinctively. Experts study the pawn-king-rook endgame not necessarily because they expect to see pawn-king-rook endgames... but because the pawn-king-rook endgame is of huge importance to middle-game pawn development. Learning to see how middle-game (and even early-game) moves develop into endgame is pretty much what being an expert is about. And MCTS best emulates that kind of thinking. True, chess is very tactical. But humans are generally bad at exhaustive searches and tactics. Humans win with positional play, and with better understanding of endgame positions. ---------- So rest assured, when I say "Monte Carlo", I'm not talking about a typical monte-carlo algorithm. I'm talking about Monte Carlo Tree Search, which is a very, very different theoretical basis than Alpha-Beta pruning.

java-man 7y ago
thank you very much for the explanation!