3 ms·
Hmmm, you're right in that humans don't play randomly. But Monte-Carlo in this instance meant "MCTS", and not "random play". I apologize for the imprecision. M
by dragontamer 8y ago
Hmmm, you're right in that humans don't play randomly. But Monte-Carlo in this instance meant "MCTS", and not "random play". I apologize for the imprecision.
MCTS is very methodological despite the name. Especially in the AlphaZero implementation, where there's no random play during inference / during games (!!). So where did the "Monte Carlo" name come from?
The "original" MCTS algorithm used random play to guestimate the strength of a position. But the "real" theory behind MCTS is the math revolving around multi-armed bandit problems. Given a large number of slot machines (in Chess or Go, the positions are conceptually seen as "slot machines" in a casino), the goal of MCTS is to find the slot-machine that gives the highest probability of winning.
In effect: classic MCTS rolls out a position all the way to the final result: win, loss, or draw. Not really in AlphaZero, but this is how MCTS was originally designed. AlphaZero shortcuts this process with a neural-network to guestimate the endgame result.
But still: the MATH, and the precise order which the search tree conducts is very heavily based in multi-armed bandit theory and random samples. Even if the algorithm in practice doesn't use randomization, all the math and understanding of the search tree is derived from randomized ideals.
I think MCTS well-represents the human thinking of the typical expert. Every play is read all the way to the end of the game instinctively. Experts study the pawn-king-rook endgame not necessarily because they expect to see pawn-king-rook endgames... but because the pawn-king-rook endgame is of huge importance to middle-game pawn development.
Learning to see how middle-game (and even early-game) moves develop into endgame is pretty much what being an expert is about. And MCTS best emulates that kind of thinking.
True, chess is very tactical. But humans are generally bad at exhaustive searches and tactics. Humans win with positional play, and with better understanding of endgame positions.
----------
So rest assured, when I say "Monte Carlo", I'm not talking about a typical monte-carlo algorithm. I'm talking about Monte Carlo Tree Search, which is a very, very different theoretical basis than Alpha-Beta pruning.
- java-man 8y agothank you very much for the explanation!