I think the breakthrough is how AlphaZero combines reinforcement learning with Monte Carlo Tree search [1]. * evaluate a position by playing lots of random gam

I think the breakthrough is how AlphaZero combines reinforcement learning with Monte Carlo Tree search [1]. * evaluate a position by playing lots of random games until the end using "likely" moves (in their order of likelihood). this gives you a "win rate" (i.e. empirical probability of winning) that is used as a position's score. * train a neural net to predict that win rate based score (without having to play random games). * use that neural net to predict which moves are "likely", thereby making the playing of random games more efficient, yielding more expressive scores, which in turn again improve the accuracy of the network. * use the above described algorithm to self-play games, where every move is determined by looking at the win-rates given by the neural-net-driven quasi-random games, continuously training the score-estimating neural net. This is just my high-level understanding without knowing too much about the topic. Monte Carlo Tree search is a little more complex than what I try to explain here. [1] https://en.wikipedia.org/wiki/Monte_Carlo_tree_search https://en.wikipedia.org/wiki/Monte_Carlo_tree_search