Yes, I remember the argument. In computer chess the first AI system to dominate humans, IBM's DeepBlue, was based entirely on search and an opening book, so sea

Yes, I remember the argument. In computer chess the first AI system to dominate humans, IBM's DeepBlue, was based entirely on search and an opening book, so search + domain knowledge. Then AlphaGo which dominated in Go, was based on search, an opening book and a pair of self-playing deep neural nets and was equipped with the rules of Go. AlphaZero that followed, dropped the opening book but was still based on search and self-playing neural nets and was still given the rules of Go (and chess and Shoggi, if memory serves). And MuZero that followed that, dropped everything but the search and the self-playing neural nets. Now, Sutton argues that some of those systems at least did not rely on domain knowledge. They all did: Monte Carlo Tree Search, used for board game-playing AI agents, is nothing else but an encoding of domain knowledge - specifically, domain knowledge about the structure of two-player, complete information games. It's the same domain knowledge that was used to create the minimax-based DeepBlue software that won against Kasparov. Neural nets have still not managed to win against human players in board games without incorporating such a strong, knowledge-dependent component, as a game-tree search. So sutton is fudging the details. Not on purpose. He's an RL person. In RL, as in planning and other disciplines, folks tend to forget all the knowledge they put into their systems in the form of inductive biases, or auxiliary (but can't-do-without) algorithms like MCTS. He's like the proverbial fish that don't know what water is, because they swim in it.