> Connectionism is at heart a correlative methodology: it recognizes patterns in historical data and makes predictions accordingly, nothing more. Neural network

> Connectionism is at heart a correlative methodology: it recognizes patterns in historical data and makes predictions accordingly, nothing more. Neural networks do not develop semantic models about their environment; they cannot reason or think abstractly; they do not have any meaningful understanding of their inputs and outputs. And the symbolists make the same mistake they did the first time. Imbuing programs with nicely-named symbols and hard coded logic does not possess them with understanding, it arrests away their ability to learn it. Simple programs do not understand themselves, they have no more awareness of the logic they are running than a mouse has awareness of its neurons. Symbols and their programs represent understanding only as much as they map to concepts in the programmer's mind, not their own. Understanding must be something that partakes in computation, not the definition of the program. Classical chess AI, built with perfect chess simulators, idealized search and expert heuristics, despite flowing through interpretable and semantically sensibile programs that humans have built, are entirely isolated from the semantics of their programs—for the names and the layout of the data structures are not properties that the program itself has any access to. Despite the limitations of machine learning, this cannot unreservedly be said of neural networks, which are demonstrably extracting semantically meaningful latent spaces from highly complex inputs as part of their computation. ML is still not self-reflective in any useful sense (it cannot hear itself think; it does not see itself learn), but at least it is handling the first level of the task on top of which we might conceivably build understanding. To MuZero, a game of chess is an aspect of its network that it could perceive, at least within a given branch of its search. And we know these networks must be building something at least knowledge-analogous, since how else could a network like GPT-2 answer questions (however imperfectly) across a range of out-of-domain tasks, like knowledge retrieval and translation? And this is why Gary Marcus' position (besides his repeated telling of false claims) misses the point. Yes, we should embed programs with priors and reasoning beyond brute connectionism—and to that, most people agree;—but this understanding cannot live in the symbols, it must by necessity live in the structure of the computation, and this structure must itself be accessible to the AI. It is this latter thing that the ML community is already doing, in likely the majority of ML papers, of which the convolutions Gary likes so much are just the tip of the mountain. It is this latter thing that explains, much against the grain of Gary's claims, why MuZero is a better network than AlphaZero.