> I thought you described a system that did what MuZero did. It still seems to me that's what you're describing. I can see your reading of it. My point wasn't

> I thought you described a system that did what MuZero did. It still seems to me that's what you're describing. I can see your reading of it. My point wasn't totally clear. What I was getting at is a system that automatically induces internal computational graphs based on its competence with natural language and its facility with solving planning problems. It's essentially a more general MuZero that is driven by natural language and can solve general computational problems over its entire learned world model. For example, when asking what is the 256th fibonacci number, instead of either retrieving from memory or confabulating a correct looking number, it constructs the right sequence of computational steps from its representation of the fibonacci sequence and runs through it. The difference is largely one of having the components of the system be self-directed through internal planning mechanisms, i.e. a cybernetic governor that can plan out general computational strategies. Natural language is a general representational framework and so a planning and computing facility over natural language representations is in my view a general problem solving framework. >To clarify, "in-context learning" is what we used to call "sampling from a model" back in my time [...] Models got better and sampling from them gets better results, so OpenAI figured they could embiggen it and call it something more impressive than "sampling". But it's sampling, not learning. The way I interpret this (analogous to sampling from a probability distribution), there is certainly a relationship between the two. But the difference is that in-context learning is much more general and flexible than anything that has come before. When the range of behavior far surpasses what has come before, its important to recognize the paradigm shift. There's a danger of expertise in old paradigms causing resistance to seeing these qualitative shifts. As Max Planck said, science progresses one funeral at a time. I can't help but see a case of this resistance in those who are downplaying the qualitative shift that in-context learning represents. >The paper you linked describes a system that was explicitly trained to predict human moves. That's probably why it can do better at simulating the way humans play chess than a system that wasn't trained to simulate the way humans play chess. The evaluation in the paper wasn't just for the network tuned to match human performance, it was also tested against Stockfish and Leela. Leela more closely matched human move choices at all human skill levels. It was also widely commented at the time that AlphaZero played a more human style of chess. The point is that this says something about how humans play chess, i.e. heavier on positional evaluation and light on calculation.