The crow never analyzes any board - that's the point of the analogy. It only hears a sequence of tokens, and it learns to predict future tokens, and TFA sets ou

The crow never analyzes any board - that's the point of the analogy. It only hears a sequence of tokens, and it learns to predict future tokens, and TFA sets out to answer whether it does that by "surface statistics" (i.e. which tokens are more likely at which times), or with a "world model" (by laying out a bunch of seeds and flipping some of them over after each token in order to simulate the state of the game). Edit: from other replies, I think I see the issue here. In TFA, "world model" is referring to the insight that "D4, E5, B3" are more than just tokens in a grammar - that they represent graph nodes with state, and that when each token occurs it toggles the states of other graph nodes. TFA is asking whether the LLM has learned that insight, as opposed to merely detecting patterns in the token sequences without any model of what they represent. You seem to be taking "world model" to mean something quite different - like "a rule-based understanding of game rules that isn't statistical" or such. TFA doesn't discuss or investigate anything along those lines, just whether the LLM is simulating the on/off state of Othello pieces.