When trained on simple logs of Othello's moves, the model learns an internal representation of the board and its pieces. It also models the strength of its oppo

When trained on simple logs of Othello's moves, the model learns an internal representation of the board and its pieces. It also models the strength of its opponent. https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 I'd be more surprised if LLMs trained on human conversations don't create any world models. Having a world model simply allows the LLM to become better at sequence prediction. No magic needed. There was another recent paper that shows that a language model is modelling things like age, gender, etc., of their conversation partner without having been explicitly trained for it