No one can experience the real world directly. Whether it’s through language, vision (which feels quite “real” but is really just a 2D projection of light that

No one can experience the real world directly. Whether it’s through language, vision (which feels quite “real” but is really just a 2D projection of light that we interpret), sound or anything else, it’s all just some byproduct of the world that we nonetheless can make useful predictions with. LLMs are more limited in their input data, but I don’t think the difference is fundamental; it’s all just different abstract representations of the world. Like how by looking at the sea we can gain some useful understanding of how it works, an LLM can, via descriptions of the sea, pick up enough information about it to answer questions about it or write text about it. It wouldn’t “find” a world model, it learns it based on its training data, it gets stored in the weights (because there’s nowhere else to store it), and it needs it because it’s making predictions about text, which often describes the world. If you don’t have a basic idea of things like cause and effect it’s difficult to write coherent text