They build internal representations of the input only, and to the extent needed, to get the statistics right. This really isn't a "world model" of factual data,

They build internal representations of the input only, and to the extent needed, to get the statistics right. This really isn't a "world model" of factual data, but rather a "source model" of what would various sources (training texts) say. The responses of the model don't represent what it understands per some internal model, because there is no "it", only models of the sources it was trained on, and it'll just as happily generate lies as truths, or smart vs dumb answers (it's all just words) if that is what its source modelling calls for. What most people mean when they say the model is hallucinating/bullshitting isn't where it has learnt a lie, but rather where is is operating "out of distribution", and is therefore (unknowingly) generating a mashup from multiple only loosely related/matching source contexts.