3 ms·
And the structure of writing in the Internet includes a lot about the world, so a good model of that must contain some form of world model to make accurate pred
by circuit10 3y ago
And the structure of writing in the Internet includes a lot about the world, so a good model of that must contain some form of world model to make accurate predictions
- JohnFen 3y agoThe only model necessary is that of the writing. So if the writing so far is "He went to the bar", the LLM can know just from the language model that a reasonable next fragment could be "...to have a drink". That doesn't require or imply that the concept of "bar", "drink", or even "He" or the concept of motion has been modelled. The model is of the patterns of tokens (words/characters/whatever).