3 ms·

This is a preprint that was just uploaded to arxiv two days ago. Don't be hasty and assume that it settles any matter at all. Many such claims have been made be

This is a preprint that was just uploaded to arxiv two days ago. Don't be hasty and assume that it settles any matter at all. Many such claims have been made before and many counter-claims also. There is still a lively debate on the subject and it will be some time before there is agreement. More generally, any scholarly article is a claim, and should never be read as automatically true. That's something to keep in mind.

circuit10 3y ago
I thought it was completely obvious that if it can predict things about the world as well as it can it has some form of world model, even if inaccurate. I’m surprised that anyone would argue that a language model doesn’t model things
- YeGoblynQueenne 3y ago
  Yes, I see what you mean. It's frustrating but the word "model" is severely overused in Computer Science and AI and it can cause a lot of confusion. Briefly, a "world model" is a theory possessed by an autonomous agent that describes the entities that exist in the world and how they interact with each other and with the agent, and that the agent can use to make decisions. This is the sense in which "model" is used when people talk about "model based" approaches to AI (such as planning, for instance, which is "the model based approach to autonomous behaviour"). A "language model" on the other hand is a statistical model of the text in a corpus. A statistical model is really a set of events, and their probabilities. The "events" in a statistical language model are usually word collocations, where a collocation is a word A found near a word B, for some measure of "near". If you've heard about "word embeddings", that's a model of word collocations in text corpora. So the claim that LLMs "have a model" is about LLMs being autonomous agents with a world-model, while "Large Language Model" is about statistical modelling. There's no reason why a statistical language model should have a world model.
  - circuit10 3y ago
    Since language often describes the world, I think a good language model must include a world model
    - YeGoblynQueenne 3y ago
      Well, this has been claimed often, namely by some of the people who have developed statistical language models [1] but it's really not obvious how that should work. Where would a language model find the world model? Where would it store it? And why would it even need it? Obviously I don't know how the human linguistic ability works but it's clear that for us, text, words, language, isn't carrying around with it a representation of the entire world. So where would a language model find it in its training text? For example if I say "I sat by the sea today", there's no representation of the sea in the word "sea", rather, the representation is inside my head and yours, and the word "sea" is only used to point to that representation [2]. Why would it ever be possible to derive the representation just from the pointer? Like I say, it's really not obvious and I don't think anyone has given a convincing explanation - it's usually words all the way down when people try. In any case, it seems that humans had most of the capabilities conferred by a world model (which we share with other animals) before we had language (which we do not - share with other animals), so it seems likely that a world model must be developed first, before linguistic ability can be built on top of it. But that's a conjecture, of course. I guess maybe, in principle, it should be possible to treat all of human cognition as some kind of hidden (latent) variable and then train a model to represent it only from its expression in text, but we don't really have any kind of modelling technique with that power. That would be a truly omnipotent model. So powerful, indeed, that it wouldn't need to do that in the first place- it would already be a kind of machine god. ______________ [1] e.g. I believe Tomáš Mikolov, one of the creators of word2vec, has claimed something like that but I'm not sure if I have a source for my own claim; more recently Ilya Sutskever, of OpenAI, has said words to that effect- I have a link somewhere I think, if you are curious. [2] That's obvious because the word "sea" doesn't mean anything to someone who doesn't know English, but they'll probably have a word for the sea in their language, so they'll have the concept, without needing a specific word for it.

circuit10 3y ago

I thought it was completely obvious that if it can predict things about the world as well as it can it has some form of world model, even if inaccurate. I’m surprised that anyone would argue that a language model doesn’t model things

YeGoblynQueenne 3y ago
Yes, I see what you mean. It's frustrating but the word "model" is severely overused in Computer Science and AI and it can cause a lot of confusion. Briefly, a "world model" is a theory possessed by an autonomous agent that describes the entities that exist in the world and how they interact with each other and with the agent, and that the agent can use to make decisions. This is the sense in which "model" is used when people talk about "model based" approaches to AI (such as planning, for instance, which is "the model based approach to autonomous behaviour"). A "language model" on the other hand is a statistical model of the text in a corpus. A statistical model is really a set of events, and their probabilities. The "events" in a statistical language model are usually word collocations, where a collocation is a word A found near a word B, for some measure of "near". If you've heard about "word embeddings", that's a model of word collocations in text corpora. So the claim that LLMs "have a model" is about LLMs being autonomous agents with a world-model, while "Large Language Model" is about statistical modelling. There's no reason why a statistical language model should have a world model.
- circuit10 3y ago
  Since language often describes the world, I think a good language model must include a world model
  - YeGoblynQueenne 3y ago
    Well, this has been claimed often, namely by some of the people who have developed statistical language models [1] but it's really not obvious how that should work. Where would a language model find the world model? Where would it store it? And why would it even need it? Obviously I don't know how the human linguistic ability works but it's clear that for us, text, words, language, isn't carrying around with it a representation of the entire world. So where would a language model find it in its training text? For example if I say "I sat by the sea today", there's no representation of the sea in the word "sea", rather, the representation is inside my head and yours, and the word "sea" is only used to point to that representation [2]. Why would it ever be possible to derive the representation just from the pointer? Like I say, it's really not obvious and I don't think anyone has given a convincing explanation - it's usually words all the way down when people try. In any case, it seems that humans had most of the capabilities conferred by a world model (which we share with other animals) before we had language (which we do not - share with other animals), so it seems likely that a world model must be developed first, before linguistic ability can be built on top of it. But that's a conjecture, of course. I guess maybe, in principle, it should be possible to treat all of human cognition as some kind of hidden (latent) variable and then train a model to represent it only from its expression in text, but we don't really have any kind of modelling technique with that power. That would be a truly omnipotent model. So powerful, indeed, that it wouldn't need to do that in the first place- it would already be a kind of machine god. ______________ [1] e.g. I believe Tomáš Mikolov, one of the creators of word2vec, has claimed something like that but I'm not sure if I have a source for my own claim; more recently Ilya Sutskever, of OpenAI, has said words to that effect- I have a link somewhere I think, if you are curious. [2] That's obvious because the word "sea" doesn't mean anything to someone who doesn't know English, but they'll probably have a word for the sea in their language, so they'll have the concept, without needing a specific word for it.