7 ms·

LLMs keep demonstrating that they have an internal "world model" representation of concepts and the relationships between them. Meanwhile ostensible "expert" h

LLMs keep demonstrating that they have an internal "world model" representation of concepts and the relationships between them. Meanwhile ostensible "expert" humans continue to regurgitate obvious falsehoods like "it's just a fancy autocomplete" or "you're being fooled by a system that just predicts the most probable next word".

anticensor 4y ago
That world model is an emergent phenomenon arisen out of training, they are not explicitly trained a world model.
- DougBTX 4y ago
  I’d phrase it the other way around, learning a world model is necessary in order to make such good token predictions.
flir 4y ago
I don't see why "internal model" and "predicts the most probable next word" are mutually exclusive.
- panarky 4y ago
  Who said they're mutually exclusive?
Jensson 4y ago
The internal "world model" are things like "GPU's are faster than CPU's", "Military hardware beats nonmilitary things" etc. Then when you ask the comparison question it categorises the items, and for the plane vs card you get military vs nonmilitary response. Or computer vs gpu you get the cpu vs gpu response or similar. I showed the example of how to reason about this in a post below. Anyway, while you can call that a "world model", that world model is very brittle since its just a bunch of heuristics, if you reverse engineer the heuristic it uses it is really easy to find cases where it fails completely. Like if you ask it to compare a modern integrated GPU to an old dedicated GPU, it will say that the old dedicated GPU is faster even though it is ancient and slow, since when people talk about dedicated vs integrated GPU they almost always say that the dedicated GPU is faster.
throwaway4aday 4y ago
It doesn't have a model it is a model and it's a model of language. It's never interacted with the world, only with text. There are patterns and relationships among words that can describe the real world, that's why we can use it to communicate complex ideas. The problem is that the map is not the territory.
- kybernetikos 4y ago
  You've never interacted with the world, only with the sense impressions that were triggered in your nervous system. All world models are inferred.
  - throwaway4aday 4y ago
    Bad argument. My sense impressions consist of far more than just text input. We don't know if consciousness consists of one model, multiple models or is something different from a statistical model but makes use of one or more of them. I also don't consist of only a nervous system, I also have an endocrine system that has very pronounced effects on my behaviour and my interpretation of input. There's a giant gap here that people keep glossing over when they treat LLMs as almost-human.
    - kybernetikos 4y ago
      Sense impressions are mediated by fairly simple neuronal firing mechanisms, somewhat different, but similar in concept to the wires we use to encode language tokens. I'm not really trying to make a point here about consciousness, although I was interested by the article - it seems to imply that gpt-3 performs better at some tasks when it has a train of thought / stream of consciousness. People keep saying that there's a giant gap, but I still don't really know what they think it is, and there seem to be a lot of points of similarity. It's very clear that these large language models are, despite their problems, much more capable than was expected by nearly everyone not very many years ago.
      - throwaway4aday 4y ago

anticensor 4y ago

That world model is an emergent phenomenon arisen out of training, they are not explicitly trained a world model.

DougBTX 4y ago
I’d phrase it the other way around, learning a world model is necessary in order to make such good token predictions.

flir 4y ago

I don't see why "internal model" and "predicts the most probable next word" are mutually exclusive.

panarky 4y ago
Who said they're mutually exclusive?

Jensson 4y ago

The internal "world model" are things like "GPU's are faster than CPU's", "Military hardware beats nonmilitary things" etc. Then when you ask the comparison question it categorises the items, and for the plane vs card you get military vs nonmilitary response. Or computer vs gpu you get the cpu vs gpu response or similar. I showed the example of how to reason about this in a post below. Anyway, while you can call that a "world model", that world model is very brittle since its just a bunch of heuristics, if you reverse engineer the heuristic it uses it is really easy to find cases where it fails completely. Like if you ask it to compare a modern integrated GPU to an old dedicated GPU, it will say that the old dedicated GPU is faster even though it is ancient and slow, since when people talk about dedicated vs integrated GPU they almost always say that the dedicated GPU is faster.

throwaway4aday 4y ago

It doesn't have a model it is a model and it's a model of language. It's never interacted with the world, only with text. There are patterns and relationships among words that can describe the real world, that's why we can use it to communicate complex ideas. The problem is that the map is not the territory.

kybernetikos 4y ago
You've never interacted with the world, only with the sense impressions that were triggered in your nervous system. All world models are inferred.
- throwaway4aday 4y ago
  Bad argument. My sense impressions consist of far more than just text input. We don't know if consciousness consists of one model, multiple models or is something different from a statistical model but makes use of one or more of them. I also don't consist of only a nervous system, I also have an endocrine system that has very pronounced effects on my behaviour and my interpretation of input. There's a giant gap here that people keep glossing over when they treat LLMs as almost-human.
  - kybernetikos 4y ago
    Sense impressions are mediated by fairly simple neuronal firing mechanisms, somewhat different, but similar in concept to the wires we use to encode language tokens. I'm not really trying to make a point here about consciousness, although I was interested by the article - it seems to imply that gpt-3 performs better at some tasks when it has a train of thought / stream of consciousness. People keep saying that there's a giant gap, but I still don't really know what they think it is, and there seem to be a lot of points of similarity. It's very clear that these large language models are, despite their problems, much more capable than was expected by nearly everyone not very many years ago.
    - throwaway4aday 4y ago