3 ms·

It turns out our word for "surface statistics" is "world model".

by dboreham 2y ago

marcosdumay 2y ago
Well, for some sufficiently platonic definition of "world".
- mdp2021 2y ago
  In a way the opposite, I'd say: the archetypes in Plato are the most stable reality and are akin to the logos that the past and future tradition hunted - knowing it is to know how things are (how things work), hence knowledge of the state of things, hence a faithful world model. To utter conformist statements spawned from surface statistics would be "doxa" - repeating "opinions".
  - marcosdumay 2y ago
    It has a profound and extensive knowledge about something. But that "something" is how words follow each other on popular media. LLMs are very firmly stuck inside the Cave Allegory.
    - mdp2021 2y ago
      If you mean that just like the experiencer in the cave, seeing shadows instead of things (really, things instead of Ideas), the machine sees words instead of things, that would be in a way very right. But we could argue it could not be impossible to create an ontology (a very descriptive ontology - "this is said to be that, and that, and that...") from language alone. Hence the question whether the ontology is there. (Actually, the question at this stage remains: "How do they work - in sufficient detail? Why the appearance of some understanding?")
      - marcosdumay 2y ago
        Yeah, what I'm saying is that something very similar to an ontology is there. (It's incomplete but extensive, not coherent, and it's deeper in details than anything anybody ever created.) It's just that it's a kind of a useless ontology, because the reality it's describing is language. Well, only "kind of useless" because it should be very useful to parse, synthesize and transform language. But it doesn't have the kind of "knowledge" that most people expect an intelligence to have. Also, its world isn't only composed of words. All of them got a very strong "Am I fooling somebody?" signal during training.
mdp2021 2y ago
World model based interfaces have an internal representation and when asked, describe its details. Surface statistics based interfaces have an internal database of what is expected, and when asked, they give a conformist output.
- naasking 2y ago
  The point is that "internal database of statistical correlations" is a world model of sorts. We all have an internal representation of the world featuring only probabilistic accuracy after all. I don't think the distinction is as clear as you want it to be.
  - mdp2021 2y ago
    > "internal database of statistical correlations" [would be] a world model of sorts Not in the sense used in the article: «memorizing “surface statistics”, i.e., a long list of correlations that do not reflect a causal model of the process generating the sequence». A very basic example: when asked "two plus two", would the interface reply "four" because it memorized a correlation of the two ideas, or because it counted at some point (many points in its development) and in that way assessed reality? That is a dramatic difference.
- exe34 2y ago
  > and when asked, describe its details. so humans don't typically have world models then. you ask most people how they arrived at their conclusions (outside of very technical fields) and they will confabulate just like an LLM. the best example is phenomenology, where people will grant themselves skills that they don't have, to reach conclusions. see also heterophenomenology, aimed at working around that: https://en.wikipedia.org/wiki/Heterophenomenology https://en.wikipedia.org/wiki/Heterophenomenology
  - mdp2021 2y ago
    That the descriptive is not the prescriptive should not be a surprise. That random people will largely have suboptimal skills should not be a surprise. Yes, many people can't think properly. Proper thinking remains there as a potential.

It turns out our word for "surface statistics" is "world model".

marcosdumay 2y ago

Well, for some sufficiently platonic definition of "world".

mdp2021 2y ago
In a way the opposite, I'd say: the archetypes in Plato are the most stable reality and are akin to the logos that the past and future tradition hunted - knowing it is to know how things are (how things work), hence knowledge of the state of things, hence a faithful world model. To utter conformist statements spawned from surface statistics would be "doxa" - repeating "opinions".
- marcosdumay 2y ago
  It has a profound and extensive knowledge about something. But that "something" is how words follow each other on popular media. LLMs are very firmly stuck inside the Cave Allegory.
  - mdp2021 2y ago
    If you mean that just like the experiencer in the cave, seeing shadows instead of things (really, things instead of Ideas), the machine sees words instead of things, that would be in a way very right. But we could argue it could not be impossible to create an ontology (a very descriptive ontology - "this is said to be that, and that, and that...") from language alone. Hence the question whether the ontology is there. (Actually, the question at this stage remains: "How do they work - in sufficient detail? Why the appearance of some understanding?")
    - marcosdumay 2y ago
      Yeah, what I'm saying is that something very similar to an ontology is there. (It's incomplete but extensive, not coherent, and it's deeper in details than anything anybody ever created.) It's just that it's a kind of a useless ontology, because the reality it's describing is language. Well, only "kind of useless" because it should be very useful to parse, synthesize and transform language. But it doesn't have the kind of "knowledge" that most people expect an intelligence to have. Also, its world isn't only composed of words. All of them got a very strong "Am I fooling somebody?" signal during training.

mdp2021 2y ago

World model based interfaces have an internal representation and when asked, describe its details. Surface statistics based interfaces have an internal database of what is expected, and when asked, they give a conformist output.

naasking 2y ago
The point is that "internal database of statistical correlations" is a world model of sorts. We all have an internal representation of the world featuring only probabilistic accuracy after all. I don't think the distinction is as clear as you want it to be.
- mdp2021 2y ago
  > "internal database of statistical correlations" [would be] a world model of sorts Not in the sense used in the article: «memorizing “surface statistics”, i.e., a long list of correlations that do not reflect a causal model of the process generating the sequence». A very basic example: when asked "two plus two", would the interface reply "four" because it memorized a correlation of the two ideas, or because it counted at some point (many points in its development) and in that way assessed reality? That is a dramatic difference.
exe34 2y ago
> and when asked, describe its details. so humans don't typically have world models then. you ask most people how they arrived at their conclusions (outside of very technical fields) and they will confabulate just like an LLM. the best example is phenomenology, where people will grant themselves skills that they don't have, to reach conclusions. see also heterophenomenology, aimed at working around that: https://en.wikipedia.org/wiki/Heterophenomenology https://en.wikipedia.org/wiki/Heterophenomenology
- mdp2021 2y ago
  That the descriptive is not the prescriptive should not be a surprise. That random people will largely have suboptimal skills should not be a surprise. Yes, many people can't think properly. Proper thinking remains there as a potential.