3 ms·

World model based interfaces have an internal representation and when asked, describe its details. Surface statistics based interfaces have an internal databas

by mdp2021 2y ago

World model based interfaces have an internal representation and when asked, describe its details. Surface statistics based interfaces have an internal database of what is expected, and when asked, they give a conformist output.

naasking 2y ago
The point is that "internal database of statistical correlations" is a world model of sorts. We all have an internal representation of the world featuring only probabilistic accuracy after all. I don't think the distinction is as clear as you want it to be.
- mdp2021 2y ago
  > "internal database of statistical correlations" [would be] a world model of sorts Not in the sense used in the article: «memorizing “surface statistics”, i.e., a long list of correlations that do not reflect a causal model of the process generating the sequence». A very basic example: when asked "two plus two", would the interface reply "four" because it memorized a correlation of the two ideas, or because it counted at some point (many points in its development) and in that way assessed reality? That is a dramatic difference.
exe34 2y ago
> and when asked, describe its details. so humans don't typically have world models then. you ask most people how they arrived at their conclusions (outside of very technical fields) and they will confabulate just like an LLM. the best example is phenomenology, where people will grant themselves skills that they don't have, to reach conclusions. see also heterophenomenology, aimed at working around that: https://en.wikipedia.org/wiki/Heterophenomenology https://en.wikipedia.org/wiki/Heterophenomenology
- mdp2021 2y ago
  That the descriptive is not the prescriptive should not be a surprise. That random people will largely have suboptimal skills should not be a surprise. Yes, many people can't think properly. Proper thinking remains there as a potential.
  - exe34 2y ago
    > Yes, many people can't think properly. Proper thinking remains there as a potential. that's a matter of faith, not evidence. by that reasoning, the same can be said about LLMs. after all, they do occasionally get it right.
    - mdp2021 2y ago
      Let me rephrase it, there could be a misunderstanding: "Surely many people cannot think properly but some have much more ability than others: the proficient ability to think well is a potential (expressed in some and not expressed in many". To transpose that to LLMs, you should present one that systematically gets it right, not occasionally. And anyway, the point was about two different processes before statement formulation: some output the strongest correlated idea ("2+2" → "4"); some look at the internal model and check its contents ("2, 2" → "1 and 1, 1 and 1: 4").
      - exe34 2y ago
        > one that systematically gets it right, not occasionally. could Einstein systematically get new symphonies right? could Feynman create tasty new dishes every single time? Could ......
        mdp2021 2y ago
        > could Einstein systematically Did (could) Einstein think about things long and hard? Yes - that is how he explained having solved problems ("How did you do it?" // "I thought about long and hard"). The artificial system in question should (1) be able to do it, and (2) do it systematically, because it is artificial.

naasking 2y ago

The point is that "internal database of statistical correlations" is a world model of sorts. We all have an internal representation of the world featuring only probabilistic accuracy after all. I don't think the distinction is as clear as you want it to be.

mdp2021 2y ago
> "internal database of statistical correlations" [would be] a world model of sorts Not in the sense used in the article: «memorizing “surface statistics”, i.e., a long list of correlations that do not reflect a causal model of the process generating the sequence». A very basic example: when asked "two plus two", would the interface reply "four" because it memorized a correlation of the two ideas, or because it counted at some point (many points in its development) and in that way assessed reality? That is a dramatic difference.

exe34 2y ago

> and when asked, describe its details. so humans don't typically have world models then. you ask most people how they arrived at their conclusions (outside of very technical fields) and they will confabulate just like an LLM. the best example is phenomenology, where people will grant themselves skills that they don't have, to reach conclusions. see also heterophenomenology, aimed at working around that: https://en.wikipedia.org/wiki/Heterophenomenology https://en.wikipedia.org/wiki/Heterophenomenology

mdp2021 2y ago
That the descriptive is not the prescriptive should not be a surprise. That random people will largely have suboptimal skills should not be a surprise. Yes, many people can't think properly. Proper thinking remains there as a potential.
- exe34 2y ago
  > Yes, many people can't think properly. Proper thinking remains there as a potential. that's a matter of faith, not evidence. by that reasoning, the same can be said about LLMs. after all, they do occasionally get it right.
  - mdp2021 2y ago
    Let me rephrase it, there could be a misunderstanding: "Surely many people cannot think properly but some have much more ability than others: the proficient ability to think well is a potential (expressed in some and not expressed in many". To transpose that to LLMs, you should present one that systematically gets it right, not occasionally. And anyway, the point was about two different processes before statement formulation: some output the strongest correlated idea ("2+2" → "4"); some look at the internal model and check its contents ("2, 2" → "1 and 1, 1 and 1: 4").
    - exe34 2y ago
      > one that systematically gets it right, not occasionally. could Einstein systematically get new symphonies right? could Feynman create tasty new dishes every single time? Could ......
      - mdp2021 2y ago
        > could Einstein systematically Did (could) Einstein think about things long and hard? Yes - that is how he explained having solved problems ("How did you do it?" // "I thought about long and hard"). The artificial system in question should (1) be able to do it, and (2) do it systematically, because it is artificial.