6 ms·

Totally agree. I think Juergen Schmidthuber has developed a lot of ideas around compression being the basis for consciousness and understanding. There was the

Totally agree. I think Juergen Schmidthuber has developed a lot of ideas around compression being the basis for consciousness and understanding. There was the paper that showed that when showing a language model Othello moves it ends up building an internal representation of the board. And now I was reading this abstract: ```Theory of mind (ToM), or the ability to impute unobservable mental states to others, is central to human social interactions, communication, empathy, self-consciousness, and morality. We administer classic false-belief tasks, widely used to test ToM in humans, to several language models, without any examples or pre-training. Our results show that models published before 2022 show virtually no ability to solve ToM tasks. Yet, the January 2022 version of GPT-3 (davinci-002) solved 70% of ToM tasks, a performance comparable with that of seven-year-old children. Moreover, its November 2022 version (davinci-003), solved 93% of ToM tasks, a performance comparable with that of nine-year-old children. These findings suggest that ToM-like ability (thus far considered to be uniquely human) may have spontaneously emerged as a byproduct of language models' improving language skills.```

adammarples 4y ago
Language is how people demonstrate that they have a good theory of mind. Solving these tasks is a proxy for people's internal theories. Language is a good way to demonstrate that ChatGPT has a good statistical model for what token comes next. Humans are not good at solving TOM tasks because they're good at guessing what token comes next, ChatGPT is not good at solving them because is has a good TOM. They're different inside.
- bottlepalm 4y ago
  What ‘statistical model’? ChatGPT isn’t a big Markov chain, it’s a neural net. It was trained on ‘full input’ transformed to ‘full output’. Generating the ‘next word’ is implementation details as it already knows the bigger idea it is trying to explain.
  - adammarples 4y ago
    By a statistical model I mean a very large, highly dimensional data manifold on which classification is done to predict the next token. The correct token can be predicted very often because somehow the common patterns in the written training data are being efficiently compressed in this highly dimensional space. Humans are also able to efficiently compress the data they are trained on, but the structures inside a human and a transformer are completely different, what happens at inference time is completely different. They both have language in common as an output and can both answer TOM questions but this doesn't mean they both have a theory of mind.
    - bottlepalm 4y ago
      Why do you think the wiring of a biological and artificial neural net are completely different? The results from both are remarkably similar.
      - adammarples 4y ago
        We know from neuroscience that real neurons are very different from ANN neurons. For example, brains don't even use backpropagation. The neurons are connected differently. They have way more going on than just activation. Humans and ML models behave very differently.

adammarples 4y ago

Language is how people demonstrate that they have a good theory of mind. Solving these tasks is a proxy for people's internal theories. Language is a good way to demonstrate that ChatGPT has a good statistical model for what token comes next. Humans are not good at solving TOM tasks because they're good at guessing what token comes next, ChatGPT is not good at solving them because is has a good TOM. They're different inside.

bottlepalm 4y ago
What ‘statistical model’? ChatGPT isn’t a big Markov chain, it’s a neural net. It was trained on ‘full input’ transformed to ‘full output’. Generating the ‘next word’ is implementation details as it already knows the bigger idea it is trying to explain.
- adammarples 4y ago
  By a statistical model I mean a very large, highly dimensional data manifold on which classification is done to predict the next token. The correct token can be predicted very often because somehow the common patterns in the written training data are being efficiently compressed in this highly dimensional space. Humans are also able to efficiently compress the data they are trained on, but the structures inside a human and a transformer are completely different, what happens at inference time is completely different. They both have language in common as an output and can both answer TOM questions but this doesn't mean they both have a theory of mind.
  - bottlepalm 4y ago
    Why do you think the wiring of a biological and artificial neural net are completely different? The results from both are remarkably similar.
    - adammarples 4y ago
      We know from neuroscience that real neurons are very different from ANN neurons. For example, brains don't even use backpropagation. The neurons are connected differently. They have way more going on than just activation. Humans and ML models behave very differently.