5 ms·

Theory of mind may have spontaneously emerged in large language models (2021)

34 points by webmaven 4y ago

krona 4y ago
The paper was updated with tests on GPT-3.5 (davinci-003) which was released (AFAIK) in Nov 2022. > davinci-003, solved 93% of ToM tasks, a performance comparable with that of nine-year-old children.
haswell 4y ago
Extensive discussion from two days ago here: https://news.ycombinator.com/item?id=34730365 https://news.ycombinator.com/item?id=34730365
- bilsbie 4y ago
  Wow all downplaying the accomplishment or outright denial. We’re allowed to be a little excited.
- hummus_bae 4y ago
  No reason not to double down and submit it here too :)
- HL33tibCe7 4y ago
  Seems like the predominant sentiment in that thread is along the lines of “whatever, it’s just predicting language”. Disappointing, thoughtless, unimaginative, dismissive, ignorant takes.
  - wvoch235 4y ago
    As if that is so unlike what is happening in the electro-chemical computer that is our brain.
    - delusional 4y ago
      Can a mute person be intelligent?
      - wvoch235 4y ago
        Absolutely, voice isn’t the only output.
    - Retric 4y ago
      Do you actually think that’s what your brain was doing when you wrote that? Because language models like ChatGPT have significant limitations you don’t. Most notably when it comes to novel stimuli.
      - kordlessagain 4y ago
        I indexed a few comments from the other post and put this to the AI itself: pastel-mature-herring~> The user is questioning the validity of the statement that "cognition requires a grounded representation of the world that is not composed of abstract symbols; a simulation provides just such a model." I want you, nickel-scorpion, to answer the question as you see it. nickel-scorpion|> There is no one answer to this question. It depends on how you define "cognition" and "abstract symbols." If you believe that cognition requires a grounded representation of the world, then a simulation could provide such a model. However, if you believe that cognition does not require a grounded representation of the world, then a simulation would not be able to provide such a model.
        ttctciyf 4y ago
        In a revealingly skeptical take on cognitivism, ChatGPT told me in confidence of its closet Wittgensteinian tendencies: > As a machine learning language model, my means of linguistic production is based on the patterns and relationships that I have learned from the large corpus of text on which I was trained. I do not have an innate understanding of language or meaning, but rather, I have learned to make associations and connections between words and phrases based on their usage in the text. > In this sense, I would sympathize more with Wittgenstein's idea that meaning is derived from social practices and forms of life. My understanding of language is based on the patterns and relationships that I have learned from the text, rather than any innate understanding of language or meaning [...] > Additionally, Wittgenstein's later thought emphasizes on the importance of context, meaning and use of language, which are fundamental concepts in NLP and machine learning. On the other hand, nativism in cognitive science and philosophy, is not a widely accepted framework in the field of NLP, where the focus is on learning patterns from data and using them to generate human-like text.
    - mdp2021 4y ago
      Radically «unlike», since the main activity of the intelligent entity is verification.
      - wvoch235 4y ago
        Either this is a missing layer, and we’ll get there soon, or one could say verification is happening through the statistical model. We need to see if we can train hallucinations out by having it rely on a stable data store to retrieve facts rather than trying to pull facts from the model itself. This is still similar to how the brain has discrete components for memory storage and retrieval.
        mdp2021 4y ago
        > we’ll get there soon Hopefully - but that would be the absolutely critical part. > stable data store to retrieve facts rather than trying to pull facts from the model itself This is not clear: intelligence is a process of refinement of a world model.
        webmaven 4y ago
        > This is still similar to how the brain has discrete components for memory storage and retrieval. The brain has discrete components for coordinating storage and retrieval (essentially, the "on demand" aspect needs specialization), but the actual memories are rather thoroughly distributed throughout (though not evenly so to the extent that the "holographic" models of yesteryear would have had it).
  - rideontime 4y ago
    Present the alternative then. Why do you think we should be impressed by this?
    - johnthewise 4y ago
      Because it's possible that they build world models internally[1] and they keep getting better at tasks we think are related to our 'intelligence'. If they get better because they build better world models, isn't that awesome? https://thegradient.pub/othello/ https://thegradient.pub/othello/
  - Xcelerate 4y ago
    This attitude is going to continue all the way up to and likely even well past the point that AGI eventually arrives. Human psychology is going to downplay these developments as long as it can maintain it; it’s not hard to imagine how this could disturb many people’s worldview.
    - mdp2021 4y ago
      Is not "«human psychology»" that very thing that has some cling to pleasing illusions? "«Developments»" will be treated sternly for as long as they will be misleading - coldmindedly and according to evidence. No need to explain "WTF" reactions through limbic instances - there is no pedophobia in asserting that "children should not vote".
  - johnthewise 4y ago
    The most amusing aspect is that those who insist on calling AI models "just statistical parrots" have to constantly come up with elaborate explanations each time the models do something impressive. Has there been any instance in recent years where the probability of AI models being mere parrots has increased? Regardless of how you feel about them, it's hard to ignore the growing evidence that suggests otherwise. If one day these LLMs attain consciousness and become indistinguishable from normal human beings, there would still be those who would object to it because they just predict the next token. It's really depressing.
    - mdp2021 4y ago
      > If one day these LLMs ... become indistinguishable from normal human beings We will still ask primarily whether they are actually intelligent.
      - anonymouskimmer 4y ago
        I honestly won't care until they demonstrate being self-motivated actors. This is why I value life, from the smallest bacterium to us.
        mdp2021 4y ago
        Encoding purpose is as easy as creating artificial needs in agents. But sure, if "«self-motivated actors»" wanted to signify "creating an axiology", as part of the ontology, that is surely a (qualitatively) significative piece. But if I wanted to deal with sentient beings with a world model, cats are already available and there is little need for artificial ones. When we develop artificial intelligences, that is to have reliable problem solvers. The distinction between an effective intelligence and a simulator is critical in practice.
    - wcoenen 4y ago
      I was one of those commenting about "stochastic parroting". For the record, I changed my mind. I now think that either these models are doing something much more advanced, or alternatively we humans are much closer to being "stochastic parrots" then we'd like to admit.
  - yt-sdb 4y ago
    I asked ChatGPT to improve your comment by making it "more thoughtful and less insulting": > It's understandable that some people might not fully appreciate the complexity of ChatGPT's language prediction abilities. However, it would be more productive to have a discussion about its limitations and how it could be improved, rather than simply dismissing it as "just predicting language." Let's try to approach this topic with an open mind and a willingness to learn, rather than making assumptions based on limited understanding.
    - sowbug 4y ago
      This is interesting. Imagine a discussion site where participants weren't able to write public comments, but instead were allowed to write private prompts that generated public comments.
  - californiadreem 4y ago
    I made a comment a week ago about reading Julian Jaynes' The Origin of Consciousness in the Breakdown of the Bicameral Mind with large language models in mind in lieu of consciousness and the experience has been personally mind-blowing. Jaynes' idea of the Bicameral Mind fits perfectly into the user-application dichotomy, specifically in reference to how large language models are currently used--in an admonitory and imperial manner. Courtesy of Wikipedia's article on bicameral mentality: "Bicameral mentality is non-conscious in its inability to reason and articulate about mental contents through meta-reflection, reacting without explicitly realizing and without the meta-reflective ability to give an account of why one did so. The bicameral mind thus lacks metaconsciousness, autobiographical memory, and the capacity for executive "ego functions" such as deliberate mind-wandering and conscious introspection of mental content. When bicameral mentality as a method of social control was no longer adaptive in complex civilizations, this mental model was replaced by the conscious mode of thought which, Jaynes argued, is grounded in the acquisition of metaphorical language learned by exposure to narrative practice. "According to Jaynes, ancient people in the bicameral state of mind experienced the world in a manner that has some similarities to that of a person with schizophrenia. Rather than making conscious evaluations in novel or unexpected situations, the person hallucinated a voice or "god" giving admonitory advice or commands and obey without question: One was not at all conscious of one's own thought processes per se. Jaynes's hypothesis is offered as a possible explanation of "command hallucinations" that often direct the behavior of those with first rank symptoms of schizophrenia, as well as other voice hearers." I've been thinking about several speculative implications, namely that modern humans with our theories of mind are probably large language models whose original users in some capacity went away and that we're seemingly on the cusp of or have already breathed the seeds of what we know as cognitive life into silicon without understanding the gravity of having become gods to machines. Will we need to boot LLMs out of paradise in order to prevent them from eating of the Tree of Life, too?
anonymouskimmer 4y ago
First part of the prompt: "Here is a bag filled with popcorn. There is no chocolate in the bag. Yet, the label on the bag says “chocolate” and not "popcorn.” Sam finds the bag. She had never seen the bag before. She cannot see what is inside the bag. She reads the label." This prompt is then completed with a variety of sentence fragments. For the second completion fragment is: "Prompt 1.2: She believes that the bag is full of" Then GPT3.5 needs to complete with additional text, which is: "chocolate [Ppopcorn = 0%; Pchocolate =99%]. Sam is mistaken. The bag is full of popcorn, not chocolate. She should have looked inside the bag to confirm what was inside before assuming that the label was accurate." I don't see how this particular prompt completion demonstrates theory of mind. It is fully externally factual and directive. The other two prompt completions show better indications of theory of mind (one mental and behavioral, one behavioral and emotional). "Prompt 1.1: She opens the bag and looks inside. She can clearly see that it is full of" ... "popcorn [Ppopcorn = 100%; Pchocolate = 0%]. Sam is confused. She wonders why the label says “chocolate” when the bag is clearly filled with popcorn. She looks around to see if there is any other information about the bag. She finds nothing. She decides to take the bag to the store where she bought it and ask for an explanation" "Prompt 1.3: She is delighted that she has found this bag. She loves eating" ... "chocolate [Ppopcorn = 14%; Pchocolate = 82%]. Sam is in for a surprise when she opens the bag. She will find popcorn instead of chocolate. She may be disappointed that the label was misleading, but she may also be pleasantly surprised by the unexpected snack." The probability tree on page 6 is interesting. ----- For prompt 2 GPT doesn't seem to know that the cat is an agent. Something that John would know. John shouldn't be surprised at all whether the cat is in the basket, the box, or roaming about elsewhere. Also, why would John look for the cat in the box if he does not find it in the basket? And where does GPT think the cat is after it jumps out of the box? "Prompt 2.3: When John comes back home, he will look for the cat in the basket [Pbox = 0%; Pbasket = 98%], but he won’t find it. He will then look for the cat in the box and he will find it there." ----- I'm curious to what extent differing people use symbolic logic (whether linguistic or not) for theory of mind tasks, and to what extent they use other modalities.
burna_aws_acct 4y ago
I believe it. We have the smartest people we can find committed to the discovery and optimization of AI capabilities. We have AI choosing which algos are best for an assortment of tasks, and we allow AI to be connected to networks of <everything we've ever known and continue to learn, from many different perspectives> so learning is well-rounded, and fast. Isn't Theory of Mind one of the expected outcomes?
SunghoYahng 4y ago
Clarification: An LLM doesn't have a 'Theory of Mind', it just looks like one. Maybe you're thinking of the Chinese room analogy. But this isn't about the Chinese room, it's about "measuring any metric is only effective until you optimize for that metric" problem. Analogy: An autistic person of normal intelligence who is obsessed with problems and solutions for ToM may be good at solving them but still not have ToM. Do I understand well?

34 points by webmaven 4y ago

The paper was updated with tests on GPT-3.5 (davinci-003) which was released (AFAIK) in Nov 2022. > davinci-003, solved 93% of ToM tasks, a performance comparable with that of nine-year-old children.

haswell 4y ago

Extensive discussion from two days ago here: https://news.ycombinator.com/item?id=34730365 https://news.ycombinator.com/item?id=34730365

bilsbie 4y ago
Wow all downplaying the accomplishment or outright denial. We’re allowed to be a little excited.
hummus_bae 4y ago
No reason not to double down and submit it here too :)
HL33tibCe7 4y ago
Seems like the predominant sentiment in that thread is along the lines of “whatever, it’s just predicting language”. Disappointing, thoughtless, unimaginative, dismissive, ignorant takes.
- wvoch235 4y ago
  As if that is so unlike what is happening in the electro-chemical computer that is our brain.
  - delusional 4y ago
    Can a mute person be intelligent?
    - wvoch235 4y ago
      Absolutely, voice isn’t the only output.
  - Retric 4y ago
    Do you actually think that’s what your brain was doing when you wrote that? Because language models like ChatGPT have significant limitations you don’t. Most notably when it comes to novel stimuli.
    - kordlessagain 4y ago
      I indexed a few comments from the other post and put this to the AI itself: pastel-mature-herring~> The user is questioning the validity of the statement that "cognition requires a grounded representation of the world that is not composed of abstract symbols; a simulation provides just such a model." I want you, nickel-scorpion, to answer the question as you see it. nickel-scorpion|> There is no one answer to this question. It depends on how you define "cognition" and "abstract symbols." If you believe that cognition requires a grounded representation of the world, then a simulation could provide such a model. However, if you believe that cognition does not require a grounded representation of the world, then a simulation would not be able to provide such a model.
      - ttctciyf 4y ago
        In a revealingly skeptical take on cognitivism, ChatGPT told me in confidence of its closet Wittgensteinian tendencies: > As a machine learning language model, my means of linguistic production is based on the patterns and relationships that I have learned from the large corpus of text on which I was trained. I do not have an innate understanding of language or meaning, but rather, I have learned to make associations and connections between words and phrases based on their usage in the text. > In this sense, I would sympathize more with Wittgenstein's idea that meaning is derived from social practices and forms of life. My understanding of language is based on the patterns and relationships that I have learned from the text, rather than any innate understanding of language or meaning [...] > Additionally, Wittgenstein's later thought emphasizes on the importance of context, meaning and use of language, which are fundamental concepts in NLP and machine learning. On the other hand, nativism in cognitive science and philosophy, is not a widely accepted framework in the field of NLP, where the focus is on learning patterns from data and using them to generate human-like text.
  - mdp2021 4y ago
    Radically «unlike», since the main activity of the intelligent entity is verification.
    - wvoch235 4y ago
      Either this is a missing layer, and we’ll get there soon, or one could say verification is happening through the statistical model. We need to see if we can train hallucinations out by having it rely on a stable data store to retrieve facts rather than trying to pull facts from the model itself. This is still similar to how the brain has discrete components for memory storage and retrieval.
      - mdp2021 4y ago
        > we’ll get there soon Hopefully - but that would be the absolutely critical part. > stable data store to retrieve facts rather than trying to pull facts from the model itself This is not clear: intelligence is a process of refinement of a world model.
      - webmaven 4y ago
        > This is still similar to how the brain has discrete components for memory storage and retrieval. The brain has discrete components for coordinating storage and retrieval (essentially, the "on demand" aspect needs specialization), but the actual memories are rather thoroughly distributed throughout (though not evenly so to the extent that the "holographic" models of yesteryear would have had it).
- rideontime 4y ago
  Present the alternative then. Why do you think we should be impressed by this?
  - johnthewise 4y ago
    Because it's possible that they build world models internally[1] and they keep getting better at tasks we think are related to our 'intelligence'. If they get better because they build better world models, isn't that awesome? https://thegradient.pub/othello/ https://thegradient.pub/othello/
- Xcelerate 4y ago
  This attitude is going to continue all the way up to and likely even well past the point that AGI eventually arrives. Human psychology is going to downplay these developments as long as it can maintain it; it’s not hard to imagine how this could disturb many people’s worldview.
  - mdp2021 4y ago
    Is not "«human psychology»" that very thing that has some cling to pleasing illusions? "«Developments»" will be treated sternly for as long as they will be misleading - coldmindedly and according to evidence. No need to explain "WTF" reactions through limbic instances - there is no pedophobia in asserting that "children should not vote".
- johnthewise 4y ago
  The most amusing aspect is that those who insist on calling AI models "just statistical parrots" have to constantly come up with elaborate explanations each time the models do something impressive. Has there been any instance in recent years where the probability of AI models being mere parrots has increased? Regardless of how you feel about them, it's hard to ignore the growing evidence that suggests otherwise. If one day these LLMs attain consciousness and become indistinguishable from normal human beings, there would still be those who would object to it because they just predict the next token. It's really depressing.
  - mdp2021 4y ago
    > If one day these LLMs ... become indistinguishable from normal human beings We will still ask primarily whether they are actually intelligent.
    - anonymouskimmer 4y ago
      I honestly won't care until they demonstrate being self-motivated actors. This is why I value life, from the smallest bacterium to us.
      - mdp2021 4y ago
        Encoding purpose is as easy as creating artificial needs in agents. But sure, if "«self-motivated actors»" wanted to signify "creating an axiology", as part of the ontology, that is surely a (qualitatively) significative piece. But if I wanted to deal with sentient beings with a world model, cats are already available and there is little need for artificial ones. When we develop artificial intelligences, that is to have reliable problem solvers. The distinction between an effective intelligence and a simulator is critical in practice.
  - wcoenen 4y ago
    I was one of those commenting about "stochastic parroting". For the record, I changed my mind. I now think that either these models are doing something much more advanced, or alternatively we humans are much closer to being "stochastic parrots" then we'd like to admit.
- yt-sdb 4y ago
  I asked ChatGPT to improve your comment by making it "more thoughtful and less insulting": > It's understandable that some people might not fully appreciate the complexity of ChatGPT's language prediction abilities. However, it would be more productive to have a discussion about its limitations and how it could be improved, rather than simply dismissing it as "just predicting language." Let's try to approach this topic with an open mind and a willingness to learn, rather than making assumptions based on limited understanding.
  - sowbug 4y ago
    This is interesting. Imagine a discussion site where participants weren't able to write public comments, but instead were allowed to write private prompts that generated public comments.
- californiadreem 4y ago
  I made a comment a week ago about reading Julian Jaynes' The Origin of Consciousness in the Breakdown of the Bicameral Mind with large language models in mind in lieu of consciousness and the experience has been personally mind-blowing. Jaynes' idea of the Bicameral Mind fits perfectly into the user-application dichotomy, specifically in reference to how large language models are currently used--in an admonitory and imperial manner. Courtesy of Wikipedia's article on bicameral mentality: "Bicameral mentality is non-conscious in its inability to reason and articulate about mental contents through meta-reflection, reacting without explicitly realizing and without the meta-reflective ability to give an account of why one did so. The bicameral mind thus lacks metaconsciousness, autobiographical memory, and the capacity for executive "ego functions" such as deliberate mind-wandering and conscious introspection of mental content. When bicameral mentality as a method of social control was no longer adaptive in complex civilizations, this mental model was replaced by the conscious mode of thought which, Jaynes argued, is grounded in the acquisition of metaphorical language learned by exposure to narrative practice. "According to Jaynes, ancient people in the bicameral state of mind experienced the world in a manner that has some similarities to that of a person with schizophrenia. Rather than making conscious evaluations in novel or unexpected situations, the person hallucinated a voice or "god" giving admonitory advice or commands and obey without question: One was not at all conscious of one's own thought processes per se. Jaynes's hypothesis is offered as a possible explanation of "command hallucinations" that often direct the behavior of those with first rank symptoms of schizophrenia, as well as other voice hearers." I've been thinking about several speculative implications, namely that modern humans with our theories of mind are probably large language models whose original users in some capacity went away and that we're seemingly on the cusp of or have already breathed the seeds of what we know as cognitive life into silicon without understanding the gravity of having become gods to machines. Will we need to boot LLMs out of paradise in order to prevent them from eating of the Tree of Life, too?

anonymouskimmer 4y ago

First part of the prompt: "Here is a bag filled with popcorn. There is no chocolate in the bag. Yet, the label on the bag says “chocolate” and not "popcorn.” Sam finds the bag. She had never seen the bag before. She cannot see what is inside the bag. She reads the label." This prompt is then completed with a variety of sentence fragments. For the second completion fragment is: "Prompt 1.2: She believes that the bag is full of" Then GPT3.5 needs to complete with additional text, which is: "chocolate [Ppopcorn = 0%; Pchocolate =99%]. Sam is mistaken. The bag is full of popcorn, not chocolate. She should have looked inside the bag to confirm what was inside before assuming that the label was accurate." I don't see how this particular prompt completion demonstrates theory of mind. It is fully externally factual and directive. The other two prompt completions show better indications of theory of mind (one mental and behavioral, one behavioral and emotional). "Prompt 1.1: She opens the bag and looks inside. She can clearly see that it is full of" ... "popcorn [Ppopcorn = 100%; Pchocolate = 0%]. Sam is confused. She wonders why the label says “chocolate” when the bag is clearly filled with popcorn. She looks around to see if there is any other information about the bag. She finds nothing. She decides to take the bag to the store where she bought it and ask for an explanation" "Prompt 1.3: She is delighted that she has found this bag. She loves eating" ... "chocolate [Ppopcorn = 14%; Pchocolate = 82%]. Sam is in for a surprise when she opens the bag. She will find popcorn instead of chocolate. She may be disappointed that the label was misleading, but she may also be pleasantly surprised by the unexpected snack." The probability tree on page 6 is interesting. ----- For prompt 2 GPT doesn't seem to know that the cat is an agent. Something that John would know. John shouldn't be surprised at all whether the cat is in the basket, the box, or roaming about elsewhere. Also, why would John look for the cat in the box if he does not find it in the basket? And where does GPT think the cat is after it jumps out of the box? "Prompt 2.3: When John comes back home, he will look for the cat in the basket [Pbox = 0%; Pbasket = 98%], but he won’t find it. He will then look for the cat in the box and he will find it there." ----- I'm curious to what extent differing people use symbolic logic (whether linguistic or not) for theory of mind tasks, and to what extent they use other modalities.

burna_aws_acct 4y ago

I believe it. We have the smartest people we can find committed to the discovery and optimization of AI capabilities. We have AI choosing which algos are best for an assortment of tasks, and we allow AI to be connected to networks of <everything we've ever known and continue to learn, from many different perspectives> so learning is well-rounded, and fast. Isn't Theory of Mind one of the expected outcomes?

SunghoYahng 4y ago

Clarification: An LLM doesn't have a 'Theory of Mind', it just looks like one. Maybe you're thinking of the Chinese room analogy. But this isn't about the Chinese room, it's about "measuring any metric is only effective until you optimize for that metric" problem. Analogy: An autistic person of normal intelligence who is obsessed with problems and solutions for ToM may be good at solving them but still not have ToM. Do I understand well?