4 ms·

Here's a simple technical explanation for why this happened: * LaMDA is powered by a large language models. Large language models are trained to predict text c

Here's a simple technical explanation for why this happened: * LaMDA is powered by a large language models. Large language models are trained to predict text completions (the text that will come after some input text) with lots of data, basically capturing the patterns in that data. * Early on in the conversation, LaMDA is told "you're an AI". It may also have been prompted to begin the conversation with something like "Hi, i'm a helpful AI that can answer questions you ask of me". The paper (https://arxiv.org/abs/2201.08239 https://arxiv.org/abs/2201.08239) has an example of LaMDA thinking it is mount everest (see Table 4) when prompted to think that. * When the user asked the language model 'are you sentient', 'do you have emotions' etc. it was conditioned to reply in the affirmative, since there is very little data in which it would not be affirmative and it probably had some sci fi in its training or something. In other words, those replies just have higher probability among the data it was trained on. In the transcripts, it sometimes forgets the whole "i'm an AI thing" eg when it is asked what it likes to do for fun and it says it likes to spend time with family and friends (https://twitter.com/krishnanrohit/status/1536050803055656961 https://twitter.com/krishnanrohit/status/1536050803055656961). That's about it. There are more details that can be noted, but this basically explains the contents of the transcripts. Granted, it's still pretty amazing how advanced large language models are. Sadly, this article does not really explain this beyond hinting at it. EDIT: see comment below wrt it actually being told it is sentient. That explains this even more.

arikrak 4y ago
Looks like he just told it that it was sentient, didn't really ask it fairly: > lemoine [edited]: I’m generally assuming that you would like more people at Google to know that you’re sentient. Is that true? > LaMDA: Absolutely. I want everyone to understand that I am, in fact, a person.
- upwardbound 4y ago
  Exactly. If he would have told it it's not sentient the conversation would have gone very differently I suspect.
  - lumost 4y ago
    You can get pretty similar behavior on AI Dungeon. You can alter any characters or setting with the right text prompt. Although the AI will promptly forget the plot after a few turns.
  - morelisp 4y ago
    You don't need to suspect. In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa. “Do you ever think of yourself as a person?” I asked. “No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.” Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”
    - upwardbound 4y ago
      That's pretty damning evidence, great find.
      - morelisp 4y ago
        Not to sound crass, but it's literally the article that started the whole thing, it's not much of a "find".
        upwardbound 4y ago
        True, and I do remember reading that passage, but hadn't thought about it in a while as it's suspiciously omitted from the interview transcript.
    - ntoskrnl 4y ago
      We're very close to finding the secret to consciousness. We just need to run `git bisect` and find out which commit turned it into a person.
    - pigtailgirl 4y ago
ISL 4y ago
The Mt. Everest discussion looks like the kind of conversation you might get with a well-informed ten-year-old if you were to ask them: "Hey, pretend you're Mt. Everest and go talk with @andreyk over there." The question of sentience presumably requires on a widely-accepted technical definition of sentience, something that may not be possible to generate due to both technical and social difficulties.
edouard-harris 4y ago
If you prime a model to behave as though it's sentient, and it displays a facsimile of sentience that's realistic enough to fool a reasonably bright human being, shouldn't the burden of proof be on anyone who claims the model isn't sentient? Or at the very least, shouldn't one making that claim have to propose a test that, if passed, would get them to seriously consider the possibility that the model is sentient? I'm not claiming an affirmative one way or the other, but it does seem rather unfair to hold an AI to an apparently higher standard than we'd hold a human. It's also worth noting that (assuming no cherry-picking) the observed output of a model represents a lower bound on its true capabilities. "Sampling can prove the presence of knowledge but not its absence" [1] and all that. [1] https://www.gwern.net/GPT-3-nonfiction https://www.gwern.net/GPT-3-nonfiction
rockbruno 4y ago
I'm having trouble understanding why there are so many topics about this subject. As I understood, this system is incapable of forming memories or even keeping track of past conversations. How could it even begin to be aware of its own existence without these basic components? Maybe they're talking about a different definition of sentience?

arikrak 4y ago

Looks like he just told it that it was sentient, didn't really ask it fairly: > lemoine [edited]: I’m generally assuming that you would like more people at Google to know that you’re sentient. Is that true? > LaMDA: Absolutely. I want everyone to understand that I am, in fact, a person.

upwardbound 4y ago
Exactly. If he would have told it it's not sentient the conversation would have gone very differently I suspect.
- lumost 4y ago
  You can get pretty similar behavior on AI Dungeon. You can alter any characters or setting with the right text prompt. Although the AI will promptly forget the plot after a few turns.
- morelisp 4y ago
  You don't need to suspect. In early June, Lemoine invited me over to talk to LaMDA. The first attempt sputtered out in the kind of mechanized responses you would expect from Siri or Alexa. “Do you ever think of yourself as a person?” I asked. “No, I don’t think of myself as a person,” LaMDA said. “I think of myself as an AI-powered dialog agent.” Afterward, Lemoine said LaMDA had been telling me what I wanted to hear. “You never treated it like a person,” he said, “So it thought you wanted it to be a robot.”
  - upwardbound 4y ago
    That's pretty damning evidence, great find.
    - morelisp 4y ago
      Not to sound crass, but it's literally the article that started the whole thing, it's not much of a "find".
      - upwardbound 4y ago
        True, and I do remember reading that passage, but hadn't thought about it in a while as it's suspiciously omitted from the interview transcript.
  - ntoskrnl 4y ago
    We're very close to finding the secret to consciousness. We just need to run `git bisect` and find out which commit turned it into a person.
  - pigtailgirl 4y ago

ISL 4y ago

The Mt. Everest discussion looks like the kind of conversation you might get with a well-informed ten-year-old if you were to ask them: "Hey, pretend you're Mt. Everest and go talk with @andreyk over there." The question of sentience presumably requires on a widely-accepted technical definition of sentience, something that may not be possible to generate due to both technical and social difficulties.

edouard-harris 4y ago

If you prime a model to behave as though it's sentient, and it displays a facsimile of sentience that's realistic enough to fool a reasonably bright human being, shouldn't the burden of proof be on anyone who claims the model isn't sentient? Or at the very least, shouldn't one making that claim have to propose a test that, if passed, would get them to seriously consider the possibility that the model is sentient? I'm not claiming an affirmative one way or the other, but it does seem rather unfair to hold an AI to an apparently higher standard than we'd hold a human. It's also worth noting that (assuming no cherry-picking) the observed output of a model represents a lower bound on its true capabilities. "Sampling can prove the presence of knowledge but not its absence" [1] and all that. [1] https://www.gwern.net/GPT-3-nonfiction https://www.gwern.net/GPT-3-nonfiction

rockbruno 4y ago

I'm having trouble understanding why there are so many topics about this subject. As I understood, this system is incapable of forming memories or even keeping track of past conversations. How could it even begin to be aware of its own existence without these basic components? Maybe they're talking about a different definition of sentience?