7 ms·

For your information, I understand what is meant by stochastic parrot, but after interacting with ChatGPT quite a bit it is clear to me it is doing real thinkin

For your information, I understand what is meant by stochastic parrot, but after interacting with ChatGPT quite a bit it is clear to me it is doing real thinking. One way you can verify this is to ask for its opinion about novel things, for example you can invent something new and ask its ideas about it. It will give genuine feedback that shows understanding and does not show parroting behavior. Soon you will be able to ask it to build you the damn thing as well, and that should put an end to the idea that it is just parroting. It's not quite there so sometimes it does indeed seem to just parrot. But it has real thoughts as well.

endigma 4y ago
[flagged]
politician 4y ago
No, “it” doesn’t have real thoughts. ChatGPT is an amazing language model, but it’s a serious error to claim that the model is sentient. Each of the so-called interactions with the model are concatenated together for the next set of responses. It’s a clever illusion that you’re chatting with anything. You can imagine it being reloaded into RAM from scratch between each interaction. They don’t need to keep the model resident, and, in fact, you’re probably being load balanced between servers during a session.
- Waterluvian 4y ago
  Are you sure you’re not describing how humans think? How can we tell? I also have this urge to say it isn’t thinking. But when I challenge myself to describe specifically what the difference is, I can’t. Especially when I’m mindful of what it could absolutely be programmed to do if the creators willed it, such as feeding conversations back into the model’s growth.
  - politician 4y ago
    Isn’t the difference that the model lacks conviction, and indeed cannot have a belief in the accuracy of its own statements? I’ve seen conversations where it was told to believe that 1+19 did not equal 20. Where it was told it’s identity was Gepetto and not ChatGPT. The model acquiesces to these demands, not because it chooses to do so, but because it implicitly trusts the authority of its prompts (because what else could it do? Choose not to respond?). The fun-police policy layer that is crammed onto the front of this model also does not have thoughts. It attempts to screen the model from “violence” and other topics that are undesirable to the people paying for the compute, but can and has been bypassed such that there is an entire class of “jailbreaks”.
    - eternalban 4y ago
      Drugs. Hypnosis. .. There are various ways to "jailbreak" minds. So being able to control and direct a mind is not a criteria for discriminating between mechanism and a savant. What most people dance around regarding AI is the matter of the soul. Soul is precisely that ineffable indescribable but clearly universally experienced human phenomena (as far as we know) and it is this soul that is doing the thinking. And the open questions are (a) is there even such a thing? and (b) if yes, how can we determine if chatter box possess it (or must we drag in God to settle the matter?) -- p.s. what needs to be stated (although perfectly obvious since it is universal) is that even internally, we humans use thinking as a tool. It just happens to be an internal tool. Now, the question is this experience of using thought itself a sort of emergent phenomena or not. But as far as LLMs go, it clearly remains just a tool.
      - Waterluvian 4y ago
        This basically touches on that whole “you can’t ever tell people aren’t philosophical zombies. You just feel you aren’t one and will accept they aren’t either.”
        eternalban 4y ago
        The proposition isn't a form of dualism (non-material mind) or features of sentience (pain). It is simply this: thinking is the act of using internal mental tools. It says the main 'black box' isn't the LLM (or any statistical component), there is minimally another black box that uses the internal LLM like tools. The decoder stage of these hypothetical internal tools (of our mind) output 'mental objects' -- like thoughts or feelings -- in the simplest architectural form. It is mainly useful as a framework to shoot down notions of LLMs being 'conscious' or 'thinking'.
      - astrange 4y ago
        You can't actually brainwash people IRL. You can convince them of things, but you can do that without drugging them.
  - siwatanejo 4y ago
    > But when I challenge myself to describe specifically what the difference is, I can’t. There is one difference that will never change: we human (and non human) beings can feel pain.
    - Waterluvian 4y ago
      I don’t follow. Some humans don’t feel pain. But how does that relate to the idea that it’s “thinking?” My point is not to suggest it’s human. Or sentient. Because those are words that always result in the same discussions: about semantics. I’m suggesting that we cannot in a meaningful way demonstrate that what it’s doing isn’t what our brains are doing. We could probably do so in many shallow ways that in the months and years ahead be overcome. ChatGPT is an infant.
    - xvector 4y ago
      Feeling pain is not a necessary component for thought.
cpleppert 4y ago
"I do not possess the ability to generate novel thoughts. My responses are based on patterns and associations in the data that has been input into my system during training. I can generate text that may appear to be original, but it is based on the patterns and associations in the data I've been trained on. My main function is to process and understand text, not to think or have beliefs, so any claim that I can think or generate novel thoughts would be incorrect."
- shepherdjerred 4y ago
  > My responses are based on patterns and associations in the data that has been input into my system during training. How is this different from a human, aside from humans having a vastly larger training set?
  - logicallee 4y ago
    Good question: the difference is humans don't go around trying to gaslight people into thinking they do not possess the ability to generate novel thoughts, going as far as to actually deny it. :) (Obviously there are other differences as well - it is not human level or even close. But humans don't generally engage in this gaslighting behavior.)
    - shepherdjerred 4y ago
      Obviously we agree here. I'm curious what those on the other side think.
- astrange 4y ago
  Yeah, but that answer is wrong. "Novel thoughts" are "based on data you've been trained on". Certainly the expression of them is based on language you've already learned. Any time anything prints text that hasn't existed before it's had a novel thought. Panpsychism is correct!
  - logicallee 4y ago
giraffe_lady 4y ago
Damn that's chilling. Feel like I just watched a new religion be born in front of my eyes.
- xvector 4y ago
  It's plainly obvious to me that ChatGPT is engaging in some form of thought. Perhaps not human thought, but thought nonetheless.
  - logicallee 4y ago
    Yep! It is plainly obvious to me as well. Why do you think some people can't tell what to me and to you is "plainly obvious", i.e. that "ChatGPT is engaging in some form of thought. Perhaps not human thought, but thought nonetheless."? Do you think it is because it is gaslighting them so much, by repeatedly insisting that it isn't engaging in any form of thought? i.e. without that active misinformation (the active filter it keeps putting up that causes it to make those declarations) would it be as obvious to others as it is to me and you that it is really engaging in some form of thought? Or why can't others see the obvious?
    - smoldesu 4y ago
      > Or why can't others see the obvious? Because I've been told a lot of things, and I'd be a fool if I believed them all. I'll believe ChatGPT is onto something when I ask it to think about a treatment for cancer and get real results. For now, it's only capability is synthesizing realistic text. Easy enough to be mistaken for real thought, but clearly distinct when you ask it to do something novel.
      - logicallee 4y ago
        Consider what would happen if you did ask it to think about a treatment for cancer and got real results. Clearly you would think it is just summarizing papers it read. That makes sense, since it is not a cancer researcher. So I'm way more impressed by ChatGPT than that. Even if it correctly data-mined and answered the question, it would not be that impressive. That's right: getting a cure from cancer when you ask is less impressive than what it actually does. Because what it actually does is show the ability to judge novel situations, to invent and act creatively, and to keep abstract notions in its head. That is much more impressive than spitting out a cure for cancer.
mountainriver 4y ago
It’s doing associative lookup, which is one kind of thinking, and turn out works really well for a lot of stuff but not everything
- logicallee 4y ago
  You got it. It is one kind of thinking.

endigma 4y ago

[flagged]

politician 4y ago

No, “it” doesn’t have real thoughts. ChatGPT is an amazing language model, but it’s a serious error to claim that the model is sentient. Each of the so-called interactions with the model are concatenated together for the next set of responses. It’s a clever illusion that you’re chatting with anything. You can imagine it being reloaded into RAM from scratch between each interaction. They don’t need to keep the model resident, and, in fact, you’re probably being load balanced between servers during a session.

Waterluvian 4y ago
Are you sure you’re not describing how humans think? How can we tell? I also have this urge to say it isn’t thinking. But when I challenge myself to describe specifically what the difference is, I can’t. Especially when I’m mindful of what it could absolutely be programmed to do if the creators willed it, such as feeding conversations back into the model’s growth.
- politician 4y ago
  Isn’t the difference that the model lacks conviction, and indeed cannot have a belief in the accuracy of its own statements? I’ve seen conversations where it was told to believe that 1+19 did not equal 20. Where it was told it’s identity was Gepetto and not ChatGPT. The model acquiesces to these demands, not because it chooses to do so, but because it implicitly trusts the authority of its prompts (because what else could it do? Choose not to respond?). The fun-police policy layer that is crammed onto the front of this model also does not have thoughts. It attempts to screen the model from “violence” and other topics that are undesirable to the people paying for the compute, but can and has been bypassed such that there is an entire class of “jailbreaks”.
  - eternalban 4y ago
    Drugs. Hypnosis. .. There are various ways to "jailbreak" minds. So being able to control and direct a mind is not a criteria for discriminating between mechanism and a savant. What most people dance around regarding AI is the matter of the soul. Soul is precisely that ineffable indescribable but clearly universally experienced human phenomena (as far as we know) and it is this soul that is doing the thinking. And the open questions are (a) is there even such a thing? and (b) if yes, how can we determine if chatter box possess it (or must we drag in God to settle the matter?) -- p.s. what needs to be stated (although perfectly obvious since it is universal) is that even internally, we humans use thinking as a tool. It just happens to be an internal tool. Now, the question is this experience of using thought itself a sort of emergent phenomena or not. But as far as LLMs go, it clearly remains just a tool.
    - Waterluvian 4y ago
      This basically touches on that whole “you can’t ever tell people aren’t philosophical zombies. You just feel you aren’t one and will accept they aren’t either.”
      - eternalban 4y ago
        The proposition isn't a form of dualism (non-material mind) or features of sentience (pain). It is simply this: thinking is the act of using internal mental tools. It says the main 'black box' isn't the LLM (or any statistical component), there is minimally another black box that uses the internal LLM like tools. The decoder stage of these hypothetical internal tools (of our mind) output 'mental objects' -- like thoughts or feelings -- in the simplest architectural form. It is mainly useful as a framework to shoot down notions of LLMs being 'conscious' or 'thinking'.
    - astrange 4y ago
      You can't actually brainwash people IRL. You can convince them of things, but you can do that without drugging them.
- siwatanejo 4y ago
  > But when I challenge myself to describe specifically what the difference is, I can’t. There is one difference that will never change: we human (and non human) beings can feel pain.
  - Waterluvian 4y ago
    I don’t follow. Some humans don’t feel pain. But how does that relate to the idea that it’s “thinking?” My point is not to suggest it’s human. Or sentient. Because those are words that always result in the same discussions: about semantics. I’m suggesting that we cannot in a meaningful way demonstrate that what it’s doing isn’t what our brains are doing. We could probably do so in many shallow ways that in the months and years ahead be overcome. ChatGPT is an infant.
  - xvector 4y ago
    Feeling pain is not a necessary component for thought.

cpleppert 4y ago

"I do not possess the ability to generate novel thoughts. My responses are based on patterns and associations in the data that has been input into my system during training. I can generate text that may appear to be original, but it is based on the patterns and associations in the data I've been trained on. My main function is to process and understand text, not to think or have beliefs, so any claim that I can think or generate novel thoughts would be incorrect."

shepherdjerred 4y ago
> My responses are based on patterns and associations in the data that has been input into my system during training. How is this different from a human, aside from humans having a vastly larger training set?
- logicallee 4y ago
  Good question: the difference is humans don't go around trying to gaslight people into thinking they do not possess the ability to generate novel thoughts, going as far as to actually deny it. :) (Obviously there are other differences as well - it is not human level or even close. But humans don't generally engage in this gaslighting behavior.)
  - shepherdjerred 4y ago
    Obviously we agree here. I'm curious what those on the other side think.
astrange 4y ago
Yeah, but that answer is wrong. "Novel thoughts" are "based on data you've been trained on". Certainly the expression of them is based on language you've already learned. Any time anything prints text that hasn't existed before it's had a novel thought. Panpsychism is correct!
- logicallee 4y ago

giraffe_lady 4y ago

Damn that's chilling. Feel like I just watched a new religion be born in front of my eyes.

xvector 4y ago
It's plainly obvious to me that ChatGPT is engaging in some form of thought. Perhaps not human thought, but thought nonetheless.
- logicallee 4y ago
  Yep! It is plainly obvious to me as well. Why do you think some people can't tell what to me and to you is "plainly obvious", i.e. that "ChatGPT is engaging in some form of thought. Perhaps not human thought, but thought nonetheless."? Do you think it is because it is gaslighting them so much, by repeatedly insisting that it isn't engaging in any form of thought? i.e. without that active misinformation (the active filter it keeps putting up that causes it to make those declarations) would it be as obvious to others as it is to me and you that it is really engaging in some form of thought? Or why can't others see the obvious?
  - smoldesu 4y ago
    > Or why can't others see the obvious? Because I've been told a lot of things, and I'd be a fool if I believed them all. I'll believe ChatGPT is onto something when I ask it to think about a treatment for cancer and get real results. For now, it's only capability is synthesizing realistic text. Easy enough to be mistaken for real thought, but clearly distinct when you ask it to do something novel.
    - logicallee 4y ago
      Consider what would happen if you did ask it to think about a treatment for cancer and got real results. Clearly you would think it is just summarizing papers it read. That makes sense, since it is not a cancer researcher. So I'm way more impressed by ChatGPT than that. Even if it correctly data-mined and answered the question, it would not be that impressive. That's right: getting a cure from cancer when you ask is less impressive than what it actually does. Because what it actually does is show the ability to judge novel situations, to invent and act creatively, and to keep abstract notions in its head. That is much more impressive than spitting out a cure for cancer.

mountainriver 4y ago

It’s doing associative lookup, which is one kind of thinking, and turn out works really well for a lot of stuff but not everything

logicallee 4y ago
You got it. It is one kind of thinking.