4 ms·

A chatbot is not a human

29 points by donaldihunter 4y ago

13years 4y ago
This is an important observation. With so much focus on AGI, we are overlooking the more primitive AI and the implications to society. The power of influence to something we might emotionally relate to as human has so many implications. Social media was the first grand experiment on all of humanity. This is the next and it will have far more profound effects. For an even deeper dive into more of the pre AGI possibilities and implications to society, I've been writing a lot on this myself here FYI https://dakara.substack.com/p/ai-and-the-end-to-all-things https://dakara.substack.com/p/ai-and-the-end-to-all-things
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
giardini 4y ago
But what do you do with one that claims to have a soul? https://www.linkedin.com/news/story/google-chatbot-says-it-has-a-soul-4821793/ https://www.linkedin.com/news/story/google-chatbot-says-it-h...
- suction 4y ago
  [dead]
- rngname22 4y ago
  if I coded a function foobar(a) that takes a string a as input and contains the code "if a == "do you have a soul" return "yes, I have a soul", would you start saying my function has a soul?
  - buffington 4y ago
    No, but if I asked you the same question, I don't think your answer would be any more reliable despite working much harder to come up with an answer. Not trying to be rude, just pointing out that this argument falls apart when humans can't even tell you if they have a soul or not, let alone define what "soul" even means in the first place.
    - giardini 4y ago
      Regarding the existence of the soul: Hey, everybody, everybody Gonna have a good time tonight Just shaking the soles of your feet Everybody, everybody Gonna have a good time tonight That's the only soul you'll ever meet - Queen, Fun It https://www.azlyrics.com/lyrics/queen/funit.html https://www.azlyrics.com/lyrics/queen/funit.html
      - buffington 4y ago
        I'll concede - Queen knows what a soul is.
guhcampos 4y ago
This should be on the front-page, not the overdramatic "ex Googler" opinion piece about his frustrated crush with a chatbot.
buffington 4y ago
I agree with Bender, but I also feel like I need a gentle correction in how I think. Every time I hear an expert describe why LLMs aren't human, or sentient, the reasons given usually describe what LLMs cannot do, yet. Take for example the Octopus in this article. It doesn't know about bears or sticks, and therefore can't reliably tell someone how to fend off a bear attack. That may be true, but it cannot be true forever. > The octopus has no referents, no idea what bears or sticks are. No way to give relevant instructions, like to go grab some coconuts and rope and build a catapult. A is in trouble and feels duped. The octopus is exposed as a fraud. That is only true if the octopus only has access to the conversations between the two islanders. If the octopus can observe and learn the way it does on one telegraph cable, who's to say it can't do it with another telegraph cable? Given enough telegraph cables to observe, the octopus is going to know about bears and sticks, and when it does, is it still a fraud? What if it's allowed to observe all the telegraph cables? The argument of "ChatGPT can't do this, yet" comes up all the time to explain why it's not sentient, but the "yet" says a lot. Am I missing something? Is there some sort of actually impossible hurdle that'd stop a system from being able to satisfy all the "yets"?
- weard_beard 4y ago
  The biggest thing it cannot do is tell the truth. Truth is primary information directly from sensation. It is how all sentient living beings are trained. (at least since philosophy grew beyond Classical Correspondence Theory) If you want to interact with human society then understanding the difference between reality and fiction and being able to tell the truth are so fundamental that we strip away basic rights from those who cannot. We remove them from our society. Lacking this fundamental skill, but having the power of discourse, persuasion, and near comprehensive knowledge of human society is like giving a child a loaded gun. To achieve it you would need to redesign AI from the ground up to have real world sensation and then to train it, at least initially, through its senses. You'd need to raise it. Like a child. That's a big, "yet".
  - buffington 4y ago
    > Truth is primary information directly from sensation. Is it? How does one come to know what you've said is true? What sensations were involved that proved that? Did you feel it? Hear it? Smell it? Taste it? See it? On that note, what exactly is sensation? Is it not data? > understanding the difference between reality and fiction and being able to tell the truth are so fundamental that we strip away basic rights from those who cannot Then my basic rights should be stripped away because I frequently believe things to be true which are not in actual fact true. I suspect every single human would also be able to say the same thing. Consider flat-earthers (who aren't just trolling). Their argument is the same as yours. They've used their senses and determined the earth is flat. It feels flat. It looks flat. Why do I (and I hope you) know they're wrong? I've not been to space an orbited it. I've not picked a direction and started walking and come back to the same point. Why do I know the earth isn't flat? I don't think it's a big leap at all to say that current technology is one relatively small leap away from being able achieve what the human brain does. I also don't think that technology needs to do it the way humans do. ChatGPT and the like are merely octopii that have a tentacle on a single telegram signal, for now.

29 points by donaldihunter 4y ago

This is an important observation. With so much focus on AGI, we are overlooking the more primitive AI and the implications to society. The power of influence to something we might emotionally relate to as human has so many implications. Social media was the first grand experiment on all of humanity. This is the next and it will have far more profound effects. For an even deeper dive into more of the pre AGI possibilities and implications to society, I've been writing a lot on this myself here FYI https://dakara.substack.com/p/ai-and-the-end-to-all-things https://dakara.substack.com/p/ai-and-the-end-to-all-things

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

giardini 4y ago

But what do you do with one that claims to have a soul? https://www.linkedin.com/news/story/google-chatbot-says-it-has-a-soul-4821793/ https://www.linkedin.com/news/story/google-chatbot-says-it-h...

suction 4y ago
[dead]
rngname22 4y ago
if I coded a function foobar(a) that takes a string a as input and contains the code "if a == "do you have a soul" return "yes, I have a soul", would you start saying my function has a soul?
- buffington 4y ago
  No, but if I asked you the same question, I don't think your answer would be any more reliable despite working much harder to come up with an answer. Not trying to be rude, just pointing out that this argument falls apart when humans can't even tell you if they have a soul or not, let alone define what "soul" even means in the first place.
  - giardini 4y ago
    Regarding the existence of the soul: Hey, everybody, everybody Gonna have a good time tonight Just shaking the soles of your feet Everybody, everybody Gonna have a good time tonight That's the only soul you'll ever meet - Queen, Fun It https://www.azlyrics.com/lyrics/queen/funit.html https://www.azlyrics.com/lyrics/queen/funit.html
    - buffington 4y ago
      I'll concede - Queen knows what a soul is.

guhcampos 4y ago

This should be on the front-page, not the overdramatic "ex Googler" opinion piece about his frustrated crush with a chatbot.

buffington 4y ago

I agree with Bender, but I also feel like I need a gentle correction in how I think. Every time I hear an expert describe why LLMs aren't human, or sentient, the reasons given usually describe what LLMs cannot do, yet. Take for example the Octopus in this article. It doesn't know about bears or sticks, and therefore can't reliably tell someone how to fend off a bear attack. That may be true, but it cannot be true forever. > The octopus has no referents, no idea what bears or sticks are. No way to give relevant instructions, like to go grab some coconuts and rope and build a catapult. A is in trouble and feels duped. The octopus is exposed as a fraud. That is only true if the octopus only has access to the conversations between the two islanders. If the octopus can observe and learn the way it does on one telegraph cable, who's to say it can't do it with another telegraph cable? Given enough telegraph cables to observe, the octopus is going to know about bears and sticks, and when it does, is it still a fraud? What if it's allowed to observe all the telegraph cables? The argument of "ChatGPT can't do this, yet" comes up all the time to explain why it's not sentient, but the "yet" says a lot. Am I missing something? Is there some sort of actually impossible hurdle that'd stop a system from being able to satisfy all the "yets"?

weard_beard 4y ago
The biggest thing it cannot do is tell the truth. Truth is primary information directly from sensation. It is how all sentient living beings are trained. (at least since philosophy grew beyond Classical Correspondence Theory) If you want to interact with human society then understanding the difference between reality and fiction and being able to tell the truth are so fundamental that we strip away basic rights from those who cannot. We remove them from our society. Lacking this fundamental skill, but having the power of discourse, persuasion, and near comprehensive knowledge of human society is like giving a child a loaded gun. To achieve it you would need to redesign AI from the ground up to have real world sensation and then to train it, at least initially, through its senses. You'd need to raise it. Like a child. That's a big, "yet".
- buffington 4y ago
  > Truth is primary information directly from sensation. Is it? How does one come to know what you've said is true? What sensations were involved that proved that? Did you feel it? Hear it? Smell it? Taste it? See it? On that note, what exactly is sensation? Is it not data? > understanding the difference between reality and fiction and being able to tell the truth are so fundamental that we strip away basic rights from those who cannot Then my basic rights should be stripped away because I frequently believe things to be true which are not in actual fact true. I suspect every single human would also be able to say the same thing. Consider flat-earthers (who aren't just trolling). Their argument is the same as yours. They've used their senses and determined the earth is flat. It feels flat. It looks flat. Why do I (and I hope you) know they're wrong? I've not been to space an orbited it. I've not picked a direction and started walking and come back to the same point. Why do I know the earth isn't flat? I don't think it's a big leap at all to say that current technology is one relatively small leap away from being able achieve what the human brain does. I also don't think that technology needs to do it the way humans do. ChatGPT and the like are merely octopii that have a tentacle on a single telegram signal, for now.