6 ms·

I did study philosophy, in particular philosophy of the mind, and all this talk excluding chatgpt from sentience is completely off the mark. They are making th

I did study philosophy, in particular philosophy of the mind, and all this talk excluding chatgpt from sentience is completely off the mark. They are making the same categorical mistake that Dr. Seattle makes in The Chinese Room experiment. Simply put, useful definitions of sentience can’t use mechanism for it to be a useful definition. First, a biologist can’t look at a human brain and tell if it is sentient, so it isn’t even useful to determine whether Fred my next door neighbor is sentient. Second, we don’t usually have the ability to take something apart before we have to make a sentience call on it. For example, if I declared that only humans with type-A blood are actually sentient (they rest being mere simulacra), you’d still need to treat everybody as if. Another way to put it is, if an alien probe of unknown construction arrives on Earth, how would we tell if it is sentient? The answer, of course, is behavior. Until there’s a crisp definition that puts human behavior in one bucket and chatgpt in another, we have to consider the possibility that both are sentient.

amf12 4y ago
> Another way to put it is, if an alien probe of unknown construction arrives on Earth, how would we tell if it is sentient? We might not be able to because of the "unknown construction". But we do know how LLMs are designed and how they operate, to understand that they are just mimicking language and aren't sentient. I feel one key aspect of "sentience" is the sensations we feel when we have emotions. LLM's definitely lack that even if we can't know if they "feel" something.
snowwrestler 4y ago
Until there’s a crisp definition that puts human behavior in one bucket and monkeys in another, we have to consider the possibility that both are sentient. Until there’s a crisp definition that puts human behavior in one bucket and dogs in another, we have to consider the possibility that both are sentient. Until there’s a crisp definition that puts human behavior in one bucket and trees in another, we have to consider the possibility that both are sentient. Until there’s a crisp definition that puts human behavior in one bucket and mountain ranges in another, we have to consider the possibility that both are sentient. This is fun. Do you see the flaw in the structure of the argument yet? It’s begging the question. The burden is not how to disprove that ChatGPT is sentient, the burden is whether it’s possible to prove that it is sentient. Otherwise it’s the rock that keeps away tigers, because do you see any tigers around here?
- Aeolun 4y ago
  > Do you see the flaw in the structure of the argument yet? Are you trying to claim that you know for certain mountains aren’t sentient? Just because we don’t think it’s worth the effort of proving mountains are sentient, is no guarantee they are not. For mountains that judgement is fairly easy to make, and it has worked out pretty well for us, but I do not think we can stretch that to every entity we want.
  - spywaregorilla 4y ago
    Mountains aren't sentient.
    - Aeolun 4y ago
      By what definition of sentient, and how do you know?
      - spywaregorilla 4y ago
        > By what definition of sentient I don't care. But part of that definition will be that mountains are not sentient. And if you come up with a definition that includes mountains as sentient things, then we're talking about different concepts and language is the issue, not the concept. > how do you know Because I'm not a dumbass
  - staticman2 4y ago
    "Are you trying to claim that you know for certain mountains aren’t sentient?" We don't know anything for sure. Philip k. Dick believed the modern world was an illusion created by Satan and it was actually the year 50 AD. But if I keep posting "How do you know computers aren't an illusion created by Satan?" anytime someone wants to discuss computer software I probably deserve to be laughed out of the conversation.
- stocknoob 4y ago
  Apes with this particular set of DNA are presumed to be fully sentient, and apes with a slightly different set are not, even though their behavior is indistinguishable until age 1 (or never, in some cases). Also, to be presumed sentient, you have to be made of meat, not metal. Look, I don’t make the rules, they are universal laws and just happen to affirm my sentience.
staticman2 4y ago
What if there's no such thing as ChatGPT and you are a butterfly dreaming it is a human discussing ChatGPT? Until there's a definitive answer to that question I'll keep spamming every ChatGPT thread with the demand people discuss butterflies. Carry on.
spywaregorilla 4y ago
People who "study philosophy" always do this dumb thing where they pounce on something for being poorly defined and then consider themselves wiser for arguing that the ambiguity of the words means that obvious non examples "could be described" and just double back to the original definition. Is a rock alive? Is my imagination of a chinese room "art"? Is anything real? > Until there’s a crisp definition that puts human behavior in one bucket and chatgpt in another, we have to consider the possibility that both are sentient. This is such a dumb sentence. Well we can't define it so we have to consider that both are described by it? No. If you can't define it you just can't use it in any meaningful way. By your same logic you would have to consider the possibility that you're a AFDUNOIAFNDMLOISABFID, because you can't define it. You can't presume that concepts have merit because they are represented by language.
- KyleBerezin 4y ago
  Sentience IS poorly defined though. It often just means the ability to perceive and feel, which most vertebrates can in a way we can understand the observe. To the point, though, you are arguing against the most basic concepts of reasoning. If we want to draw a line in the sand of what is sentient and what is not, it involves making a more detailed definition of the word itself. This has been a topic of discussion for a long time, but now the ability gap between the human mind and computer is narrowing, these questions are becoming more pressing.
  - spywaregorilla 4y ago
    > Sentience IS poorly defined though. The nature of anything that is poorly defined is inherently poorly defined. The only logical way to discuss a poorly defined concept is to acknowledge that it's not a singular concept and to identify the logic that you actually care about. You cannot draw a line in the sand without recognizing that the concept itself is fictitious and its up to you to draw the line in the way you choose. There is no right answer. There is merely consensus for the purpose of conversation. And if you have no consensus, then don't bother having the conversation. Because you're being stupid again. When people imagine this vague concept, and then start to give examples of things that are and are not aligned with that concept, and then start disagreeing because their visions of the vague concept differ... they're just being idiots. You're just trying to bottom-up back into a position that could have been defined as poorly defined from the beginning because you thought the word "sentience" or whatever gave it legitimacy as a concept. If I'm going to discuss sentience, my definition involves intelligence that is aware of itself, exists strictly in continuous time, and has motivations about the itself and its relationship to the world. I absolutely do not care about anyone else's definition. We use the word definition but that's a language problem. Your definition of sentience is not a competing definition with mine. it is a different thing in a shared name space because language isn't good enough. if you disagree with my definition, you disagree not with existence of my opinion, but with my namespace allocation of it. and that does not matter. maybe you have an interesting idea that I would like to hear and incorporate into my view. But it's still just a fuzzy name space for many things. not a singular concept that has one definition we've yet to arrive at. What is X == What ought to be represented by the word X in our crappy language? When you realize this, it becomes apparent that "what is X" is a very boring question and obvious why pretty much every philosophical debate on "What is X" ends up with some dumb appeal to non cognizance.
KyleBerezin 4y ago
I think part of the issue is the that we use sentience in a binary sense. Like there is some kind of abstract line you can pass that makes you sentient. I personally feel that it is more useful to look at it as a spectrum. A lot of factors play into it, but as a society we already do this. The idea that some animals are closer to human levels of sentients has caused some people to avoid consuming those animals, such as dolphins vs. fish, or dogs vs. chickens. No one has put numbers to it as far as I know, but I feel that people/society has drawn conclusions based on an unspoken ranking. If we were to make a numerical scale of sentience, I could imagine that perhaps humans wouldn't be a '10', either as a whole or in any one category. Whatever traits combine to make sentience, it is hard to believe that we embody the theoretical maximum of all of those traits.
xmonkee 4y ago
Dr Searle isn’t making a mistake, behaviorists are just unbelievably stubborn. Just because you cannot measure internal experience doesn’t mean we cant talk about it. That’s just being dishonest. I also used to be a behaviorist but I grew up and became comfortable with not being able to objectify everything. Its okay. There is a subjective dimension. In fact, its great.
- chpatrick 4y ago
  Who's to say one's subjective experience isn't another's latent space.
  - xmonkee 4y ago
    what does that mean?
    - chpatrick 4y ago
      I mean all the mysterious stuff that makes your experience special and subjective is also kicking around somewhere in the execution of say GPT.
      - xmonkee 4y ago
        The proof of my subjective experience is my subjective experience of it, not the fact that I can complete sentences. So saying chatgpt has consciousness because it can talk is a complete non-sequitor.
        chpatrick 4y ago
        The problem is that you're the only one who experiences it so you can't prove it to anyone else. How do you know ChatGPT doesn't experience anything?
        xmonkee 4y ago
shakna 4y ago
For starters, it is sapience, not sentience, that people are looking for (thanks Star Trek). Sentience is nothing more than the ability to perceive pain, something that some trees and fungi have.
guhcampos 4y ago
I threw in some keywords in my comment, not in vain, but very conscientiously. If sentience is hard to define, we can "take it apart" and try to reason about its individual components. I threw some of these possible components there to drive the discussion. Does GPT have Intention? I'm absolutely sure it does not. My dog definitely has. Does GPT has desires? I'm sure it doesn't. I'm conflicted about my dog. Is GPT capable of judgment? I'm sure it's not. I know my dog has some modest capacity for it. So, can GPT demonstrate Will? I don't think it can, and will ever be able to. So I'd never consider it Sentient, not even to the limited, partial, sense I could consider my dog to be. This discussion is important because when AI starts to "make decisions" it must be absolutely clear to everyone involved that these decisions are not rational, nor emotional, nor instinctive: they are simply reproductions of statistically probable outcomes based on its inputs. You can use an AI to help determine if the text of a law contradicts a pre-existing legislation, but you can't ever use an AI to determine if a law is fair, ethical, benefitial or detrimental. My biggest fear is not AIs going rogue, it's people like this "former Google employee" spreading misinformation and influencing important political decisions based on poor, misinformed judgment, as we've seen multiple times in the past with everything from GMOs, Nuclear Power, Climate Change, Vaccines and so forth.
hnfong 4y ago
The Chinese Room thought experiment is kind of a different mistake. In the Chinese room argument, Searle attempted to imagine an AI system that was "obviously not sentient" because it was made up of hard rules and facts (presumably legible to us humans), and it was argued that it should be obvious that the system wasn't sentient. For the ChatGPT case it's not clear the Chinese room argument even applies. First, it's not even a Chinese room, it's trained by a neural network, an a neural network is inspired by the structure of our brains. There's no "set of instructions in English" for anyone to follow, but it's a rather overwhelming set of neurons somehow producing the results that it was trained on, and as an outsider we have no idea what's going on inside individual neurons -- just like our brains. I'm seeing a lot of people claim that "we know how ChatGPT is created, and so it's not sentient", maybe this looks like the Chinese room fallacy (I'm calling it as such), but it's like claiming we know the laws of physics (Schrödinger's equation etc.) and then claiming we know how the brain works because it's all quantum physics. The fact is we know how to build GPT but we don't actually know how it works. Besides what happens to humanity when we learn more and more about how our own brains work? I'm pretty comfortable with the idea that we're not sentient (yay), but I'm pretty sure those same people would then come up with other nonsensical arguments to prove that they are different from the robots we created.
- larryfreeman 4y ago
  I believe that it is straight forward to show how Searle's Chinese Room argument can be applied to ChatGPT. The idea is not the "process" for building the rules but the "process" where the rules are applied. True sentience requires a "true" understanding of the information that is being presented (otherwise, it is not reflective of sentience but rather trickery). The Chinese Room applies because ChatGPT or any other computer is generating text purely through logic, that is, logical rules. To make this point clear, let's assume just for argument's sake that at some quantum level, the electrical activity underlying a computer is partially sentient (I am not making this claim -- just using it as an example). Even if this were true, this would not prove that the sentience is involved with the text being generated by the same computer. This underlying sentience would be like the "person" in the Chinese room. The text generated are the Chinese words which would be completely independent from the "sentience" following the rules. Generating text at sufficient complexity to fool the average person does not prove that "sentience" is occurring. Generating text consistently in a way that is only explainable through sentience is what is required. ChatGPT, because it is easily shown to be flawed, does not rise to this level of evidence. In the Chinese Room example, the complexity of what could be displayed by the rulebooks would have a limited complexity (though, the surprise of ChatGPT is that this complexity is greater than many of us would have supposed). Just like the story of Clever Hans the Horse (see https://en.wikipedia.org/wiki/Clever_Hans https://en.wikipedia.org/wiki/Clever_Hans), once the limitation of the process is realized, it becomes clear that there is not "true" understanding of what is generated which means that the rules, in themselves, do not reflect "true" sentience.
imtringued 4y ago
But behavior alone cannot be an indicator for sentience as you can simply record behavior and play it back. Sentience has more to do with sensory perception and feedback loops than mental capability or intelligence. If a future AI attains sentience on a commodity processor, that wouldn't imply that the program is what achieved sentience, rather the computer running the program and all computers before had sentience and the AI is just an interface between the computer and humans that lets it express its sentience in a way that can be understood by humans. If you really believed that behavior is what drives sentience, then non verbal people with autism would have very low sentience and if they somehow overcome their non verbalism, they would gain sentience throughout their life, which is at odds with the fact that many people believe that animals in factory farms are sentient for instance, despite their poor language skills and complete lack of effort needed to achieve sentience.

amf12 4y ago

> Another way to put it is, if an alien probe of unknown construction arrives on Earth, how would we tell if it is sentient? We might not be able to because of the "unknown construction". But we do know how LLMs are designed and how they operate, to understand that they are just mimicking language and aren't sentient. I feel one key aspect of "sentience" is the sensations we feel when we have emotions. LLM's definitely lack that even if we can't know if they "feel" something.

snowwrestler 4y ago

Until there’s a crisp definition that puts human behavior in one bucket and monkeys in another, we have to consider the possibility that both are sentient. Until there’s a crisp definition that puts human behavior in one bucket and dogs in another, we have to consider the possibility that both are sentient. Until there’s a crisp definition that puts human behavior in one bucket and trees in another, we have to consider the possibility that both are sentient. Until there’s a crisp definition that puts human behavior in one bucket and mountain ranges in another, we have to consider the possibility that both are sentient. This is fun. Do you see the flaw in the structure of the argument yet? It’s begging the question. The burden is not how to disprove that ChatGPT is sentient, the burden is whether it’s possible to prove that it is sentient. Otherwise it’s the rock that keeps away tigers, because do you see any tigers around here?

Aeolun 4y ago
> Do you see the flaw in the structure of the argument yet? Are you trying to claim that you know for certain mountains aren’t sentient? Just because we don’t think it’s worth the effort of proving mountains are sentient, is no guarantee they are not. For mountains that judgement is fairly easy to make, and it has worked out pretty well for us, but I do not think we can stretch that to every entity we want.
- spywaregorilla 4y ago
  Mountains aren't sentient.
  - Aeolun 4y ago
    By what definition of sentient, and how do you know?
    - spywaregorilla 4y ago
      > By what definition of sentient I don't care. But part of that definition will be that mountains are not sentient. And if you come up with a definition that includes mountains as sentient things, then we're talking about different concepts and language is the issue, not the concept. > how do you know Because I'm not a dumbass
- staticman2 4y ago
  "Are you trying to claim that you know for certain mountains aren’t sentient?" We don't know anything for sure. Philip k. Dick believed the modern world was an illusion created by Satan and it was actually the year 50 AD. But if I keep posting "How do you know computers aren't an illusion created by Satan?" anytime someone wants to discuss computer software I probably deserve to be laughed out of the conversation.
stocknoob 4y ago
Apes with this particular set of DNA are presumed to be fully sentient, and apes with a slightly different set are not, even though their behavior is indistinguishable until age 1 (or never, in some cases). Also, to be presumed sentient, you have to be made of meat, not metal. Look, I don’t make the rules, they are universal laws and just happen to affirm my sentience.

staticman2 4y ago

What if there's no such thing as ChatGPT and you are a butterfly dreaming it is a human discussing ChatGPT? Until there's a definitive answer to that question I'll keep spamming every ChatGPT thread with the demand people discuss butterflies. Carry on.

spywaregorilla 4y ago

People who "study philosophy" always do this dumb thing where they pounce on something for being poorly defined and then consider themselves wiser for arguing that the ambiguity of the words means that obvious non examples "could be described" and just double back to the original definition. Is a rock alive? Is my imagination of a chinese room "art"? Is anything real? > Until there’s a crisp definition that puts human behavior in one bucket and chatgpt in another, we have to consider the possibility that both are sentient. This is such a dumb sentence. Well we can't define it so we have to consider that both are described by it? No. If you can't define it you just can't use it in any meaningful way. By your same logic you would have to consider the possibility that you're a AFDUNOIAFNDMLOISABFID, because you can't define it. You can't presume that concepts have merit because they are represented by language.

KyleBerezin 4y ago
Sentience IS poorly defined though. It often just means the ability to perceive and feel, which most vertebrates can in a way we can understand the observe. To the point, though, you are arguing against the most basic concepts of reasoning. If we want to draw a line in the sand of what is sentient and what is not, it involves making a more detailed definition of the word itself. This has been a topic of discussion for a long time, but now the ability gap between the human mind and computer is narrowing, these questions are becoming more pressing.
- spywaregorilla 4y ago
  > Sentience IS poorly defined though. The nature of anything that is poorly defined is inherently poorly defined. The only logical way to discuss a poorly defined concept is to acknowledge that it's not a singular concept and to identify the logic that you actually care about. You cannot draw a line in the sand without recognizing that the concept itself is fictitious and its up to you to draw the line in the way you choose. There is no right answer. There is merely consensus for the purpose of conversation. And if you have no consensus, then don't bother having the conversation. Because you're being stupid again. When people imagine this vague concept, and then start to give examples of things that are and are not aligned with that concept, and then start disagreeing because their visions of the vague concept differ... they're just being idiots. You're just trying to bottom-up back into a position that could have been defined as poorly defined from the beginning because you thought the word "sentience" or whatever gave it legitimacy as a concept. If I'm going to discuss sentience, my definition involves intelligence that is aware of itself, exists strictly in continuous time, and has motivations about the itself and its relationship to the world. I absolutely do not care about anyone else's definition. We use the word definition but that's a language problem. Your definition of sentience is not a competing definition with mine. it is a different thing in a shared name space because language isn't good enough. if you disagree with my definition, you disagree not with existence of my opinion, but with my namespace allocation of it. and that does not matter. maybe you have an interesting idea that I would like to hear and incorporate into my view. But it's still just a fuzzy name space for many things. not a singular concept that has one definition we've yet to arrive at. What is X == What ought to be represented by the word X in our crappy language? When you realize this, it becomes apparent that "what is X" is a very boring question and obvious why pretty much every philosophical debate on "What is X" ends up with some dumb appeal to non cognizance.

KyleBerezin 4y ago

I think part of the issue is the that we use sentience in a binary sense. Like there is some kind of abstract line you can pass that makes you sentient. I personally feel that it is more useful to look at it as a spectrum. A lot of factors play into it, but as a society we already do this. The idea that some animals are closer to human levels of sentients has caused some people to avoid consuming those animals, such as dolphins vs. fish, or dogs vs. chickens. No one has put numbers to it as far as I know, but I feel that people/society has drawn conclusions based on an unspoken ranking. If we were to make a numerical scale of sentience, I could imagine that perhaps humans wouldn't be a '10', either as a whole or in any one category. Whatever traits combine to make sentience, it is hard to believe that we embody the theoretical maximum of all of those traits.

xmonkee 4y ago

Dr Searle isn’t making a mistake, behaviorists are just unbelievably stubborn. Just because you cannot measure internal experience doesn’t mean we cant talk about it. That’s just being dishonest. I also used to be a behaviorist but I grew up and became comfortable with not being able to objectify everything. Its okay. There is a subjective dimension. In fact, its great.

chpatrick 4y ago
Who's to say one's subjective experience isn't another's latent space.
- xmonkee 4y ago
  what does that mean?
  - chpatrick 4y ago
    I mean all the mysterious stuff that makes your experience special and subjective is also kicking around somewhere in the execution of say GPT.
    - xmonkee 4y ago
      The proof of my subjective experience is my subjective experience of it, not the fact that I can complete sentences. So saying chatgpt has consciousness because it can talk is a complete non-sequitor.
      - chpatrick 4y ago
        The problem is that you're the only one who experiences it so you can't prove it to anyone else. How do you know ChatGPT doesn't experience anything?
        xmonkee 4y ago

shakna 4y ago

For starters, it is sapience, not sentience, that people are looking for (thanks Star Trek). Sentience is nothing more than the ability to perceive pain, something that some trees and fungi have.

guhcampos 4y ago

I threw in some keywords in my comment, not in vain, but very conscientiously. If sentience is hard to define, we can "take it apart" and try to reason about its individual components. I threw some of these possible components there to drive the discussion. Does GPT have Intention? I'm absolutely sure it does not. My dog definitely has. Does GPT has desires? I'm sure it doesn't. I'm conflicted about my dog. Is GPT capable of judgment? I'm sure it's not. I know my dog has some modest capacity for it. So, can GPT demonstrate Will? I don't think it can, and will ever be able to. So I'd never consider it Sentient, not even to the limited, partial, sense I could consider my dog to be. This discussion is important because when AI starts to "make decisions" it must be absolutely clear to everyone involved that these decisions are not rational, nor emotional, nor instinctive: they are simply reproductions of statistically probable outcomes based on its inputs. You can use an AI to help determine if the text of a law contradicts a pre-existing legislation, but you can't ever use an AI to determine if a law is fair, ethical, benefitial or detrimental. My biggest fear is not AIs going rogue, it's people like this "former Google employee" spreading misinformation and influencing important political decisions based on poor, misinformed judgment, as we've seen multiple times in the past with everything from GMOs, Nuclear Power, Climate Change, Vaccines and so forth.

hnfong 4y ago

The Chinese Room thought experiment is kind of a different mistake. In the Chinese room argument, Searle attempted to imagine an AI system that was "obviously not sentient" because it was made up of hard rules and facts (presumably legible to us humans), and it was argued that it should be obvious that the system wasn't sentient. For the ChatGPT case it's not clear the Chinese room argument even applies. First, it's not even a Chinese room, it's trained by a neural network, an a neural network is inspired by the structure of our brains. There's no "set of instructions in English" for anyone to follow, but it's a rather overwhelming set of neurons somehow producing the results that it was trained on, and as an outsider we have no idea what's going on inside individual neurons -- just like our brains. I'm seeing a lot of people claim that "we know how ChatGPT is created, and so it's not sentient", maybe this looks like the Chinese room fallacy (I'm calling it as such), but it's like claiming we know the laws of physics (Schrödinger's equation etc.) and then claiming we know how the brain works because it's all quantum physics. The fact is we know how to build GPT but we don't actually know how it works. Besides what happens to humanity when we learn more and more about how our own brains work? I'm pretty comfortable with the idea that we're not sentient (yay), but I'm pretty sure those same people would then come up with other nonsensical arguments to prove that they are different from the robots we created.

larryfreeman 4y ago
I believe that it is straight forward to show how Searle's Chinese Room argument can be applied to ChatGPT. The idea is not the "process" for building the rules but the "process" where the rules are applied. True sentience requires a "true" understanding of the information that is being presented (otherwise, it is not reflective of sentience but rather trickery). The Chinese Room applies because ChatGPT or any other computer is generating text purely through logic, that is, logical rules. To make this point clear, let's assume just for argument's sake that at some quantum level, the electrical activity underlying a computer is partially sentient (I am not making this claim -- just using it as an example). Even if this were true, this would not prove that the sentience is involved with the text being generated by the same computer. This underlying sentience would be like the "person" in the Chinese room. The text generated are the Chinese words which would be completely independent from the "sentience" following the rules. Generating text at sufficient complexity to fool the average person does not prove that "sentience" is occurring. Generating text consistently in a way that is only explainable through sentience is what is required. ChatGPT, because it is easily shown to be flawed, does not rise to this level of evidence. In the Chinese Room example, the complexity of what could be displayed by the rulebooks would have a limited complexity (though, the surprise of ChatGPT is that this complexity is greater than many of us would have supposed). Just like the story of Clever Hans the Horse (see https://en.wikipedia.org/wiki/Clever_Hans https://en.wikipedia.org/wiki/Clever_Hans), once the limitation of the process is realized, it becomes clear that there is not "true" understanding of what is generated which means that the rules, in themselves, do not reflect "true" sentience.

imtringued 4y ago

But behavior alone cannot be an indicator for sentience as you can simply record behavior and play it back. Sentience has more to do with sensory perception and feedback loops than mental capability or intelligence. If a future AI attains sentience on a commodity processor, that wouldn't imply that the program is what achieved sentience, rather the computer running the program and all computers before had sentience and the AI is just an interface between the computer and humans that lets it express its sentience in a way that can be understood by humans. If you really believed that behavior is what drives sentience, then non verbal people with autism would have very low sentience and if they somehow overcome their non verbalism, they would gain sentience throughout their life, which is at odds with the fact that many people believe that animals in factory farms are sentient for instance, despite their poor language skills and complete lack of effort needed to achieve sentience.