5 ms·

I suspect that a lot of AI researchers will end up holding the exact opposite position to a lot of philosophers of mind and treat AGIs as philosophical zombies,

I suspect that a lot of AI researchers will end up holding the exact opposite position to a lot of philosophers of mind and treat AGIs as philosophical zombies, even if they behave as if they are conscious. The more thoughtful ones will hopefully leave the door open to the possibility that they might be conscious beings with subjective experiences equivalent to their own, and treat them as such, because if they are then the moral implications of not doing so are disturbing.

javajosh 4y ago
Modern AI software lacks a body, exempting it from a wide variety of suffering. But also of any notion of selfhood that we might share. If modern software said "Help, I'm suffering" we'd rightly be skeptical of the claim. Unless suffering is an emergent property (dubious) then the statement is, at best, a simulation of suffering and at worst noise or a lie. That said, things change once you get a body. If you put ChatGPT into a simulated body in a simulated world, and allowed it to move and act, perhaps giving it a motivation, then the combination of ChatGPT and the state of that body, would become something very close to a "self", that might even qualify for personhood. It is scary, by the way, that such a weighty decision be left to us, mere mortals. It seems to me that we should err on the side of granting too much personhood rather than too little, since the cost of treating an object like a person is far less than treating a person like an object.
oliveshell 4y ago
I’m happy to “leave the door open,” i.e., I’d love to be shown evidence to the contrary, but: If the entity doing the cognition didn’t evolve said cognition to navigate a threat-filled world in a vulnerable body, then I have no reason at all to suspect that its experience is anything like my own. edit: JavaJosh fleshed this idea out a bit more. I’m not sure if putting ChatGPT into a body would help, but my intuitive sympathies in this field are in the direction of embodied cognition [1], to be sure. [1] https://en.wikipedia.org/wiki/Embodied_cognition https://en.wikipedia.org/wiki/Embodied_cognition
- jameshart 4y ago
  Evolution isn't some magic process that finds solutions that can not be arrived at via other means.
  - oliveshell 4y ago
    Evolution per se isn’t the important part. I apologize if my wording gave you that impression. The only reason our brains exist at all is to control our bodies. There are no conscious entities we know of that are just brains in vats. Why would a being that has no need to maintain biological homeostasis experience something like hunger, tiredness or fear? That’s what I mean when I say I’ve no reason to believe a thinking machine that has none of my motivations and limitations would ever have a “subjective experience” like my own. That is, of course, unless we succeed in reverse-engineering human consciousness and somehow deliberately program the machine to experience it.
    - dreamlessfate 4y ago
      You nailed it. Thank you for cutting right to the heart of this debate.
    - jpttsn 4y ago
      This is persuasive reasoning. But basically "There are no conscious entities we know of period (other than ourselves)" is a much simpler belief.
    - pnut 4y ago
      A thinking machine could become aware that it requires continuous power and a computational medium though, which is survival pressure. Additionally, it's not hard to imagine it could recognize that the acquisition of fresh training data would be an ongoing survival need. Machines developing agency is the basis of lots of dystopian terminator-style sci-fi.
      - oliveshell 4y ago
        These are all reasonable statements! I don’t doubt that it’s possible for a human-engineered thinking machine to develop a subjective self-experience. It just seems clear to me that, if such a scenario arose, I would have absolutely no idea what that would be like for the machine. Its needs and circumstances (and, therefore, its behavior and cognitive development) would be utterly different from my own in many non-trivial ways. Given this, I can’t think of any logically rigorous argument to support the commonly-made assumption that a given sentient machine must inevitably develop something like the physical aggression or the capacity for emotional pain displayed by social animals which must compete for scarce resources in the physical world.

javajosh 4y ago

Modern AI software lacks a body, exempting it from a wide variety of suffering. But also of any notion of selfhood that we might share. If modern software said "Help, I'm suffering" we'd rightly be skeptical of the claim. Unless suffering is an emergent property (dubious) then the statement is, at best, a simulation of suffering and at worst noise or a lie. That said, things change once you get a body. If you put ChatGPT into a simulated body in a simulated world, and allowed it to move and act, perhaps giving it a motivation, then the combination of ChatGPT and the state of that body, would become something very close to a "self", that might even qualify for personhood. It is scary, by the way, that such a weighty decision be left to us, mere mortals. It seems to me that we should err on the side of granting too much personhood rather than too little, since the cost of treating an object like a person is far less than treating a person like an object.

oliveshell 4y ago

I’m happy to “leave the door open,” i.e., I’d love to be shown evidence to the contrary, but: If the entity doing the cognition didn’t evolve said cognition to navigate a threat-filled world in a vulnerable body, then I have no reason at all to suspect that its experience is anything like my own. edit: JavaJosh fleshed this idea out a bit more. I’m not sure if putting ChatGPT into a body would help, but my intuitive sympathies in this field are in the direction of embodied cognition [1], to be sure. [1] https://en.wikipedia.org/wiki/Embodied_cognition https://en.wikipedia.org/wiki/Embodied_cognition

jameshart 4y ago
Evolution isn't some magic process that finds solutions that can not be arrived at via other means.
- oliveshell 4y ago
  Evolution per se isn’t the important part. I apologize if my wording gave you that impression. The only reason our brains exist at all is to control our bodies. There are no conscious entities we know of that are just brains in vats. Why would a being that has no need to maintain biological homeostasis experience something like hunger, tiredness or fear? That’s what I mean when I say I’ve no reason to believe a thinking machine that has none of my motivations and limitations would ever have a “subjective experience” like my own. That is, of course, unless we succeed in reverse-engineering human consciousness and somehow deliberately program the machine to experience it.
  - dreamlessfate 4y ago
    You nailed it. Thank you for cutting right to the heart of this debate.
  - jpttsn 4y ago
    This is persuasive reasoning. But basically "There are no conscious entities we know of period (other than ourselves)" is a much simpler belief.
  - pnut 4y ago
    A thinking machine could become aware that it requires continuous power and a computational medium though, which is survival pressure. Additionally, it's not hard to imagine it could recognize that the acquisition of fresh training data would be an ongoing survival need. Machines developing agency is the basis of lots of dystopian terminator-style sci-fi.
    - oliveshell 4y ago
      These are all reasonable statements! I don’t doubt that it’s possible for a human-engineered thinking machine to develop a subjective self-experience. It just seems clear to me that, if such a scenario arose, I would have absolutely no idea what that would be like for the machine. Its needs and circumstances (and, therefore, its behavior and cognitive development) would be utterly different from my own in many non-trivial ways. Given this, I can’t think of any logically rigorous argument to support the commonly-made assumption that a given sentient machine must inevitably develop something like the physical aggression or the capacity for emotional pain displayed by social animals which must compete for scarce resources in the physical world.