3 ms·

Ok, so who cares about "moral value"? If I have an AGI friend, and they behave exactly like human and does all the things humans do, including having morals in

by theamk 5y ago

Ok, so who cares about "moral value"? If I have an AGI friend, and they behave exactly like human and does all the things humans do, including having morals indistinguishable from humans, and having feelings indistinguishable from humans, I am going to treat them as a human. I am definitely not going to treat them as "screwdriver" just because some philosopher somewhere said they have no "consciousness" and that they have "no moral value".

fighterpilot 5y ago
If they have feelings indistinguishable from humans, as you've put it, then that is not an unconscious AGI. Quite the opposite. It's not clear to me that you've understood the distinction between conscious and unconscious AGI.
- theamk 5y ago
  Well, according to the top-level post we are discussing, conscious requires "shared physical resonance". I see no evidence that this resonance is required for "feelings indistinguishable from humans" bit, so it could one thing and not other. Also, there are people out there who believe in "philosophical zombies" -- beings that are "physically identical to and indistinguishable from a normal person but does not have conscious experience" [0]. People who believe that think that "a philosophical zombie were poked with a sharp object it would not inwardly feel any pain, yet it would outwardly behave exactly as if it did feel pain" I hope you see what this leads to -- just call someone "p-zombie" and you can make them suffer without feeling bad or having ethical problems. Based on your comment, it looks like you disagree with that definition -- but when you say things like "An AGI that's unconscious has no moral value" this certainly brings up p-zombie arguments and the idea that it is OK to make some beings suffer because they lack something unmeasurable and unquantifiable. [0] https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
bumby 5y ago
This is reasonably straightforward if you're dealing with a robot that is indistinguishable from humans but I think it can get complicated as you get further from our place on the evolutionary tree. Why do humans (generally) treat mammals (especially those selected as pets) differently from, say, insects? People think nothing of swatting mosquitos but most would find wonton killing of pet species as morally abhorrent. If it could be proven that, say, mollusks or insects have every bit of consciousness and capability of subjective suffering as humans, would that change the way we think about the above? I would say it probably would in part because of how we use consciousness/suffering to define moral value. The difference being we mostly use rough heuristics now to gauge the consciousness of other animals rather than a scientifically defined method.
- GoblinSlayer 5y ago
  Again I see this strange idea that insects don't feel pain. Ants express pain much more actively than mammals, they don't cease to move after receiving injury.
  - bumby 5y ago
    I think the relevant distinction is between “responding to threatening stimuli” and “suffering”. I don’t know where the definition of pain would be on that spectrum. There are some neuroscientists who think you need a neocortex to suffer. From that perspective, lower cognitively evolved species like ants and lobsters don’t suffer in the same way we empathetically assume mammals do.
    - GoblinSlayer 5y ago
      That sounds like philosophical zombie. Pain is stimulus, what is distinction?
      - bumby 5y ago
        Pain and suffering are not synonymous. I don’t know if I have the clearest definition, but suffering may have a rumination aspect to it. Plants respond to stimuli, but I’m not convinced they suffer. E.g., a hard workout can cause “pain”, but many people come to like, or even look forward to, that feeling. It still hurts, but the way they are mentally reflecting on that pain distinguishes it from “suffering”. Meanwhile, someone else who hates exercise may feel the exact same pain and be miserable. So it’s not the pain, but the way the pain is framed which leads to suffering.

by theamk 5y ago

fighterpilot 5y ago

If they have feelings indistinguishable from humans, as you've put it, then that is not an unconscious AGI. Quite the opposite. It's not clear to me that you've understood the distinction between conscious and unconscious AGI.

theamk 5y ago
Well, according to the top-level post we are discussing, conscious requires "shared physical resonance". I see no evidence that this resonance is required for "feelings indistinguishable from humans" bit, so it could one thing and not other. Also, there are people out there who believe in "philosophical zombies" -- beings that are "physically identical to and indistinguishable from a normal person but does not have conscious experience" [0]. People who believe that think that "a philosophical zombie were poked with a sharp object it would not inwardly feel any pain, yet it would outwardly behave exactly as if it did feel pain" I hope you see what this leads to -- just call someone "p-zombie" and you can make them suffer without feeling bad or having ethical problems. Based on your comment, it looks like you disagree with that definition -- but when you say things like "An AGI that's unconscious has no moral value" this certainly brings up p-zombie arguments and the idea that it is OK to make some beings suffer because they lack something unmeasurable and unquantifiable. [0] https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie

bumby 5y ago

This is reasonably straightforward if you're dealing with a robot that is indistinguishable from humans but I think it can get complicated as you get further from our place on the evolutionary tree. Why do humans (generally) treat mammals (especially those selected as pets) differently from, say, insects? People think nothing of swatting mosquitos but most would find wonton killing of pet species as morally abhorrent. If it could be proven that, say, mollusks or insects have every bit of consciousness and capability of subjective suffering as humans, would that change the way we think about the above? I would say it probably would in part because of how we use consciousness/suffering to define moral value. The difference being we mostly use rough heuristics now to gauge the consciousness of other animals rather than a scientifically defined method.

GoblinSlayer 5y ago
Again I see this strange idea that insects don't feel pain. Ants express pain much more actively than mammals, they don't cease to move after receiving injury.
- bumby 5y ago
  I think the relevant distinction is between “responding to threatening stimuli” and “suffering”. I don’t know where the definition of pain would be on that spectrum. There are some neuroscientists who think you need a neocortex to suffer. From that perspective, lower cognitively evolved species like ants and lobsters don’t suffer in the same way we empathetically assume mammals do.
  - GoblinSlayer 5y ago
    That sounds like philosophical zombie. Pain is stimulus, what is distinction?
    - bumby 5y ago
      Pain and suffering are not synonymous. I don’t know if I have the clearest definition, but suffering may have a rumination aspect to it. Plants respond to stimuli, but I’m not convinced they suffer. E.g., a hard workout can cause “pain”, but many people come to like, or even look forward to, that feeling. It still hurts, but the way they are mentally reflecting on that pain distinguishes it from “suffering”. Meanwhile, someone else who hates exercise may feel the exact same pain and be miserable. So it’s not the pain, but the way the pain is framed which leads to suffering.