6 ms·

Strong disagree. It might be the only thing that does actually matter. Rocks, present-day computers and other unconscious objects have no moral value because th

Strong disagree. It might be the only thing that does actually matter. Rocks, present-day computers and other unconscious objects have no moral value because they're incapable of suffering and experiences. An AGI that's unconscious has no moral value either. It is just a tool, no different in concept and moral worth to my screwdriver.

Isinlor 5y ago
Explain then, what should the difference between you and your screwdriver be to me? Why should I believe you on a word that you are conscious and not believe a smart algorithm?
- bumby 5y ago
  I'm a bit out of my lane here so maybe somebody can jump in with a better understanding. >what should the difference between you and your screwdriver be to me? If we can't define consciousness, I can't prove to you that I'm not effectively an unconscious zombie. We create social contracts based, in part, on empathy. I.e., I can estimate what it "feels like" to be you and therefore extend certain rights to you as a recognition of that subjective conscious experience. If you were simply an unconscious moist robut, there's nothing immoral about me subjugating you, or stomping on your foot, or anything else that relies on rights or subjective suffering. So I'll assert the question back to you: since you can't prove I'm conscious, what prevents you from flippantly treating me just as you would a screwdriver?
  - theamk 5y ago
    Probably because most people don't care about consciousness and treat people like people if they can relate to them? You said it yourself -- "we create social contracts based, in part, on empathy". As long as empathy works, we should be treating others nice. Trying to replace simple and understandable things like "empathy" with vague "consciousness" leads to dangerous roads. One moment you are talking philosophy and next moment you are declaring beings you don't like as "unconscious zombie" and you are enslaving and subjugating them.
    - bumby 5y ago
      We're saying the same thing, I think. I'm just asserting that ideas like empathy are rooted in an understanding of subjective experience. I.e., they are based on the idea of consciousness, even if we struggle to define it. I would hope better definitions lead to more clarity, not less. To build on your example, a robust definition of consciousness could prevent declaring someone an "unconscious zombie". The opposite may be true, however. It would potentially cause problems for lots of people if we could prove certain subjects (plants, for example) have subjective experience.
  - GoblinSlayer 5y ago
    So you flippantly kill plants?
    - bumby 5y ago
      I do. I was just weeding my yard yesterday. To the same point, we generally don't have reservations about that because we don't ascribe them moral value. As I understand it, even the environmental protections for endangered plants are based in the plant's potential utility for humans, not their inherent moral value.
      - GoblinSlayer 5y ago
        Life has no value for you? Plants aren't inert and defend themselves from harm. Given enough reason it's legal to kill humans too. Sometimes it's mandatory, e.g. at war, when utility of murder is more important than their inherent moral value.
        bumby 5y ago
        >Life has no value for you? Sure. And, while I know you're just trying to communicate a different stance it borders on straw-manning. It's not that life has no value. I know there are some ideologies that don't think any life (including plants) should be harmed [1], most forms of animal life (including humans) are predicated on other life forms dying for them to continue. To me, the more relevant concern regarding this conversation is 'suffering' not 'life'. To that end, if we can define consciousness and use that to defend the position that plants have subjective experience and suffer, I would probably stop weeding my yard. [1] https://en.wikipedia.org/wiki/Ahimsa_in_Jainism https://en.wikipedia.org/wiki/Ahimsa_in_Jainism
        GoblinSlayer 5y ago
        Your inability to understand murder is relevant. Law imposes punishment even when one didn't harm anyone, and you happily kill living beings.
  - Isinlor 5y ago
    The biggest factor is that evolution made me to be a social creature. As a product of that evolution, I was also socialized to be empathic. Notion of consciousness is not necessary to create and explain morality. BTW - We call people who will treat you like a screwdriver a psychopath (or bureaucrat :P). And people like that exists, they apparently do not estimate what it "feels like" to be you. Maybe because it does not feel anything like to be them? Or maybe consciousness is just property of existence and interaction, in the same way a color charge is a property of a quark, and so consciousness has nothing to do with high level behavior. There are also people with lock-in syndrome. In most extreme cases they can not interact in any way besides maybe changing patterns of their brain activity in response to what's happening. We can try to establish trough contrived communication protocols if their responses make sense, sometimes just to ask if they even want to live like that. There are also people in persistent vegetative state, where we are not be able to establish clear communication protocol. The whole thing is probably a multidimensional spectrum, depending on what damage these people suffered. So, it may be extremely hard to tell if they are aware and whether we should kill them / stop their life support or not.
- fighterpilot 5y ago
  Well that's the question of solipsism. You don't know that I'm conscious and I can never prove it. However, I can offer you a reason to believe I am conscious, which is that my brain is very similar in structure to yours. That's not conclusive evidence, but it might be enough to persuade you.
theamk 5y ago
Ok, so who cares about "moral value"? If I have an AGI friend, and they behave exactly like human and does all the things humans do, including having morals indistinguishable from humans, and having feelings indistinguishable from humans, I am going to treat them as a human. I am definitely not going to treat them as "screwdriver" just because some philosopher somewhere said they have no "consciousness" and that they have "no moral value".
- fighterpilot 5y ago
  If they have feelings indistinguishable from humans, as you've put it, then that is not an unconscious AGI. Quite the opposite. It's not clear to me that you've understood the distinction between conscious and unconscious AGI.
  - theamk 5y ago
    Well, according to the top-level post we are discussing, conscious requires "shared physical resonance". I see no evidence that this resonance is required for "feelings indistinguishable from humans" bit, so it could one thing and not other. Also, there are people out there who believe in "philosophical zombies" -- beings that are "physically identical to and indistinguishable from a normal person but does not have conscious experience" [0]. People who believe that think that "a philosophical zombie were poked with a sharp object it would not inwardly feel any pain, yet it would outwardly behave exactly as if it did feel pain" I hope you see what this leads to -- just call someone "p-zombie" and you can make them suffer without feeling bad or having ethical problems. Based on your comment, it looks like you disagree with that definition -- but when you say things like "An AGI that's unconscious has no moral value" this certainly brings up p-zombie arguments and the idea that it is OK to make some beings suffer because they lack something unmeasurable and unquantifiable. [0] https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
- bumby 5y ago
  This is reasonably straightforward if you're dealing with a robot that is indistinguishable from humans but I think it can get complicated as you get further from our place on the evolutionary tree. Why do humans (generally) treat mammals (especially those selected as pets) differently from, say, insects? People think nothing of swatting mosquitos but most would find wonton killing of pet species as morally abhorrent. If it could be proven that, say, mollusks or insects have every bit of consciousness and capability of subjective suffering as humans, would that change the way we think about the above? I would say it probably would in part because of how we use consciousness/suffering to define moral value. The difference being we mostly use rough heuristics now to gauge the consciousness of other animals rather than a scientifically defined method.
FinanceAnon 5y ago
What defines something's "moral value"? What is "suffering"? Is it not just a signal that triggers certain pathways in the brain? Could it not be built/simulated? Is suffering necessary for something to be considered conscious? What does it mean to "experience" something?
- fighterpilot 5y ago
  Experience = qualia. No, suffering isn't a necessary condition for consciousness. We can make rats euphoric in a lab with electrodes and they are presumably conscious despite a lack of suffering. Suffering is just one example of a conscious state. On the question of moral value, we aren't going to resolve that here. But if anything has moral value, it ought to be consciousness. That is what most people value in their personal ethics and relationships.
  - FinanceAnon 5y ago
    I don't think human-defined moral value has much to do with consciousness. A sacred object (or animal), might have a high moral value in certain cultures, but it doesn't mean it's highly conscious.
pitspotter2 5y ago
Yes, consciousness may well be the most important thing. Ultimately it's of no use relying on legal frameworks and so on without acknowledging this. Laws can be overturned or bypassed too easily. Ethics fluctuate according to fashion. We shouldn't be cruel to animals because there's something it's like to be a bear, for instance, including being a suffering bear. It's worth distinguishing between AI and AGI. An AI is a non-conscious tool (like a go-playing program or an image analyser) whereas an AGI is a conscious, general intelligence which can learn anything it wants to when the opportunity arises. (I'm claiming here that general intelligence is not possible without consciousness.) Animals are intelligent, conscious and can learn many specific things but don't possess a general learning ability because they lack language and hence they cannot think freely. A tool can be used and discarded at whim. A general intelligence, person, or animal should not be, because it is a subject.

Isinlor 5y ago

Explain then, what should the difference between you and your screwdriver be to me? Why should I believe you on a word that you are conscious and not believe a smart algorithm?

bumby 5y ago
I'm a bit out of my lane here so maybe somebody can jump in with a better understanding. >what should the difference between you and your screwdriver be to me? If we can't define consciousness, I can't prove to you that I'm not effectively an unconscious zombie. We create social contracts based, in part, on empathy. I.e., I can estimate what it "feels like" to be you and therefore extend certain rights to you as a recognition of that subjective conscious experience. If you were simply an unconscious moist robut, there's nothing immoral about me subjugating you, or stomping on your foot, or anything else that relies on rights or subjective suffering. So I'll assert the question back to you: since you can't prove I'm conscious, what prevents you from flippantly treating me just as you would a screwdriver?
- theamk 5y ago
  Probably because most people don't care about consciousness and treat people like people if they can relate to them? You said it yourself -- "we create social contracts based, in part, on empathy". As long as empathy works, we should be treating others nice. Trying to replace simple and understandable things like "empathy" with vague "consciousness" leads to dangerous roads. One moment you are talking philosophy and next moment you are declaring beings you don't like as "unconscious zombie" and you are enslaving and subjugating them.
  - bumby 5y ago
    We're saying the same thing, I think. I'm just asserting that ideas like empathy are rooted in an understanding of subjective experience. I.e., they are based on the idea of consciousness, even if we struggle to define it. I would hope better definitions lead to more clarity, not less. To build on your example, a robust definition of consciousness could prevent declaring someone an "unconscious zombie". The opposite may be true, however. It would potentially cause problems for lots of people if we could prove certain subjects (plants, for example) have subjective experience.
- GoblinSlayer 5y ago
  So you flippantly kill plants?
  - bumby 5y ago
    I do. I was just weeding my yard yesterday. To the same point, we generally don't have reservations about that because we don't ascribe them moral value. As I understand it, even the environmental protections for endangered plants are based in the plant's potential utility for humans, not their inherent moral value.
    - GoblinSlayer 5y ago
      Life has no value for you? Plants aren't inert and defend themselves from harm. Given enough reason it's legal to kill humans too. Sometimes it's mandatory, e.g. at war, when utility of murder is more important than their inherent moral value.
      - bumby 5y ago
        >Life has no value for you? Sure. And, while I know you're just trying to communicate a different stance it borders on straw-manning. It's not that life has no value. I know there are some ideologies that don't think any life (including plants) should be harmed [1], most forms of animal life (including humans) are predicated on other life forms dying for them to continue. To me, the more relevant concern regarding this conversation is 'suffering' not 'life'. To that end, if we can define consciousness and use that to defend the position that plants have subjective experience and suffer, I would probably stop weeding my yard. [1] https://en.wikipedia.org/wiki/Ahimsa_in_Jainism https://en.wikipedia.org/wiki/Ahimsa_in_Jainism
        GoblinSlayer 5y ago
        Your inability to understand murder is relevant. Law imposes punishment even when one didn't harm anyone, and you happily kill living beings.
- Isinlor 5y ago
  The biggest factor is that evolution made me to be a social creature. As a product of that evolution, I was also socialized to be empathic. Notion of consciousness is not necessary to create and explain morality. BTW - We call people who will treat you like a screwdriver a psychopath (or bureaucrat :P). And people like that exists, they apparently do not estimate what it "feels like" to be you. Maybe because it does not feel anything like to be them? Or maybe consciousness is just property of existence and interaction, in the same way a color charge is a property of a quark, and so consciousness has nothing to do with high level behavior. There are also people with lock-in syndrome. In most extreme cases they can not interact in any way besides maybe changing patterns of their brain activity in response to what's happening. We can try to establish trough contrived communication protocols if their responses make sense, sometimes just to ask if they even want to live like that. There are also people in persistent vegetative state, where we are not be able to establish clear communication protocol. The whole thing is probably a multidimensional spectrum, depending on what damage these people suffered. So, it may be extremely hard to tell if they are aware and whether we should kill them / stop their life support or not.
fighterpilot 5y ago
Well that's the question of solipsism. You don't know that I'm conscious and I can never prove it. However, I can offer you a reason to believe I am conscious, which is that my brain is very similar in structure to yours. That's not conclusive evidence, but it might be enough to persuade you.

theamk 5y ago

Ok, so who cares about "moral value"? If I have an AGI friend, and they behave exactly like human and does all the things humans do, including having morals indistinguishable from humans, and having feelings indistinguishable from humans, I am going to treat them as a human. I am definitely not going to treat them as "screwdriver" just because some philosopher somewhere said they have no "consciousness" and that they have "no moral value".

fighterpilot 5y ago
If they have feelings indistinguishable from humans, as you've put it, then that is not an unconscious AGI. Quite the opposite. It's not clear to me that you've understood the distinction between conscious and unconscious AGI.
- theamk 5y ago
  Well, according to the top-level post we are discussing, conscious requires "shared physical resonance". I see no evidence that this resonance is required for "feelings indistinguishable from humans" bit, so it could one thing and not other. Also, there are people out there who believe in "philosophical zombies" -- beings that are "physically identical to and indistinguishable from a normal person but does not have conscious experience" [0]. People who believe that think that "a philosophical zombie were poked with a sharp object it would not inwardly feel any pain, yet it would outwardly behave exactly as if it did feel pain" I hope you see what this leads to -- just call someone "p-zombie" and you can make them suffer without feeling bad or having ethical problems. Based on your comment, it looks like you disagree with that definition -- but when you say things like "An AGI that's unconscious has no moral value" this certainly brings up p-zombie arguments and the idea that it is OK to make some beings suffer because they lack something unmeasurable and unquantifiable. [0] https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie
bumby 5y ago
This is reasonably straightforward if you're dealing with a robot that is indistinguishable from humans but I think it can get complicated as you get further from our place on the evolutionary tree. Why do humans (generally) treat mammals (especially those selected as pets) differently from, say, insects? People think nothing of swatting mosquitos but most would find wonton killing of pet species as morally abhorrent. If it could be proven that, say, mollusks or insects have every bit of consciousness and capability of subjective suffering as humans, would that change the way we think about the above? I would say it probably would in part because of how we use consciousness/suffering to define moral value. The difference being we mostly use rough heuristics now to gauge the consciousness of other animals rather than a scientifically defined method.

FinanceAnon 5y ago

What defines something's "moral value"? What is "suffering"? Is it not just a signal that triggers certain pathways in the brain? Could it not be built/simulated? Is suffering necessary for something to be considered conscious? What does it mean to "experience" something?

fighterpilot 5y ago
Experience = qualia. No, suffering isn't a necessary condition for consciousness. We can make rats euphoric in a lab with electrodes and they are presumably conscious despite a lack of suffering. Suffering is just one example of a conscious state. On the question of moral value, we aren't going to resolve that here. But if anything has moral value, it ought to be consciousness. That is what most people value in their personal ethics and relationships.
- FinanceAnon 5y ago
  I don't think human-defined moral value has much to do with consciousness. A sacred object (or animal), might have a high moral value in certain cultures, but it doesn't mean it's highly conscious.

pitspotter2 5y ago

Yes, consciousness may well be the most important thing. Ultimately it's of no use relying on legal frameworks and so on without acknowledging this. Laws can be overturned or bypassed too easily. Ethics fluctuate according to fashion. We shouldn't be cruel to animals because there's something it's like to be a bear, for instance, including being a suffering bear. It's worth distinguishing between AI and AGI. An AI is a non-conscious tool (like a go-playing program or an image analyser) whereas an AGI is a conscious, general intelligence which can learn anything it wants to when the opportunity arises. (I'm claiming here that general intelligence is not possible without consciousness.) Animals are intelligent, conscious and can learn many specific things but don't possess a general learning ability because they lack language and hence they cannot think freely. A tool can be used and discarded at whim. A general intelligence, person, or animal should not be, because it is a subject.