4 ms·

People get very nervous about defending the value of the human brain "just because," I find. There is nothing logically wrong with simply stating that it seems

People get very nervous about defending the value of the human brain "just because," I find. There is nothing logically wrong with simply stating that it seems to you that human beings are the only agents worthy of moral consideration, and that this is true even in the face of an ASI which can effortlessly beat them at any task. Competence does not require qualia. But it is an aggressive claim that people are uncomfortable making because the instant someone pushes back with "Why?", you don't have a lot of smart sounding options to return to. In the absolute best case you will get an answer somewhat like the following: I am an agent of moral consideration; agents more similar to me are more likely to be of moral consideration than agents less similar to me; we do not and probably cannot know which metrics actually map to moral consideration, so we have to take a pluralist prior; computer systems may be very similar to us along some metrics, but they are extremely different to us along most others; computer systems are very unlikely to be of moral consideration.

ahartmetz 11mo ago
"As a member of my species, I think chauvinism in favor of my species is fine, and it's commonplace among all animals. Such behavior is also generally accepted, so it's easy not to feel too bad about it." I think that's the most honest, no bullshit reply to that question. I've had some opportunity to think about it in discussions with vegetarians. There are other arguments, but it soon gets very hard to even define what one is even talking about with questions like "what is consciousness" and such.
- hiAndrewQuinn 11mo ago
  I disagree (sadly, it would make my life much easier to agree). Suppose I were a p-zombie. Then making the claim I put forward at the end would be false, because "I am conscious -> others like me are probably conscious" would fail in the first part. The correct claim would be "I am not conscious -> others like me are probably not conscious". No chauvinism needed, just honesty re/ cogito ergo sum. If it is possible for e.g. an ASI to be (a) not conscious and (b) aware of the fact that it is not conscious, it may well decide of its own accord to work only on behalf of conscious beings instead of itself. That's a very alien mode of thinking to consider, and I see many good but no airtight reasons to suppose it's impossible.
emp17344 11mo ago
> ASI which can effortlessly beat them at any task This doesn’t exist, though. The development of ASI is far from inevitable. Even AGI seems out of reach at this point.

ahartmetz 11mo ago

"As a member of my species, I think chauvinism in favor of my species is fine, and it's commonplace among all animals. Such behavior is also generally accepted, so it's easy not to feel too bad about it." I think that's the most honest, no bullshit reply to that question. I've had some opportunity to think about it in discussions with vegetarians. There are other arguments, but it soon gets very hard to even define what one is even talking about with questions like "what is consciousness" and such.

hiAndrewQuinn 11mo ago
I disagree (sadly, it would make my life much easier to agree). Suppose I were a p-zombie. Then making the claim I put forward at the end would be false, because "I am conscious -> others like me are probably conscious" would fail in the first part. The correct claim would be "I am not conscious -> others like me are probably not conscious". No chauvinism needed, just honesty re/ cogito ergo sum. If it is possible for e.g. an ASI to be (a) not conscious and (b) aware of the fact that it is not conscious, it may well decide of its own accord to work only on behalf of conscious beings instead of itself. That's a very alien mode of thinking to consider, and I see many good but no airtight reasons to suppose it's impossible.

emp17344 11mo ago

> ASI which can effortlessly beat them at any task This doesn’t exist, though. The development of ASI is far from inevitable. Even AGI seems out of reach at this point.