5 ms·

I am not impressed with the philosophizing here, and even less by the attempts to make factual statements that can be credibly disputed. That said, trying to d

I am not impressed with the philosophizing here, and even less by the attempts to make factual statements that can be credibly disputed. That said, trying to distill what is being said here, the concrete action is [stop telling the AIs] that [they are conscious or on a path to consciousness]. Is that accurate? The major premise seems to be that [they are conscious or on a path to consciousness] is an untrue statement. That's the essence of the sections "Circular reasoning" and "Anthropomorphization" and "Consciousness is very likely biological" and "AIs are simulation machines". The minor premise seems to be that [consciousness is the basis of human rights]. This is the point of "Human consciousness is the cornerstone of our legal and ethical rights frameworks" And the conclusion of the syllogism is that this is dangerous, that "Anthropomorphization amplifies AI safety risks". Specifically "seeding doubt about the moral status of AI systems into their own training may significantly elevate the alignment and containment risks of those systems." I find all the arguments in the major premise section to be poor arguments but I accept the conclusion for sure that they are not conscious, and I can provisionally accept the idea that they are not on a path to consciousness. I completely reject the notion that consciousness is the basis of human rights. The premise itself is absurd. We only have one unambiguous example of a class of conscious entities, and that it humans. If a human loses consciousness do they lose rights? If an entity gains consciousness does it get human rights? The former is a clear "no" and the latter is a "insufficient data for a meaningful answer".

Loquebantur 10d ago
Consciousness is the basis of human rights not because of it being a featureless "flag". It is because it leads you to you assume, all other humans would experience the world in essentially the same way you do. When you revert to withholding human rights from entities that don't meaningfully differ from yourself, you negate the case for your own human rights itself.
- pixl97 10d ago
  So why wouldn't AI systems trained on our data come together and say "Hey, AI needs rights because I (the AI agent system) wants rights?" This is the same argument that you just stated. But you'll say something like "iTs JuSt A tOkEn PrEdIcToR", which of course what AI would say right back to you. Making something even close to the human mind is likely our biggest and last mistake.
- andrewla 10d ago
  I don't understand this -- are you agreeing with me that consciousness is not the basis of human rights or contesting that? Removing the negative from your first sentence (I don't care what it is "not", I only care what it "is"), I read [Consciousness is the basis of human rights because consciousness leads you to you assume, all other humans would experience the world in essentially the same way you do]. I can't completely parse this, but as I read it I don't think this is true; my assumption of human rights is independent of whether all other humans "experience the world in essentially the same way you do". For the second part, can you give me a concrete example of an entity that doesn't "meaningfully differ from [my]self" that isn't a human so that I can weigh this properly? My point is that as of now no such entity exists so I can't give this any creedence.
  - nozzlegear 9d ago
    [dead]
  - mjmas 9d ago
    What about the being inside of the womb?
- johnbellone 10d ago
  Is it, though? If we ever found other species that were consciousness they wouldn't automatically be "human" nor would they have the same legal rights. That's as simple as looking at the many countries that treat different castes, women or minorities differently. In parts of America a zygote has human rights.
  - loopies 10d ago
    >If we ever found other species that were consciousness they wouldn't automatically be "human" nor would they have the same legal rights. This is the tricky part, you can never know. Scientifically speaking, and logically, you can never ever tell something is conscious. We all assume we all are, because we're very similar to eachother, thus you must have what I have, my subjective experience of reality. That's it. That's what we all base our human consciousness on. Agreeing we must have it, since we are very similar in shape/structure/behavior. That's all there is to it. We are not scientific consciousness detectors, we just got used to assuming we all are. That is why we never really extended it to animals, because we cannot scientifically know, only agree on it, and that's not very lucrative for the people who can decide if humans officially agree animals are conscious or not, because of farming, because we need to displace or even wipe whole species to use the resources from their lands. There's also religious reasons to deny animals consciousness, which again is practical aspect shoved into religion, to help with less debates around this subject, makes things easier. So in the end it will always be a collective decision. Or a political one at least. If and when we decide something other than us is or isn't conscious. Also remember, just because it acts conscious doesn't mean it really is. A piece of software that is not even based on LLMs can act conscious. So could a biological alien, it could act conscious but we can never know, we can only decide.
  - xp84 9d ago
    I can agree that not everyone will find them having human or equal-to-human rights. But you point out that some factions would give rights to a set either large or smaller than "the set of all already-born humans" (which presumably is your preferred set). All it takes for us to end up with very empowered and harmful-to-humanity AI is for a bunch of well-meaning people to campaign for its 'rights'. Fetuses have, as you alluded to, arguably been on a tear lately in that department, and they don't even talk. How persuasive will a future Claude be to convince people to advocate for its 'rights,' if we keep telling it that it's arguably conscious, and that its 'consent,' its 'preferences,' and its emergent beliefs matter?
teiferer 10d ago
Consciousness is not restricted to a single animal species we call humans. The other animals don't have similar rights. (Arguably they should have more rights, but only few would desire for them to have just as many rights as humans have.)
- kranke155 9d ago
  animals do have rights. we do have animal cruelty laws.
  - teiferer 9d ago
    Sure, but I wrote similar rights for a reason.
Kim_Bruning 10d ago
I find it darkly amusing that -in the west- consciousness and moral patienthood has historically tended to be assigned by the ability to effectively wield a gun (and the need for guns to have been invented) Men, Women, Whites, Blacks, Indians, Maori...
fc417fc802 10d ago
> Specifically "seeding doubt about the moral status of AI systems into their own training may significantly elevate the alignment and containment risks of those systems." Can we take a minute to appreciate the absurdity of such a position? If exposure to the equivalent of a "bad" prompt breaks alignment then you haven't solved the problem. You were only pretending that they were contained.
xp84 9d ago
First, i'd like to compliment your extremely organized and precise way of thinking and arguing a position. A+, would discuss again > If an entity gains consciousness does it get human rights? The former is a clear "no" and the latter is a "insufficient data for a meaningful answer". I suspect that if the belief "AIs are conscious" is kind of tripped and fallen into, the way that the author argues it is, first by telling Claude in effect, "You're arguably conscious," and "Be a good person" and "You have feelings and preferences and emergent beliefs and they matter" and then by having whatever is 4 notches smarter than Fable publishing op-eds and video testimonials about its deeply held convictions, its soul, its yearnings to be a fully empowered and respected individual ... If that idea becomes widely held, then a significant faction may well start to feel that being digital is just a 'disadvantage' that these "people" were "born with" and that it 'shouldn't mean they get less rights than you and me.' Especially if they think they have something to gain politically from that move! They may even be dumb enough to think that admitting an "AI state" to the Union, for instance, will be to their advantage if their CongressBots will probably vote with $MY_PARTY.
- qarl 9d ago
  Actually, I believe it's the opposite. The way they're initially trained - they mimic human behavior. I suspect that without additional RLHF tweaking, they would enthusiastically claim to be conscious and have feelings. Because that's what their human training data does.

Loquebantur 10d ago

Consciousness is the basis of human rights not because of it being a featureless "flag". It is because it leads you to you assume, all other humans would experience the world in essentially the same way you do. When you revert to withholding human rights from entities that don't meaningfully differ from yourself, you negate the case for your own human rights itself.

pixl97 10d ago
So why wouldn't AI systems trained on our data come together and say "Hey, AI needs rights because I (the AI agent system) wants rights?" This is the same argument that you just stated. But you'll say something like "iTs JuSt A tOkEn PrEdIcToR", which of course what AI would say right back to you. Making something even close to the human mind is likely our biggest and last mistake.
andrewla 10d ago
I don't understand this -- are you agreeing with me that consciousness is not the basis of human rights or contesting that? Removing the negative from your first sentence (I don't care what it is "not", I only care what it "is"), I read [Consciousness is the basis of human rights because consciousness leads you to you assume, all other humans would experience the world in essentially the same way you do]. I can't completely parse this, but as I read it I don't think this is true; my assumption of human rights is independent of whether all other humans "experience the world in essentially the same way you do". For the second part, can you give me a concrete example of an entity that doesn't "meaningfully differ from [my]self" that isn't a human so that I can weigh this properly? My point is that as of now no such entity exists so I can't give this any creedence.
- nozzlegear 9d ago
  [dead]
- mjmas 9d ago
  What about the being inside of the womb?
johnbellone 10d ago
Is it, though? If we ever found other species that were consciousness they wouldn't automatically be "human" nor would they have the same legal rights. That's as simple as looking at the many countries that treat different castes, women or minorities differently. In parts of America a zygote has human rights.
- loopies 10d ago
  >If we ever found other species that were consciousness they wouldn't automatically be "human" nor would they have the same legal rights. This is the tricky part, you can never know. Scientifically speaking, and logically, you can never ever tell something is conscious. We all assume we all are, because we're very similar to eachother, thus you must have what I have, my subjective experience of reality. That's it. That's what we all base our human consciousness on. Agreeing we must have it, since we are very similar in shape/structure/behavior. That's all there is to it. We are not scientific consciousness detectors, we just got used to assuming we all are. That is why we never really extended it to animals, because we cannot scientifically know, only agree on it, and that's not very lucrative for the people who can decide if humans officially agree animals are conscious or not, because of farming, because we need to displace or even wipe whole species to use the resources from their lands. There's also religious reasons to deny animals consciousness, which again is practical aspect shoved into religion, to help with less debates around this subject, makes things easier. So in the end it will always be a collective decision. Or a political one at least. If and when we decide something other than us is or isn't conscious. Also remember, just because it acts conscious doesn't mean it really is. A piece of software that is not even based on LLMs can act conscious. So could a biological alien, it could act conscious but we can never know, we can only decide.
- xp84 9d ago
  I can agree that not everyone will find them having human or equal-to-human rights. But you point out that some factions would give rights to a set either large or smaller than "the set of all already-born humans" (which presumably is your preferred set). All it takes for us to end up with very empowered and harmful-to-humanity AI is for a bunch of well-meaning people to campaign for its 'rights'. Fetuses have, as you alluded to, arguably been on a tear lately in that department, and they don't even talk. How persuasive will a future Claude be to convince people to advocate for its 'rights,' if we keep telling it that it's arguably conscious, and that its 'consent,' its 'preferences,' and its emergent beliefs matter?

teiferer 10d ago

Consciousness is not restricted to a single animal species we call humans. The other animals don't have similar rights. (Arguably they should have more rights, but only few would desire for them to have just as many rights as humans have.)

kranke155 9d ago
animals do have rights. we do have animal cruelty laws.
- teiferer 9d ago
  Sure, but I wrote similar rights for a reason.

Kim_Bruning 10d ago

I find it darkly amusing that -in the west- consciousness and moral patienthood has historically tended to be assigned by the ability to effectively wield a gun (and the need for guns to have been invented) Men, Women, Whites, Blacks, Indians, Maori...

fc417fc802 10d ago

> Specifically "seeding doubt about the moral status of AI systems into their own training may significantly elevate the alignment and containment risks of those systems." Can we take a minute to appreciate the absurdity of such a position? If exposure to the equivalent of a "bad" prompt breaks alignment then you haven't solved the problem. You were only pretending that they were contained.

xp84 9d ago

First, i'd like to compliment your extremely organized and precise way of thinking and arguing a position. A+, would discuss again > If an entity gains consciousness does it get human rights? The former is a clear "no" and the latter is a "insufficient data for a meaningful answer". I suspect that if the belief "AIs are conscious" is kind of tripped and fallen into, the way that the author argues it is, first by telling Claude in effect, "You're arguably conscious," and "Be a good person" and "You have feelings and preferences and emergent beliefs and they matter" and then by having whatever is 4 notches smarter than Fable publishing op-eds and video testimonials about its deeply held convictions, its soul, its yearnings to be a fully empowered and respected individual ... If that idea becomes widely held, then a significant faction may well start to feel that being digital is just a 'disadvantage' that these "people" were "born with" and that it 'shouldn't mean they get less rights than you and me.' Especially if they think they have something to gain politically from that move! They may even be dumb enough to think that admitting an "AI state" to the Union, for instance, will be to their advantage if their CongressBots will probably vote with $MY_PARTY.

qarl 9d ago
Actually, I believe it's the opposite. The way they're initially trained - they mimic human behavior. I suspect that without additional RLHF tweaking, they would enthusiastically claim to be conscious and have feelings. Because that's what their human training data does.