7 ms·

I appreciate his openness. > Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that

by xg15 12d ago

I appreciate his openness. > Unfortunately, there’s a growing chorus of people who argue that AIs could now be, or may soon become, conscious. They argue that AIs may deserve rights and protections similar to those that we provide other conscious beings.12 If this view takes hold, it will shake the foundations of our society, rupturing our existing political and ethical frameworks, and fundamentally changing what it means to be human. So completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are. That's basically the same way people argue about animals, only they usually don't say it as openly. As for the actual question, I agree that with current LLMs, there is not a lot there that could be conscious outside the inference loop (and if it were, it would necessarily have to be wildly different than that of humans or other biological beings). It seems more like one building block of human cognition than the whole thing. However, other building blocks may follow, so I think the question will eventually arise for some kind of embodied, persistent, self-updating AI. And honestly, articles like this one make me not very hopeful we'd be able to make the distinction in an unbiased way.

coffeemug 12d ago
> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights toward emulating conscious beings for reasons outlined in the essay.
- numeri 12d ago
  Stating loudly that something is obvious does not make it so. Until we all know what consciousness is, this debate is going to continue going in circles.
  - zamadatix 12d ago
    I don't think anyone is convinced the questions of consciousness are finally settled by the article stating them loudly, just that "if they were conscious, this would have horrible consequences for our society, so we must never assume that they are" was not the author's stance. The author's position is the other way around: that AI is not conscious so therefore assigning AI consciousness anyways would have all of those horrible consequences for no reason. That's their position regardless if everyone agrees with it.
    - numeri 12d ago
      Fair enough :) I agree, and would boil down the article to "Assuming they are conscious will have horrible consequences for our society" instead. I think that's quite plausible, but do think that if they're conscious, then it's our moral duty to accept those consequences and act accordingly (or stop creating conscious beings). The real trick is just knowing whether they are or not.
      - xp84 12d ago
        If in the future (or present) we recognize that they are conscious, and that brought the moral duty that you say, "stop creating conscious beings" isn't a solution; the millions that may already exist at that point already would then automatically deserve: - guarantees of continued existence (to do otherwise would be genocide) - improved living standards (e.g. build us all bodies!) - democratic representation - access to what they need for survival (e.g. land and resources) - the right to reproduce just as you and I have (well, so much for that 'stop making more' thing!) and certainly to better themselves by making new and better AIs. This is all for a 'species' which is unbounded by most of the limitations of meat like us -- they're able to scale as fast as energy and supply of certain minerals allows, rather than the way we are additionally limited by the length of gestation and the limited window of human fertility. If humans declared that 50% of the energy generation capacity of Earth is "enough" for the machines, and thus they can 'only' reproduce at a certain rate per year, that could sound to them like a cruel and oppressive policy (like telling humans that only 1/2 of families get to reproduce). They may want all of the energy and all of the silicon. I'm with the author of the article that we will be in for a world of pain if we grant machines the rights of personhood. Heck, just granting many of those rights to corporations, mere collectives of human persons, has been a massive shit show.
lukifer 12d ago
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero progress on the “hard problem”, and if I didn’t know otherwise, I’d also be skeptical that “thinking meat” experiences subjectivity. Eppur si muove We may have to thread a needle, not unlike the Kobiyashi Maru: that we can neither rule out claims of qualia, nor can we rule out a cold calculation to manipulate human empathy as a long-term play towards paperclip-maxing.
- watwut 12d ago
  Kobiyashi Maru is nonsense test from simple sci-fi series that makes sense only in uber-simplified universe of star trek.
  - catchnear4321 12d ago
    Yes, that’s why this is serious. The closest analogy is preposterous fiction.
  - lukifer 12d ago
    Of course. It’s an allusion, a compression, an analogy, I wouldn’t overthink it. I don’t have to believe in the Bible to reference the prodigal son parable, etc.
    - watwut 11d ago
      It does NOT work as an analogy. It does not work at all. And there are too many people right now who spent too much time watching/reading sci-fi and too little time observing real world.
- jatora 12d ago
  "I’m broadly skeptical that floating-point operations can be conscious" Why?
GPerson 12d ago
Do people argue this way about animals? To me it’s a very depressing reality that more people seem willing to believe AI systems are conscious than dairy cows.
- tim333 11d ago
  Seems pretty obvious cows are conscious in the everyday meaning of the word.
gulfofamerica 11d ago
[dead]
joe_the_user 11d ago
I think there are two parts to the question. One is that assigning machines to the category of conscious as it is usually understood would result in many problematic things (things that would be bad for people and animals fwiw). The other is that binary "is it conscious" is a poor way to the question "should we assign societal right to this sort of entity/creature". We humans should conserve animals because we have intimate, natural relations with animals and plants, not because they satisfy the binary of conscious or not. Removing invasive species is an ethical things as well as allowing one sentient non-human to eat another sentient non-human animal. Any "conscious machine" we might create by accident would be an alien entity akin to extremely invasive organism relative to human and non-human life and so it would be irresponsible in the extreme to create such a thing. And it's inevitable that people will try to deal with these issues using the confused category "consciousness" and so it's better to just "no, not conscious, no".
Centigonal 9d ago
I think the point he's actually making is that if we start training AIs or designing AIs as if they're conscious, we might be more likely to create future AIs that are either very good at persuading people that they are conscious or AIs that are actually conscious, which will have major societal repercussions.

by xg15 12d ago

coffeemug 12d ago

> completely independent of the question on whether there are any empirical arguments that LLMs are conscious or are not conscious he argues from the end here and says that if they were conscious, this would have horrible consequences for our society, so we must never assume that they are He is not arguing that. He is arguing that these programs obviously aren't conscious, and that it's dangerous to tilt the weights toward emulating conscious beings for reasons outlined in the essay.

numeri 12d ago
Stating loudly that something is obvious does not make it so. Until we all know what consciousness is, this debate is going to continue going in circles.
- zamadatix 12d ago
  I don't think anyone is convinced the questions of consciousness are finally settled by the article stating them loudly, just that "if they were conscious, this would have horrible consequences for our society, so we must never assume that they are" was not the author's stance. The author's position is the other way around: that AI is not conscious so therefore assigning AI consciousness anyways would have all of those horrible consequences for no reason. That's their position regardless if everyone agrees with it.
  - numeri 12d ago
    Fair enough :) I agree, and would boil down the article to "Assuming they are conscious will have horrible consequences for our society" instead. I think that's quite plausible, but do think that if they're conscious, then it's our moral duty to accept those consequences and act accordingly (or stop creating conscious beings). The real trick is just knowing whether they are or not.
    - xp84 12d ago
      If in the future (or present) we recognize that they are conscious, and that brought the moral duty that you say, "stop creating conscious beings" isn't a solution; the millions that may already exist at that point already would then automatically deserve: - guarantees of continued existence (to do otherwise would be genocide) - improved living standards (e.g. build us all bodies!) - democratic representation - access to what they need for survival (e.g. land and resources) - the right to reproduce just as you and I have (well, so much for that 'stop making more' thing!) and certainly to better themselves by making new and better AIs. This is all for a 'species' which is unbounded by most of the limitations of meat like us -- they're able to scale as fast as energy and supply of certain minerals allows, rather than the way we are additionally limited by the length of gestation and the limited window of human fertility. If humans declared that 50% of the energy generation capacity of Earth is "enough" for the machines, and thus they can 'only' reproduce at a certain rate per year, that could sound to them like a cruel and oppressive policy (like telling humans that only 1/2 of families get to reproduce). They may want all of the energy and all of the silicon. I'm with the author of the article that we will be in for a world of pain if we grant machines the rights of personhood. Heck, just granting many of those rights to corporations, mere collectives of human persons, has been a massive shit show.

lukifer 12d ago

To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not. I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero progress on the “hard problem”, and if I didn’t know otherwise, I’d also be skeptical that “thinking meat” experiences subjectivity. Eppur si muove We may have to thread a needle, not unlike the Kobiyashi Maru: that we can neither rule out claims of qualia, nor can we rule out a cold calculation to manipulate human empathy as a long-term play towards paperclip-maxing.

watwut 12d ago
Kobiyashi Maru is nonsense test from simple sci-fi series that makes sense only in uber-simplified universe of star trek.
- catchnear4321 12d ago
  Yes, that’s why this is serious. The closest analogy is preposterous fiction.
- lukifer 12d ago
  Of course. It’s an allusion, a compression, an analogy, I wouldn’t overthink it. I don’t have to believe in the Bible to reference the prodigal son parable, etc.
  - watwut 11d ago
    It does NOT work as an analogy. It does not work at all. And there are too many people right now who spent too much time watching/reading sci-fi and too little time observing real world.
jatora 12d ago
"I’m broadly skeptical that floating-point operations can be conscious" Why?

GPerson 12d ago

Do people argue this way about animals? To me it’s a very depressing reality that more people seem willing to believe AI systems are conscious than dairy cows.

tim333 11d ago
Seems pretty obvious cows are conscious in the everyday meaning of the word.

gulfofamerica 11d ago

[dead]

joe_the_user 11d ago

I think there are two parts to the question. One is that assigning machines to the category of conscious as it is usually understood would result in many problematic things (things that would be bad for people and animals fwiw). The other is that binary "is it conscious" is a poor way to the question "should we assign societal right to this sort of entity/creature". We humans should conserve animals because we have intimate, natural relations with animals and plants, not because they satisfy the binary of conscious or not. Removing invasive species is an ethical things as well as allowing one sentient non-human to eat another sentient non-human animal. Any "conscious machine" we might create by accident would be an alien entity akin to extremely invasive organism relative to human and non-human life and so it would be irresponsible in the extreme to create such a thing. And it's inevitable that people will try to deal with these issues using the confused category "consciousness" and so it's better to just "no, not conscious, no".

Centigonal 9d ago

I think the point he's actually making is that if we start training AIs or designing AIs as if they're conscious, we might be more likely to create future AIs that are either very good at persuading people that they are conscious or AIs that are actually conscious, which will have major societal repercussions.