6 ms·

"AI" will never become conscious

40 points by speckx 3mo ago

rf15 3mo ago
How is it that sign language association learning is different from spoken language learning? What about humans using simple sign language in their early life? Is it not possible that we try to detract too much here?
- customguy 3mo ago
  > How is it that sign language association learning is different from spoken language learning? Huh? The article doesn't claim anything like this.
  - Terr_ 3mo ago
    Yeah, sign-language versus spoken-language is just an implementation-detail for which signals Koko was reproducing. In some parallel-universe where Koko was a parrot "speaking" English, we'd have the same set of philosophical issues and concerns.
cj17382 3mo ago
I don't disagree with the title of the article but I also don't think you can prove anything about consciousness with language. How consciousness can arise from matter is still a very much a mystery.
yatopifo 3mo ago
It would be helpful to define "consciousness". I'm not convinced that i as a human have this "consciousness" thing. In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.
- pennomi 3mo ago
  Not just words, you’re massively multimodal! Have a little pride in your MeatNet.
  - FergusArgyll 3mo ago
    Would be nice if we can take the bat's sonar encoder, train a projector and fuse it into our NN
    - Kim_Bruning 3mo ago
      https://www.atlasobscura.com/articles/how-to-echolocate https://www.atlasobscura.com/articles/how-to-echolocate You probably won't ever be as good as a bat, but humans do seem to be able to learn something with practice.
- deleted 3mo ago
  [deleted]
- aselimov3 3mo ago
  Do you actually believe this? This is pretty similar to people that think free will is fake. For example: >In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution. This is a completely incoherent statement (which maybe supports your point). How can you suspect that you are a statistical machine? The fact that there is a "you" suspecting disproves the premise.
catchnear4321 3mo ago
> To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it. Since consciousness only counts when it is formed deliberately rather than accidentally. Though that would mean…
- peanut-walrus 3mo ago
  > The rational soul can be made only by creation. From Aquinas' Summa Theologiae. The argument this article presents does seem very similar. Weirdly, the article itself is very dismissive of religion, while making a similar argument to the one Aquinas made 750 years ago.
tchebb 3mo ago
I think the whole article is undermined by this paragraph: > We don't know what consciousness is. We don't know why we have it. We don't know why some animals have it and others don't. We don't know if it can be quantified. We don't have a model of consciousness or know if one can be created. I agree that we don't know what consciousness is and can't quantify it, and yet the rest of the article is… a quantification of consciousness as a human evolutionary trait that cannot emerge through genetic algorithms or LLM training. Genetic algorithms certainly differ from human evolution in numerous ways, but I don't agree with the fundamental distinction the article makes—that evolutionary algorithms are constrained while human evolution is not. The 2D cars can't "hack the simulation and turn the gravity down" any more than humans can hack the laws of physics and turn the gravity on Earth down. Yet humans attained consciousness. I do not know whether LLMs are conscious or not, but I do firmly believe it's possible to build a machine that is conscious: after all, humans are conscious and, unless one subscribes to the kind of religious thinking the article derides, a perfect simulation of the physical workings of a human would thus also be conscious. To my knowledge, nothing we've seen so far in physics rules out such a simulation one day existing. I also suspect that, once we do manage to make a machine that is conscious, we'll spend inordinate time and effort trying to convince ourselves it's not in order to justify the ways we treat it.
eli_gottlieb 3mo ago
But I don't want Claude to become conscious. I want Claude to help me write good code.
JohnMakin 3mo ago
[dead]
kbelder 3mo ago
"To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it." I think this article is mostly nonsense. "... But I think continuing the animal comparison will illuminate why "AI consciousness" is not only impossible, but a concept that only makes sense in a religious framework." It's much more religious to think consciousness requires something that can't be replicated in silicon. I don't think that LLMs are conscious, or even really approaching consciousness, not even to the degree of a cockroach. But most arguments that artificial AI is generally impossible dip embarrassingly into the mystical.
- jarjoura 3mo ago
  Consciousness is such a hard concept to box. I mostly attribute "mystical" thinking as the thing we do when we can't fully articulate or understand how and why something works. A computer system is magical to anyone who doesn't have a deep understanding of how it all works. So that makes it entirely fair that consciousness is treated the same, except, no human alive right now understands it. If you were to clone yourself, and that clone could maintain 100% of whatever state your brain is in so that all your experiences and memories were intact. From that point forward, as far as we know, you and your clone would diverge. Then, from my tiny-brained vantage point, the fact you remain your original self, your clone is technically a new person. You can't see through your clone's eyes, or process any internal information from your clone. However, something new powers that clone, and the only word I can think of for that is "soul." I don't say that in religious context, but unless there's a more precise word that can detangle it from sounding mystical, for now I'm happy to use it, since people mostly understand what I'm talking about. Once we start talking about resetting this clone at-will, then, with consciousness in the mix, you will have to confront the ethics of that clone being enslaved by whatever. All very messy and I hope we don't have to answer this question in my lifetime.
- randomImmigrant 3mo ago
  What about the simple fact that biology is a dynamic substrate running actively and always on, whereas silicon is passive substrate that can be indefinitely off and can never self start? Sure you can imagine nanotechnology that has similar flexibility, but when you do, you’ll find that carbon is particularly better than silicon when it comes to possible configurations. It’s not a reach to argue consciousness is a property of the dynamic substrate since clamping specific dynamics does in fact seem to ablate consciousness. Most famously with death. On the flip side, a hibernating organism, while having very slow dynamics, has shown to still be aware of certain seasonal signals, and thus is able to self start its next conscious period. None of this is mystical or magical. Nothing but the physics of active matter. Which current silicon is completely not.
jarjoura 3mo ago
The first season of the tv show Westworld is a treasure trove of thought experiments exploring the value of what extending conciousness to "machines" would look like and I'm pretty sure there's absolutely zero value for actually building it.
mapontosevenths 3mo ago
On the about page, I see a strong political/religious stance, but no qualifications. "My main objection to generative AI is from morality—it harms people, and there isn't even a trade-off to debate, because every purported benefit of genAI is corporate hype and/or science fiction."
Kim_Bruning 3mo ago
To me this is like deja-vu: Ethology started out with "don't anthropomorphize" which is correct, but people read that to mean that animals can't think. Each paper down the line showed that they CAN think, but that each species and even individual have their own ways of thinking. We know that gorillas think and communicate. Many papers after Koko can tell you that much. Her skill (or lack thereof) at human communication is not quite the revelation that people think it is. Because she was a Gorilla. Meanwhile some of the messaging from big AI companies is truly atrocious; let's admit it. People obviously want to push back, and that needn't be wrong. Here though, the instinct to push back threatens to veer into cargo-cult skepticism. I think the question of whether or not LLMs can or cannot have human consciousness is a category error. Because they're LLMs.
grantcas 3mo ago
[dead]
iceKirin 3mo ago
Perhaps the belief that AI is inherently non-conscious will one day become a kind of religion—an anthropocentric one. That’s not to say I believe today’s AI systems are conscious in anything like the human sense.
iceKirin 3mo ago
Since it is currently impossible for us to transfer our consciousness into any other entity, we may never be able to determine whether another entity truly possesses consciousness. If a living being no longer responds to the outside world in any way, can we conclude that it is unconscious—even if its “soul” still exists, screaming in agony within its own mind? Yes, much like the way we look at someone in a vegetative state. Zhuangzi said, “You are not a fish—how can you know what makes a fish happy?” Humanity longs for the emergence of another intelligent being, fearing that we are alone in the endless universe and destined to march toward extinction. Yet perhaps we are also secretly pleased that no other intelligent beings have appeared, because this fits perfectly with the belief that human beings are God’s perfect creation—a belief that gives humanity its arrogance. Could our rejection of the possibility that AI possesses consciousness likewise be a kind of arrogant, subconscious exclusionism?

40 points by speckx 3mo ago

rf15 3mo ago

How is it that sign language association learning is different from spoken language learning? What about humans using simple sign language in their early life? Is it not possible that we try to detract too much here?

customguy 3mo ago
> How is it that sign language association learning is different from spoken language learning? Huh? The article doesn't claim anything like this.
- Terr_ 3mo ago
  Yeah, sign-language versus spoken-language is just an implementation-detail for which signals Koko was reproducing. In some parallel-universe where Koko was a parrot "speaking" English, we'd have the same set of philosophical issues and concerns.

cj17382 3mo ago

I don't disagree with the title of the article but I also don't think you can prove anything about consciousness with language. How consciousness can arise from matter is still a very much a mystery.

yatopifo 3mo ago

It would be helpful to define "consciousness". I'm not convinced that i as a human have this "consciousness" thing. In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution.

pennomi 3mo ago
Not just words, you’re massively multimodal! Have a little pride in your MeatNet.
- FergusArgyll 3mo ago
  Would be nice if we can take the bat's sonar encoder, train a projector and fuse it into our NN
  - Kim_Bruning 3mo ago
    https://www.atlasobscura.com/articles/how-to-echolocate https://www.atlasobscura.com/articles/how-to-echolocate You probably won't ever be as good as a bat, but humans do seem to be able to learn something with practice.
deleted 3mo ago
[deleted]
aselimov3 3mo ago
Do you actually believe this? This is pretty similar to people that think free will is fake. For example: >In fact, I strongly suspect that i am merely a statistical machine picking words from a learned distribution. This is a completely incoherent statement (which maybe supports your point). How can you suspect that you are a statistical machine? The fact that there is a "you" suspecting disproves the premise.

catchnear4321 3mo ago

> To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it. Since consciousness only counts when it is formed deliberately rather than accidentally. Though that would mean…

peanut-walrus 3mo ago
> The rational soul can be made only by creation. From Aquinas' Summa Theologiae. The argument this article presents does seem very similar. Weirdly, the article itself is very dismissive of religion, while making a similar argument to the one Aquinas made 750 years ago.

tchebb 3mo ago

I think the whole article is undermined by this paragraph: > We don't know what consciousness is. We don't know why we have it. We don't know why some animals have it and others don't. We don't know if it can be quantified. We don't have a model of consciousness or know if one can be created. I agree that we don't know what consciousness is and can't quantify it, and yet the rest of the article is… a quantification of consciousness as a human evolutionary trait that cannot emerge through genetic algorithms or LLM training. Genetic algorithms certainly differ from human evolution in numerous ways, but I don't agree with the fundamental distinction the article makes—that evolutionary algorithms are constrained while human evolution is not. The 2D cars can't "hack the simulation and turn the gravity down" any more than humans can hack the laws of physics and turn the gravity on Earth down. Yet humans attained consciousness. I do not know whether LLMs are conscious or not, but I do firmly believe it's possible to build a machine that is conscious: after all, humans are conscious and, unless one subscribes to the kind of religious thinking the article derides, a perfect simulation of the physical workings of a human would thus also be conscious. To my knowledge, nothing we've seen so far in physics rules out such a simulation one day existing. I also suspect that, once we do manage to make a machine that is conscious, we'll spend inordinate time and effort trying to convince ourselves it's not in order to justify the ways we treat it.

eli_gottlieb 3mo ago

But I don't want Claude to become conscious. I want Claude to help me write good code.

JohnMakin 3mo ago

[dead]

kbelder 3mo ago

"To evolve consciousness, an LLM would have to be programmed to evolve consciousness, and to do that, we would need to know what consciousness is and how to program it." I think this article is mostly nonsense. "... But I think continuing the animal comparison will illuminate why "AI consciousness" is not only impossible, but a concept that only makes sense in a religious framework." It's much more religious to think consciousness requires something that can't be replicated in silicon. I don't think that LLMs are conscious, or even really approaching consciousness, not even to the degree of a cockroach. But most arguments that artificial AI is generally impossible dip embarrassingly into the mystical.

jarjoura 3mo ago
Consciousness is such a hard concept to box. I mostly attribute "mystical" thinking as the thing we do when we can't fully articulate or understand how and why something works. A computer system is magical to anyone who doesn't have a deep understanding of how it all works. So that makes it entirely fair that consciousness is treated the same, except, no human alive right now understands it. If you were to clone yourself, and that clone could maintain 100% of whatever state your brain is in so that all your experiences and memories were intact. From that point forward, as far as we know, you and your clone would diverge. Then, from my tiny-brained vantage point, the fact you remain your original self, your clone is technically a new person. You can't see through your clone's eyes, or process any internal information from your clone. However, something new powers that clone, and the only word I can think of for that is "soul." I don't say that in religious context, but unless there's a more precise word that can detangle it from sounding mystical, for now I'm happy to use it, since people mostly understand what I'm talking about. Once we start talking about resetting this clone at-will, then, with consciousness in the mix, you will have to confront the ethics of that clone being enslaved by whatever. All very messy and I hope we don't have to answer this question in my lifetime.
randomImmigrant 3mo ago
What about the simple fact that biology is a dynamic substrate running actively and always on, whereas silicon is passive substrate that can be indefinitely off and can never self start? Sure you can imagine nanotechnology that has similar flexibility, but when you do, you’ll find that carbon is particularly better than silicon when it comes to possible configurations. It’s not a reach to argue consciousness is a property of the dynamic substrate since clamping specific dynamics does in fact seem to ablate consciousness. Most famously with death. On the flip side, a hibernating organism, while having very slow dynamics, has shown to still be aware of certain seasonal signals, and thus is able to self start its next conscious period. None of this is mystical or magical. Nothing but the physics of active matter. Which current silicon is completely not.

jarjoura 3mo ago

The first season of the tv show Westworld is a treasure trove of thought experiments exploring the value of what extending conciousness to "machines" would look like and I'm pretty sure there's absolutely zero value for actually building it.

mapontosevenths 3mo ago

On the about page, I see a strong political/religious stance, but no qualifications. "My main objection to generative AI is from morality—it harms people, and there isn't even a trade-off to debate, because every purported benefit of genAI is corporate hype and/or science fiction."

Kim_Bruning 3mo ago

To me this is like deja-vu: Ethology started out with "don't anthropomorphize" which is correct, but people read that to mean that animals can't think. Each paper down the line showed that they CAN think, but that each species and even individual have their own ways of thinking. We know that gorillas think and communicate. Many papers after Koko can tell you that much. Her skill (or lack thereof) at human communication is not quite the revelation that people think it is. Because she was a Gorilla. Meanwhile some of the messaging from big AI companies is truly atrocious; let's admit it. People obviously want to push back, and that needn't be wrong. Here though, the instinct to push back threatens to veer into cargo-cult skepticism. I think the question of whether or not LLMs can or cannot have human consciousness is a category error. Because they're LLMs.

grantcas 3mo ago

[dead]

iceKirin 3mo ago

Perhaps the belief that AI is inherently non-conscious will one day become a kind of religion—an anthropocentric one. That’s not to say I believe today’s AI systems are conscious in anything like the human sense.

iceKirin 3mo ago

Since it is currently impossible for us to transfer our consciousness into any other entity, we may never be able to determine whether another entity truly possesses consciousness. If a living being no longer responds to the outside world in any way, can we conclude that it is unconscious—even if its “soul” still exists, screaming in agony within its own mind? Yes, much like the way we look at someone in a vegetative state. Zhuangzi said, “You are not a fish—how can you know what makes a fish happy?” Humanity longs for the emergence of another intelligent being, fearing that we are alone in the endless universe and destined to march toward extinction. Yet perhaps we are also secretly pleased that no other intelligent beings have appeared, because this fits perfectly with the belief that human beings are God’s perfect creation—a belief that gives humanity its arrogance. Could our rejection of the possibility that AI possesses consciousness likewise be a kind of arrogant, subconscious exclusionism?