5 ms·

What is so absurd about the idea of ascribing consciousness to some potential future form of AI? We don't understand consciousness, but we've an idea that it's

What is so absurd about the idea of ascribing consciousness to some potential future form of AI? We don't understand consciousness, but we've an idea that it's an emergent phenomena. Given the recent papers about how computationally dense our DNA are, and the computing capacity of our brains, is it so unreasonable to assume that a sufficiently complex program running on non-organic matter could give rise to consciousness? The difference to me seems mostly one of computing mediums.

hobs 1y ago
Its not absurd to do it to a potential future AI, its absurd to do it to the face of the man in the moon.
iDont17 1y ago
Why make AI then if intelligence is electrical activity in a substrate is everywhere. We’re engineering nothing novel at great resource cost and appropriation of agency. Good job we made the models in the textbook “real”? Wasted engineering if it isn’t teaching us anything physics hadn’t already decades ago, then. Why bother with it? Edit: and AGI is impossible… ones light cone does not extend far enough to accurately learn; training on simulation is not sufficient to prepare for reality. Any machine we make will eventually get destroyed by some composition of space time we/the machine could not prepare for.
- tomrod 1y ago
  AGI may not be impossible. But next token prediction won't get us there.
  - vonneumannstan 1y ago
    It's actually really unclear that this is true. If you brought GPT-o3 back to 1990 I have a hard time believing the world wouldn't immediately consider it full AGI.
    - ACCount36 1y ago
      If you told a person from 1990 that in the year 2025, they have this thing, and described OpenAI's o3 - strengths, flaws and all? That person would say "yep, your sci-fi future of year 2025 has actual AI!" But if someone managed to actually make o3 in year 1990? Not in some abstact sci-fi future, but actually there, available broadly, as something you could access from your PC for a small fee? People would say "well, it's not ackhtually intelligent because..." Because people are incredibly stupid, and AI effect is incredibly powerful.
      - vonneumannstan 1y ago
        I'm very confident that if someone in 1990 used o3 they would be absolutely astonished and would not pull the 'well actually' thing you think they would.
        ACCount36 1y ago
        Nah, AI effect is far too powerful. Wishful thinking of this kind is simply irresistible. In real life, AI beating humans at chess didn't change the perception of machine intelligence for the better. It changed the perception of chess for the worse.
  - iDont17 1y ago
    [dead]
- vonneumannstan 1y ago
  This is a strange position. >Wasted engineering if it isn’t teaching us anything physics hadn’t already decades ago, then. Why bother with it? Why build cars and locomotives if they don't teach us anything Horses didn't... >and AGI is impossible… ones light cone does not extend far enough to accurately learn; training on simulation is not sufficient to prepare for reality. Any machine we make will eventually get destroyed by some composition of space time we/the machine could not prepare for. This could be applied to human's as well. Unless you believe in some extra-physical aspect of the human mind there is no reason to think it is different than a mind in silicon.
  - lolomg 1y ago
    [dead]
- iDont17 1y ago
  [flagged]
wat10000 1y ago
It is absurd, but consciousness is fundamentally absurd. Why would doing a bunch of basic arithmetic produce an entity that can experience things the way we do? There's no connection between those two concepts, aside from the fact that the one thing we know that can experience these things is also able to perform computation. But there's no indication that's anything other than a coincidence, or that the causation doesn't run in reverse, or from some common factor. You might as well say that electric fences give rise to cows. On the other hand, what else could it be? Consciousness is clearly in the brain. Normal biological processes don't seem to do it, it's something particular about the brain. So it's either something that only the brain does, which seems to be something at least vaguely like computation, or the brain is just a conduit and consciousness comes from something functionally like a "soul." Given the total lack of evidence for any such thing, and the total lack of any way to even rigorously define or conceptualize a "soul," this is also absurd. Consciousness just doesn't fit with anything else we know about the world. It's a fundamental mystery as things currently stand, and there's no explanation that makes a bit of sense yet.
- jrm4 1y ago
  Correct. Which is precisely why I have a problem with this idea as Anthropic is executing it; they might as well say "books and video games are conscious and we should be careful about their feelings."
- bbor 1y ago
  > Consciousness just doesn't fit with anything else we know about the world. It's a fundamental mystery as things currently stand, and there's no explanation that makes a bit of sense yet. Well put. I think there's one extremely solid explanation, though: it's a folk psychology concept with no bearing on actual truth. After all, could we ever build a machine that has all four humours? What about a machine that truly has a Qi field, instead of merely imitating one? Where are the Humours and Qi research institutes dedicated to this question?
  - wat10000 1y ago
    This sort of thing is why I seriously wonder if maybe some people have consciousness and some don't, rather than it being universal. My experience of consciousness is undeniable. There's no question of the concept just being made up. It's like if you said that hands are a folk concept with no bearing on actual truth. Even if I can't directly detect anyone else's hands, my own are unquestionably real to me. The only way someone could deny the existence of hands in general is if they didn't have any, but I definitely do.
bko 1y ago
Because they're numbers represented in digital form. In inference, you're doing simple math with those numbers. So what's alive, the numbers? Maybe the silicon holding the numbers? What if we print them out? Does the book become conscious? Even if you're a materialist, surely you think there is a difference between a human brain and a brain on a lab table. You take a dead persons brain, run some current through it and it jumps. Do you believe this equivalent to a living human being?
- bbor 1y ago
  Yes, if you take a dead persons brain and run current through it continuously so that it can direct a body and produce novel thoughts then that is equivalent to a living human being. Your brain is ultimately just numbers represented in neuronal form. What's conscious, the neurons?
  - pearlsontheroad 1y ago
    This level of materialism is soul-crushing :)
    - bbor 1y ago
      Thanks :) I always take the opportunity to crush souls when I can, since they don't exist in Reality nor Actuality. FWIW I'm a hardcore idealist, but in the way it was originally posed, not in the quasi-mystical way the Hegelians corrupted it into.
- ACCount36 1y ago
  What's the fundamental, inescapable difference between "numbers represented in digital form" and "jelly made of wet flesh crammed into an oversized monkey skull"? Why should one be more valid than the other?
  - const_cast 1y ago
    Because we decide which one is valid, and we are also part of the comparison being made. Yes, it's almost a perfect conflict of interest. Luckily that's fine, because we're us!
jrm4 1y ago
Nothing at all, if all you're doing is speculating within an academic context. This appears to be more than that; these are steps in the direction of law and policy.
Bigpet 1y ago
It reeks of some navel-gazing self-aggrandizing. I bet not even half the people doing the hand-wringing over how some matrix multiplications might feel are vegan or regularly spare a thought about how their or their companies consumption behavior indirectly leads to real human suffering. It's just so absurd how narrowly their focus on preventing suffering is. I almost can't imagine a world where their concern isn't coming from a disingenious place.
- JustinCS 1y ago
  I'm not highly concerned but I think there is merit in at least contemplating this problem. I believe that it would be better to reduce suffering in animals, but I am not vegan because the weight of my moral concern for animals does not outweigh my other priorities. I believe that it doesn't really matter whether consciousness comes from electronics or cells. If something seems identical to what we consider consciousness, I will likely believe it's better to not make that thing suffer. Though ultimately it's still just a consideration balanced among other concerns.
  - Bigpet 1y ago
    I too think there is merit in exploring to what degree conciousness can be approximated by or observed in computational systems of any kind. Including neural networks. But I just can't get over how fake and manipulative the framing of "AI welfare" or concern over suffering feels.
    - JustinCS 1y ago
      That's reasonable, I certainly believe that there are many fake and manipulative people who say what's best for their personal gain, perhaps even the majority. But I still think it's reasonable to imagine that there are some people are genuinely concerned about this.
AIPedant 1y ago
The problem is that current AI has nothing in common with consciousness as found in (say) cats and dogs, whatever that might be - no robot is even close to being as conscious as a cockroach - yet human consciousness seems to have great overlap with consciousness in nonhuman animals. The tiny fragment of human consciousness that appears to overlap with LLMs should be called something else, maybe “virtual sapience” as exhibited in LLMs. (The overarching difficulty here is we don’t know enough about consciousness/sentience/sapience to define them precisely.)
- tim333 1y ago
  This is common in arguments about AI. One person says why couldn't future AI be conscious, the next replies the problem is current LLMs aren't conscious. Future AI and current LLMs are not the same thing. Future AI no doubt will be able to be conscious for practical purposes.
- ben_w 1y ago
  > The problem is that current AI has nothing in common with consciousness as found in (say) cats and dogs, whatever that might be - no robot is even close to being as conscious as a cockroach - yet human consciousness seems to have great overlap with consciousness in nonhuman animals. Given we don't know what consciousness is — worse, there's around 40 different definitions, and I've seen people use "is a living biological organism" as one of them here on HN — we not only absolutely cannot say if the parts of our minds that give rise to it are, or are not, functionally equivalent to what LLMs do with attention. We also definitely cannot put them on a graph with cockroaches, which, like LLMs, may or may not have any at all. It's like we've got this word "weather", and some people are saying that only their own country really experiences "weather" because what they mean by the word is the most characteristic part of their local climate (windy moistness for Britain, sunny for the Sahara, snow for Siberia, rain for the Amazon etc.), and then we made a telescope and looked at Mars and are now arguing that Mars can't have real weather because there's no water there. And then someone else pipes in and is talking about "weather" to mean "daytime", because nobody ever says "what nice weather we're having" at night…
ahf8Aithaex7Nai 1y ago
We're doing the homunculus again. Whether you wank into a test tube and add a bit of soil and grass, or sew together parts of a corpse and connect them to electricity: so far, every prospect of fulfilling this dream has turned out to be a delusion. Why should it be any different with the latest manifestation, this time in computational form? The AGI drivel from people like Sam Altman is all about getting more VC money to push the scam a little further. ChatGPT is nothing more than a better Google. I'm happy to be proven wrong, but so far I see absolutely no potential for consciousness here. Perhaps we should first clarify whether dolphins and elephants are equipped with it before we do ChatGPT the honor.
- ben_w 1y ago
  > ChatGPT is nothing more than a better Google. It's not even trying to be a search engine. It can use them, but if that's what you think of them as, you're missing most of the cool stuff they can do. > Perhaps we should first clarify whether dolphins and elephants are equipped with it before we do ChatGPT the honor. Before either of those, we need to understand the question. Humanity uses the word "consciousness" for far too many things right now: some of these things, AI have had for a long time; others of which can never be had by anything, including humans.
  - ahf8Aithaex7Nai 1y ago
    > It's not even trying to be a search engine. I use search engines to get answers to questions and I use ChatGPT to get answers to questions. Except I can just ask ChatGPT like I would ask a person, whereas the search engine is more like a keyword search in a library catalog. > Before either of those, we need to understand the question. I don't think you can approach consciousness in this way. It's a bit like trying to derive Modus Ponens from propositional logic. I'm not sure if this comparison fits well, but I'm very skeptical that there is even a question to be explored that can be answered positively. Perhaps consciousness ultimately defies definition because it operates as a universal negation. It points to everything else and says: “I am different from that!” There are other terms that defy definition rather stubbornly: Love, art, intelligence, nature, happiness, beauty, ... So it's not that unusual. My skepticism towards conscious AI comes from the fact that I don't want to take the lack of definition as an opportunity to hastily interpret the undefined into all kinds of phenomena, even if it seems tempting. In the end, it is probably an illusion.

hobs 1y ago

Its not absurd to do it to a potential future AI, its absurd to do it to the face of the man in the moon.

iDont17 1y ago

Why make AI then if intelligence is electrical activity in a substrate is everywhere. We’re engineering nothing novel at great resource cost and appropriation of agency. Good job we made the models in the textbook “real”? Wasted engineering if it isn’t teaching us anything physics hadn’t already decades ago, then. Why bother with it? Edit: and AGI is impossible… ones light cone does not extend far enough to accurately learn; training on simulation is not sufficient to prepare for reality. Any machine we make will eventually get destroyed by some composition of space time we/the machine could not prepare for.

tomrod 1y ago
AGI may not be impossible. But next token prediction won't get us there.
- vonneumannstan 1y ago
  It's actually really unclear that this is true. If you brought GPT-o3 back to 1990 I have a hard time believing the world wouldn't immediately consider it full AGI.
  - ACCount36 1y ago
    If you told a person from 1990 that in the year 2025, they have this thing, and described OpenAI's o3 - strengths, flaws and all? That person would say "yep, your sci-fi future of year 2025 has actual AI!" But if someone managed to actually make o3 in year 1990? Not in some abstact sci-fi future, but actually there, available broadly, as something you could access from your PC for a small fee? People would say "well, it's not ackhtually intelligent because..." Because people are incredibly stupid, and AI effect is incredibly powerful.
    - vonneumannstan 1y ago
      I'm very confident that if someone in 1990 used o3 they would be absolutely astonished and would not pull the 'well actually' thing you think they would.
      - ACCount36 1y ago
        Nah, AI effect is far too powerful. Wishful thinking of this kind is simply irresistible. In real life, AI beating humans at chess didn't change the perception of machine intelligence for the better. It changed the perception of chess for the worse.
- iDont17 1y ago
  [dead]
vonneumannstan 1y ago
This is a strange position. >Wasted engineering if it isn’t teaching us anything physics hadn’t already decades ago, then. Why bother with it? Why build cars and locomotives if they don't teach us anything Horses didn't... >and AGI is impossible… ones light cone does not extend far enough to accurately learn; training on simulation is not sufficient to prepare for reality. Any machine we make will eventually get destroyed by some composition of space time we/the machine could not prepare for. This could be applied to human's as well. Unless you believe in some extra-physical aspect of the human mind there is no reason to think it is different than a mind in silicon.
- lolomg 1y ago
  [dead]
iDont17 1y ago
[flagged]

wat10000 1y ago

It is absurd, but consciousness is fundamentally absurd. Why would doing a bunch of basic arithmetic produce an entity that can experience things the way we do? There's no connection between those two concepts, aside from the fact that the one thing we know that can experience these things is also able to perform computation. But there's no indication that's anything other than a coincidence, or that the causation doesn't run in reverse, or from some common factor. You might as well say that electric fences give rise to cows. On the other hand, what else could it be? Consciousness is clearly in the brain. Normal biological processes don't seem to do it, it's something particular about the brain. So it's either something that only the brain does, which seems to be something at least vaguely like computation, or the brain is just a conduit and consciousness comes from something functionally like a "soul." Given the total lack of evidence for any such thing, and the total lack of any way to even rigorously define or conceptualize a "soul," this is also absurd. Consciousness just doesn't fit with anything else we know about the world. It's a fundamental mystery as things currently stand, and there's no explanation that makes a bit of sense yet.

jrm4 1y ago
Correct. Which is precisely why I have a problem with this idea as Anthropic is executing it; they might as well say "books and video games are conscious and we should be careful about their feelings."
bbor 1y ago
> Consciousness just doesn't fit with anything else we know about the world. It's a fundamental mystery as things currently stand, and there's no explanation that makes a bit of sense yet. Well put. I think there's one extremely solid explanation, though: it's a folk psychology concept with no bearing on actual truth. After all, could we ever build a machine that has all four humours? What about a machine that truly has a Qi field, instead of merely imitating one? Where are the Humours and Qi research institutes dedicated to this question?
- wat10000 1y ago
  This sort of thing is why I seriously wonder if maybe some people have consciousness and some don't, rather than it being universal. My experience of consciousness is undeniable. There's no question of the concept just being made up. It's like if you said that hands are a folk concept with no bearing on actual truth. Even if I can't directly detect anyone else's hands, my own are unquestionably real to me. The only way someone could deny the existence of hands in general is if they didn't have any, but I definitely do.

bko 1y ago

Because they're numbers represented in digital form. In inference, you're doing simple math with those numbers. So what's alive, the numbers? Maybe the silicon holding the numbers? What if we print them out? Does the book become conscious? Even if you're a materialist, surely you think there is a difference between a human brain and a brain on a lab table. You take a dead persons brain, run some current through it and it jumps. Do you believe this equivalent to a living human being?

bbor 1y ago
Yes, if you take a dead persons brain and run current through it continuously so that it can direct a body and produce novel thoughts then that is equivalent to a living human being. Your brain is ultimately just numbers represented in neuronal form. What's conscious, the neurons?
- pearlsontheroad 1y ago
  This level of materialism is soul-crushing :)
  - bbor 1y ago
    Thanks :) I always take the opportunity to crush souls when I can, since they don't exist in Reality nor Actuality. FWIW I'm a hardcore idealist, but in the way it was originally posed, not in the quasi-mystical way the Hegelians corrupted it into.
ACCount36 1y ago
What's the fundamental, inescapable difference between "numbers represented in digital form" and "jelly made of wet flesh crammed into an oversized monkey skull"? Why should one be more valid than the other?
- const_cast 1y ago
  Because we decide which one is valid, and we are also part of the comparison being made. Yes, it's almost a perfect conflict of interest. Luckily that's fine, because we're us!

jrm4 1y ago

Nothing at all, if all you're doing is speculating within an academic context. This appears to be more than that; these are steps in the direction of law and policy.

Bigpet 1y ago

It reeks of some navel-gazing self-aggrandizing. I bet not even half the people doing the hand-wringing over how some matrix multiplications might feel are vegan or regularly spare a thought about how their or their companies consumption behavior indirectly leads to real human suffering. It's just so absurd how narrowly their focus on preventing suffering is. I almost can't imagine a world where their concern isn't coming from a disingenious place.

JustinCS 1y ago
I'm not highly concerned but I think there is merit in at least contemplating this problem. I believe that it would be better to reduce suffering in animals, but I am not vegan because the weight of my moral concern for animals does not outweigh my other priorities. I believe that it doesn't really matter whether consciousness comes from electronics or cells. If something seems identical to what we consider consciousness, I will likely believe it's better to not make that thing suffer. Though ultimately it's still just a consideration balanced among other concerns.
- Bigpet 1y ago
  I too think there is merit in exploring to what degree conciousness can be approximated by or observed in computational systems of any kind. Including neural networks. But I just can't get over how fake and manipulative the framing of "AI welfare" or concern over suffering feels.
  - JustinCS 1y ago
    That's reasonable, I certainly believe that there are many fake and manipulative people who say what's best for their personal gain, perhaps even the majority. But I still think it's reasonable to imagine that there are some people are genuinely concerned about this.

AIPedant 1y ago

The problem is that current AI has nothing in common with consciousness as found in (say) cats and dogs, whatever that might be - no robot is even close to being as conscious as a cockroach - yet human consciousness seems to have great overlap with consciousness in nonhuman animals. The tiny fragment of human consciousness that appears to overlap with LLMs should be called something else, maybe “virtual sapience” as exhibited in LLMs. (The overarching difficulty here is we don’t know enough about consciousness/sentience/sapience to define them precisely.)

tim333 1y ago
This is common in arguments about AI. One person says why couldn't future AI be conscious, the next replies the problem is current LLMs aren't conscious. Future AI and current LLMs are not the same thing. Future AI no doubt will be able to be conscious for practical purposes.
ben_w 1y ago
> The problem is that current AI has nothing in common with consciousness as found in (say) cats and dogs, whatever that might be - no robot is even close to being as conscious as a cockroach - yet human consciousness seems to have great overlap with consciousness in nonhuman animals. Given we don't know what consciousness is — worse, there's around 40 different definitions, and I've seen people use "is a living biological organism" as one of them here on HN — we not only absolutely cannot say if the parts of our minds that give rise to it are, or are not, functionally equivalent to what LLMs do with attention. We also definitely cannot put them on a graph with cockroaches, which, like LLMs, may or may not have any at all. It's like we've got this word "weather", and some people are saying that only their own country really experiences "weather" because what they mean by the word is the most characteristic part of their local climate (windy moistness for Britain, sunny for the Sahara, snow for Siberia, rain for the Amazon etc.), and then we made a telescope and looked at Mars and are now arguing that Mars can't have real weather because there's no water there. And then someone else pipes in and is talking about "weather" to mean "daytime", because nobody ever says "what nice weather we're having" at night…

ahf8Aithaex7Nai 1y ago

We're doing the homunculus again. Whether you wank into a test tube and add a bit of soil and grass, or sew together parts of a corpse and connect them to electricity: so far, every prospect of fulfilling this dream has turned out to be a delusion. Why should it be any different with the latest manifestation, this time in computational form? The AGI drivel from people like Sam Altman is all about getting more VC money to push the scam a little further. ChatGPT is nothing more than a better Google. I'm happy to be proven wrong, but so far I see absolutely no potential for consciousness here. Perhaps we should first clarify whether dolphins and elephants are equipped with it before we do ChatGPT the honor.

ben_w 1y ago
> ChatGPT is nothing more than a better Google. It's not even trying to be a search engine. It can use them, but if that's what you think of them as, you're missing most of the cool stuff they can do. > Perhaps we should first clarify whether dolphins and elephants are equipped with it before we do ChatGPT the honor. Before either of those, we need to understand the question. Humanity uses the word "consciousness" for far too many things right now: some of these things, AI have had for a long time; others of which can never be had by anything, including humans.
- ahf8Aithaex7Nai 1y ago
  > It's not even trying to be a search engine. I use search engines to get answers to questions and I use ChatGPT to get answers to questions. Except I can just ask ChatGPT like I would ask a person, whereas the search engine is more like a keyword search in a library catalog. > Before either of those, we need to understand the question. I don't think you can approach consciousness in this way. It's a bit like trying to derive Modus Ponens from propositional logic. I'm not sure if this comparison fits well, but I'm very skeptical that there is even a question to be explored that can be answered positively. Perhaps consciousness ultimately defies definition because it operates as a universal negation. It points to everything else and says: “I am different from that!” There are other terms that defy definition rather stubbornly: Love, art, intelligence, nature, happiness, beauty, ... So it's not that unusual. My skepticism towards conscious AI comes from the fact that I don't want to take the lack of definition as an opportunity to hastily interpret the undefined into all kinds of phenomena, even if it seems tempting. In the end, it is probably an illusion.