6 ms·

Right here on HN, where you'd think that most people would be able to think better, there's no shortage of AI fanboys who seriously try to frame the idea that L

Right here on HN, where you'd think that most people would be able to think better, there's no shortage of AI fanboys who seriously try to frame the idea that LLMs may be conscious, and that we can't really be sure if we're conscious, since, you know, science doesn't know what causes consciousness. I guess me feeling conscious, and thus directing much of my life and activity around that notion is just a big trick, and i'm no different from ChatGPT, only inferior and with lower electrical needs.

IshKebab 1y ago
This is a bad faith representation of what people are actually saying which is: 1. As far as we know there is nothing non-physical in the brain (a soul or magical quantum microtubules or whatever). Everything that happens in your head obeys the laws of physics. 2. We definitely are conscious. 3. Therefore consciousness can arise from purely physical processes. 4. Any physical process can be computed. And biological thought appears as far as we can see to simply be an extraordinarily complex computation. 5. Therefore it is possible for a sufficiently fast & appropriately programmed computer to be conscious. 6. LLMs bear many resemblances to human thinking (even though they're obviously different in many ways too), and we've established that computers can be conscious, so you can't trivially rule out LLMs being conscious. At least for now, I don't think anyone sane says today's LLMs are actually conscious. (Though consciousness is clearly a continuum... so maybe they're as conscious as a worm or whatever.) The main point is that people trivially dismissing AI consciousness or saying "it can't think" because it's only matrix multiplications are definitely wrong. It might not be conscious because consciousness requires more complexity, or maybe it requires some algorithmic differences (e.g. on-line learning). But it definitely isn't not-conscious just because it is maths or running on a computer or deterministic or ...
- Veen 1y ago
  The "as far as we know" in premise one is hand-waving the critical point. We don't know how or if conscious arises from purely physical systems. It may be physical, but inadequately replicated by computer systems (it may be non-computable). It may not be physical in the way we use that concept. We just don't know. And the fact that we don't know puts the conclusion in doubt. Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness, even if it mimics some of the qualities we associate with consciousness.
  - stevenhuang 1y ago
    Rather, the fact we don't know means... > you can't trivially rule out LLMs being conscious. Which is the actual conclusion. That's all. The one trying to read more into it by claiming a stronger conclusion than warranted (claiming this is all "doubtful") is you. > Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness This is incoherent.
    - Veen 1y ago
      The actual conclusion: "We've established that computers can be conscious, so LLMs might be" The foregoing argument does not establish that computers can be conscious. It assumes consciousness arises from purely physical interactions AND that any physical process can be computed AND that computation can produce consciousness. It provides no reason to believe any of those things, and therefore no reason to believe computers or LLMs can be conscious. Once you remove the unjustifed assumptions, it boils down to this: LLMs can do something that looks a bit like what some conscious beings do so they might be conscious. The final paragraph is not incoherent.
      - stevenhuang 1y ago
        > It assumes consciousness arises from purely physical interactions AND that any physical process can be computed AND that computation can produce consciousness Yes, and with these assumptions, then "We've established that computers can be conscious, so LLMs might be". Again that is all that is being claimed. The one insisting the argument is anything more persuasive than that is you. You seem hung up on this claim, but need I remind you it's only an existence proof, whether it's actually possible/probable/likely is an entirely different claim, and is not what is being claimed. > Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness > If consciousness arises from a physical system, a physical system cannot produce consciousness > If A not A This argument is a contradiction, it's incoherent. If you're trying to draw a distinction between physical systems and computation on physical systems, well that's a moot point as every tested theory in physics uses mathematics that is computable. Perhaps reality is not computable. If so then yes, that would mean computation alone will not produce the same kind of consciousness we have. That leaves open other kinds of consciousness, but that's another discussion.
- rsanheim 1y ago
  > Any physical process can be computed Um, really? How so? Show me the computations that model gut biome. Or pain, or the immune system, or how plants communicate, or how the big bang happened (or didn’t). And now define “physical” wrt things like pain, or sensation, or the body, or consciousness. We know so little about so much! it’s amazing when people speak with such certainty about things like consciousness and computation. I don’t disagree with your main point. I would just say we don’t know very much about our own consciousness and mindbody. And humans have been studying that at least a few thousand years. Not everyone on HN or who works with or studies AI believes AGI is just the next level on some algorithm or scaling level we haven’t unlocked yet.
  - jstanley 1y ago
    > Show me the computations Just because computations exist that model a physical process to an arbitrary degree of accuracy doesn't mean we know what those computations are.
    - tsimionescu 1y ago
      But the reality is that we don't know if many of those things are computable. We just think it's likely they are, since the models we have of very simplified versions of them, or of very approximate versions, are computable. But no one can use a physical model or computer to precisely compute, say, the fall of sand (where the exact position and momentum of every grain of sand is precisely known) - so we have no actual proof that the actual physical process is computable. And, in fact, our best fundamental physical theory is not fully computable, at least in some sense - quantum measurement is not a deterministic process, so it's not exactly computable.
      - stevenhuang 1y ago
        QM is computable on classical turing machines, it is just not efficient.
        tsimionescu 1y ago
- yusina 1y ago
  These discussions remind me an awful lot of discussions between my religios and my atheist and my agnostic friends. Reglious friend: Well there must be a god, I can feel him, and how else would all this marvellous reality around us have come to existence? Atheist friend: How can you claim that? Where is the evidence? There could be many other explanations, we just don't know, and because of that it doesn't make sense to postulate the existence of a god. And besides, it's just a marketing stunt of the powerful to stay in power. Agnostic friend: Why does any of this matter? Nature is beautiful, sometimes cruel, I'm glad I'm alive, and whether there is a god or not, I don't know or really care, it's insubstantial to my life. That's exactly how the discussions around LLM consciousness or intelligence or sentience always go if you put enough tech folks into the conversation.
deleted 1y ago
[deleted]
imiric 1y ago
> Right here on HN, where you'd think that most people would be able to think better, there's no shortage of AI fanboys who seriously try to frame the idea that LLMs may be conscious Why does this surprise you? This is a community hosted by a Silicon Valley venture capital firm, and many of its members are either employed by or own companies in the AI industry. They are personally invested in pushing the narrative being criticized here, so the counter arguments are expected.
grantcas 1y ago
[dead]

IshKebab 1y ago

This is a bad faith representation of what people are actually saying which is: 1. As far as we know there is nothing non-physical in the brain (a soul or magical quantum microtubules or whatever). Everything that happens in your head obeys the laws of physics. 2. We definitely are conscious. 3. Therefore consciousness can arise from purely physical processes. 4. Any physical process can be computed. And biological thought appears as far as we can see to simply be an extraordinarily complex computation. 5. Therefore it is possible for a sufficiently fast & appropriately programmed computer to be conscious. 6. LLMs bear many resemblances to human thinking (even though they're obviously different in many ways too), and we've established that computers can be conscious, so you can't trivially rule out LLMs being conscious. At least for now, I don't think anyone sane says today's LLMs are actually conscious. (Though consciousness is clearly a continuum... so maybe they're as conscious as a worm or whatever.) The main point is that people trivially dismissing AI consciousness or saying "it can't think" because it's only matrix multiplications are definitely wrong. It might not be conscious because consciousness requires more complexity, or maybe it requires some algorithmic differences (e.g. on-line learning). But it definitely isn't not-conscious just because it is maths or running on a computer or deterministic or ...

Veen 1y ago
The "as far as we know" in premise one is hand-waving the critical point. We don't know how or if conscious arises from purely physical systems. It may be physical, but inadequately replicated by computer systems (it may be non-computable). It may not be physical in the way we use that concept. We just don't know. And the fact that we don't know puts the conclusion in doubt. Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness, even if it mimics some of the qualities we associate with consciousness.
- stevenhuang 1y ago
  Rather, the fact we don't know means... > you can't trivially rule out LLMs being conscious. Which is the actual conclusion. That's all. The one trying to read more into it by claiming a stronger conclusion than warranted (claiming this is all "doubtful") is you. > Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness This is incoherent.
  - Veen 1y ago
    The actual conclusion: "We've established that computers can be conscious, so LLMs might be" The foregoing argument does not establish that computers can be conscious. It assumes consciousness arises from purely physical interactions AND that any physical process can be computed AND that computation can produce consciousness. It provides no reason to believe any of those things, and therefore no reason to believe computers or LLMs can be conscious. Once you remove the unjustifed assumptions, it boils down to this: LLMs can do something that looks a bit like what some conscious beings do so they might be conscious. The final paragraph is not incoherent.
    - stevenhuang 1y ago
      > It assumes consciousness arises from purely physical interactions AND that any physical process can be computed AND that computation can produce consciousness Yes, and with these assumptions, then "We've established that computers can be conscious, so LLMs might be". Again that is all that is being claimed. The one insisting the argument is anything more persuasive than that is you. You seem hung up on this claim, but need I remind you it's only an existence proof, whether it's actually possible/probable/likely is an entirely different claim, and is not what is being claimed. > Even if consciousness arises from a physical system, it doesn't follow that computation on a physical substrate can produce consciousness > If consciousness arises from a physical system, a physical system cannot produce consciousness > If A not A This argument is a contradiction, it's incoherent. If you're trying to draw a distinction between physical systems and computation on physical systems, well that's a moot point as every tested theory in physics uses mathematics that is computable. Perhaps reality is not computable. If so then yes, that would mean computation alone will not produce the same kind of consciousness we have. That leaves open other kinds of consciousness, but that's another discussion.
rsanheim 1y ago
> Any physical process can be computed Um, really? How so? Show me the computations that model gut biome. Or pain, or the immune system, or how plants communicate, or how the big bang happened (or didn’t). And now define “physical” wrt things like pain, or sensation, or the body, or consciousness. We know so little about so much! it’s amazing when people speak with such certainty about things like consciousness and computation. I don’t disagree with your main point. I would just say we don’t know very much about our own consciousness and mindbody. And humans have been studying that at least a few thousand years. Not everyone on HN or who works with or studies AI believes AGI is just the next level on some algorithm or scaling level we haven’t unlocked yet.
- jstanley 1y ago
  > Show me the computations Just because computations exist that model a physical process to an arbitrary degree of accuracy doesn't mean we know what those computations are.
  - tsimionescu 1y ago
    But the reality is that we don't know if many of those things are computable. We just think it's likely they are, since the models we have of very simplified versions of them, or of very approximate versions, are computable. But no one can use a physical model or computer to precisely compute, say, the fall of sand (where the exact position and momentum of every grain of sand is precisely known) - so we have no actual proof that the actual physical process is computable. And, in fact, our best fundamental physical theory is not fully computable, at least in some sense - quantum measurement is not a deterministic process, so it's not exactly computable.
    - stevenhuang 1y ago
      QM is computable on classical turing machines, it is just not efficient.
      - tsimionescu 1y ago
yusina 1y ago
These discussions remind me an awful lot of discussions between my religios and my atheist and my agnostic friends. Reglious friend: Well there must be a god, I can feel him, and how else would all this marvellous reality around us have come to existence? Atheist friend: How can you claim that? Where is the evidence? There could be many other explanations, we just don't know, and because of that it doesn't make sense to postulate the existence of a god. And besides, it's just a marketing stunt of the powerful to stay in power. Agnostic friend: Why does any of this matter? Nature is beautiful, sometimes cruel, I'm glad I'm alive, and whether there is a god or not, I don't know or really care, it's insubstantial to my life. That's exactly how the discussions around LLM consciousness or intelligence or sentience always go if you put enough tech folks into the conversation.

deleted 1y ago

[deleted]

imiric 1y ago

> Right here on HN, where you'd think that most people would be able to think better, there's no shortage of AI fanboys who seriously try to frame the idea that LLMs may be conscious Why does this surprise you? This is a community hosted by a Silicon Valley venture capital firm, and many of its members are either employed by or own companies in the AI industry. They are personally invested in pushing the narrative being criticized here, so the counter arguments are expected.

grantcas 1y ago

[dead]