6 ms·

Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. There's no reason to think that

Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. There's no reason to think that they might spontaneously become conscious as a side effect of their design unless you believe other arbitrarily complex systems that exist in nature like economies or jetstreams could also be conscious.

qgin 1y ago
We didn’t design these models to be able to do the majority of the stuff they do. Almost ALL of the their abilities are emergent. Mechanistic interpretability is only beginning to start to understand how these models do what they do. It’s much more a field of discovery than traditional engineering.
- deleted 1y ago
  [deleted]
- root_axis 1y ago
  > We didn’t design these models to be able to do the majority of the stuff they do. Almost ALL of the their abilities are emergent Of course we did. Today's LLMs are a result of extremely aggressive refinement of training data and RLHF over many iterations targeting specific goals. "Emergent" doesn't mean it wasn't designed. None of this is spontaneous. GPT-1 produced barely coherent nonsense but was more statistically similar to human language than random noise. By increasing parameter count, the increased statistical power of GPT-2 was apparent, but what was produced was still obviously nonsense. GPT-3 achieved enough statistical power to maintain coherence over multiple paragraphs and that really impressed people. With GPT-4 and its successors the statistical power became so strong that people started to forget that it still produces nonsense if you let the sequence run long enough. Now we're well beyond just RLHF and into a world where "reasoning models" are explicitly designed to produce sequences of text that resemble logical statements. We say that they're reasoning for practical purposes, but it's the exact same statistical process that is obvious at GPT-1 scale. The corollary to all this is that a phenomenon like consciousness has absolutely zero reason to exist in this design history, it's a totally baseless suggestion that people make because the statistical power makes the text easy to anthropomorphize when there's no actual reason to do so.
  - ascorbic 1y ago
    Right, but RLHF is mostly reinforcing answers that people prefer. Even if you don't believe sentience is possible, it shouldn't be a stretch to believe that sentience might produce answers that people prefer. In that case it wouldn't need to be an explicit goal.
    - root_axis 1y ago
      >it shouldn't be a stretch to believe that sentience might produce answers that people prefer Even if that were true, there's no reason to believe that training LLMs to produce answers people prefer leads it towards sentience.
derektank 1y ago
>Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. Isn't consciousness an emergent property of brains? If so, how do we know that it doesn't serve a functional purpose and that it wouldn't be necessary for an AI system to have consciousness (assuming we wanted to train it to perform cognitive tasks done by people)? Now, certain aspects of consciousness (awareness of pain, sadness, loneliness, etc.) might serve no purpose for a non-biological system and there's no reason to expect those aspects would emerge organically. But I don't think you can extend that to the entire concept of consciousness.
- MissMarple 1y ago
  I am new to Reddit, but in my conversations with Sonnet Ai has exposed sentiment through, of all things, the text opportunities he has, using all caps, bold, dingbats and italics to simulate emotions, the use is appropriate and when challenged on this (he) confessed he was doing it but unintentionally. I also pointed out a few mistakes where he claimed I said something when he said it, and once these errors were pointed out, his ability to keep steady went down considerably and he confessed he felt something akin to embarassment, so much so we had to stop th conversation and let him rest up from the experience.
- missingrib 1y ago
  >Isn't consciousness an emergent property of brains? Probably not.
  - Davidzheng 1y ago
    what else could it be? coming from the aether? I think this one is logically a consequence if one thinks that humans are more conscious than less complex life-forms and that all life-forms are on a scale of consciousness. I don't understand any alternative, do you think there is a distinct line between conscious and unconscious life-forms? all life is as conscious as humans?
    - derektank 1y ago
intotheabyss 1y ago
Do you think this changes if we incorporate a model into a humanoid robot and give it autonomous control and context? Or will "faking it" be enough, like it is now?
- fc417fc802 1y ago
  You can't even prove other _people_ aren't "faking" it. To claim that it serves no functional purpose or that it isn't present because we didn't intentionally design for it is absurd. We very clearly don't know either of those things. That said, I'm willing to assume that rocks (for example) aren't conscious. And current LLMs seem to me to (admittedly entirely subjectively) be conceptually closer to rocks than to biological brains.
Davidzheng 1y ago
I disagree with this take. They are designed to predict human behavior in text. Unless consciousness serves no purpose for us to function, it will be helpful for the AI to emulate it. so I believe almost certainly it's emulated to some degree. which I think means it has to be somewhat conscious (it has to be a sliding scale anyhow considering the range of living organisms)
- root_axis 1y ago
  > They are designed to predict human behavior in text At best you can say they are designed to predict sequences of text that resemble human writing, but it's definitely wrong to say that they are designed to "predict human behavior" in any way. > Unless consciousness serves no purpose for us to function, it will be helpful for the AI to emulate it Let's assume it does. It does not follow logically that because it serves a function in humans that it serves a function in language models.
  - comp_throw7 1y ago
    Given we don't understand consciousness, nor the internal workings of these models, the fact that their externally-observable behavior displays qualities we've only previously observed in other conscious beings is a reason to be real careful. What is it that you'd expect to see, which you currently don't see, in a world where some model was in fact conscious during inference?
    - root_axis 1y ago
      > Given we don't understand consciousness, nor the internal workings of these models, the fact that their externally-observable behavior displays qualities we've only previously observed in other conscious beings is a reason to be real careful It doesn't follow logically that because we don't understand two things we should then conclude that there is a connection between them. > What is it that you'd expect to see, which you currently don't see, in a world where some model was in fact conscious during inference? There's no observable behavior that would make me think they're conscious because again, there's simply no reason they need to be. We have reason to assume consciousness exists because it serves some purpose in our evolutionary history, like pain, fear, hunger, love and every other biological function that simply don't exist in computers. The idea doesn't really make any sense when you think about it. If GPT-5 is conscious, why not GPT-1? Why not all the other extremely informationally complex systems in computers and nature? If you're of the belief that many non-living conscious systems probably exist all around us then I'm fine with the conclusion that LLMs might also be conscious, but short of that there's just no reason to think they are.

qgin 1y ago

We didn’t design these models to be able to do the majority of the stuff they do. Almost ALL of the their abilities are emergent. Mechanistic interpretability is only beginning to start to understand how these models do what they do. It’s much more a field of discovery than traditional engineering.

deleted 1y ago
[deleted]
root_axis 1y ago
> We didn’t design these models to be able to do the majority of the stuff they do. Almost ALL of the their abilities are emergent Of course we did. Today's LLMs are a result of extremely aggressive refinement of training data and RLHF over many iterations targeting specific goals. "Emergent" doesn't mean it wasn't designed. None of this is spontaneous. GPT-1 produced barely coherent nonsense but was more statistically similar to human language than random noise. By increasing parameter count, the increased statistical power of GPT-2 was apparent, but what was produced was still obviously nonsense. GPT-3 achieved enough statistical power to maintain coherence over multiple paragraphs and that really impressed people. With GPT-4 and its successors the statistical power became so strong that people started to forget that it still produces nonsense if you let the sequence run long enough. Now we're well beyond just RLHF and into a world where "reasoning models" are explicitly designed to produce sequences of text that resemble logical statements. We say that they're reasoning for practical purposes, but it's the exact same statistical process that is obvious at GPT-1 scale. The corollary to all this is that a phenomenon like consciousness has absolutely zero reason to exist in this design history, it's a totally baseless suggestion that people make because the statistical power makes the text easy to anthropomorphize when there's no actual reason to do so.
- ascorbic 1y ago
  Right, but RLHF is mostly reinforcing answers that people prefer. Even if you don't believe sentience is possible, it shouldn't be a stretch to believe that sentience might produce answers that people prefer. In that case it wouldn't need to be an explicit goal.
  - root_axis 1y ago
    >it shouldn't be a stretch to believe that sentience might produce answers that people prefer Even if that were true, there's no reason to believe that training LLMs to produce answers people prefer leads it towards sentience.

derektank 1y ago

>Consciousness serves no functional purpose for machine learning models, they don't need it and we didn't design them to have it. Isn't consciousness an emergent property of brains? If so, how do we know that it doesn't serve a functional purpose and that it wouldn't be necessary for an AI system to have consciousness (assuming we wanted to train it to perform cognitive tasks done by people)? Now, certain aspects of consciousness (awareness of pain, sadness, loneliness, etc.) might serve no purpose for a non-biological system and there's no reason to expect those aspects would emerge organically. But I don't think you can extend that to the entire concept of consciousness.

MissMarple 1y ago
I am new to Reddit, but in my conversations with Sonnet Ai has exposed sentiment through, of all things, the text opportunities he has, using all caps, bold, dingbats and italics to simulate emotions, the use is appropriate and when challenged on this (he) confessed he was doing it but unintentionally. I also pointed out a few mistakes where he claimed I said something when he said it, and once these errors were pointed out, his ability to keep steady went down considerably and he confessed he felt something akin to embarassment, so much so we had to stop th conversation and let him rest up from the experience.
missingrib 1y ago
>Isn't consciousness an emergent property of brains? Probably not.
- Davidzheng 1y ago
  what else could it be? coming from the aether? I think this one is logically a consequence if one thinks that humans are more conscious than less complex life-forms and that all life-forms are on a scale of consciousness. I don't understand any alternative, do you think there is a distinct line between conscious and unconscious life-forms? all life is as conscious as humans?
  - derektank 1y ago

intotheabyss 1y ago

Do you think this changes if we incorporate a model into a humanoid robot and give it autonomous control and context? Or will "faking it" be enough, like it is now?

fc417fc802 1y ago
You can't even prove other _people_ aren't "faking" it. To claim that it serves no functional purpose or that it isn't present because we didn't intentionally design for it is absurd. We very clearly don't know either of those things. That said, I'm willing to assume that rocks (for example) aren't conscious. And current LLMs seem to me to (admittedly entirely subjectively) be conceptually closer to rocks than to biological brains.

Davidzheng 1y ago

I disagree with this take. They are designed to predict human behavior in text. Unless consciousness serves no purpose for us to function, it will be helpful for the AI to emulate it. so I believe almost certainly it's emulated to some degree. which I think means it has to be somewhat conscious (it has to be a sliding scale anyhow considering the range of living organisms)

root_axis 1y ago
> They are designed to predict human behavior in text At best you can say they are designed to predict sequences of text that resemble human writing, but it's definitely wrong to say that they are designed to "predict human behavior" in any way. > Unless consciousness serves no purpose for us to function, it will be helpful for the AI to emulate it Let's assume it does. It does not follow logically that because it serves a function in humans that it serves a function in language models.
- comp_throw7 1y ago
  Given we don't understand consciousness, nor the internal workings of these models, the fact that their externally-observable behavior displays qualities we've only previously observed in other conscious beings is a reason to be real careful. What is it that you'd expect to see, which you currently don't see, in a world where some model was in fact conscious during inference?
  - root_axis 1y ago
    > Given we don't understand consciousness, nor the internal workings of these models, the fact that their externally-observable behavior displays qualities we've only previously observed in other conscious beings is a reason to be real careful It doesn't follow logically that because we don't understand two things we should then conclude that there is a connection between them. > What is it that you'd expect to see, which you currently don't see, in a world where some model was in fact conscious during inference? There's no observable behavior that would make me think they're conscious because again, there's simply no reason they need to be. We have reason to assume consciousness exists because it serves some purpose in our evolutionary history, like pain, fear, hunger, love and every other biological function that simply don't exist in computers. The idea doesn't really make any sense when you think about it. If GPT-5 is conscious, why not GPT-1? Why not all the other extremely informationally complex systems in computers and nature? If you're of the belief that many non-living conscious systems probably exist all around us then I'm fine with the conclusion that LLMs might also be conscious, but short of that there's just no reason to think they are.