6 ms·

We don't even know what the pre-requisites for consciousness are so we have no way of knowing. LLMs have emergent behavior that is reminiscent of language formi

We don't even know what the pre-requisites for consciousness are so we have no way of knowing. LLMs have emergent behavior that is reminiscent of language forming brains, but they're also missing a lot of properties that are probably necessary? Mainly continuity over time, more integrated memory, and a better sense of space and time? Brains use the rhythm and timing of neuronal firings, and the length of axons effects computation, they do a lot of different things with signal and patterns, but in any case without knowing what consciousness is I don't know which of those things are required.

throwuxiytayq 6mo ago
Clive Wearing's memory lasts for less than 30 seconds, so he has no memory of being awake before now. He is permanently in a state of feeling like he has just woken up, observing his surroundings for the first time. Clive Wearing's mind has no time continuity and basically zero memory integration. Is he not conscious? There's interviews with the guy. Where on the scale [No mind <-> Clive Wearing <-> Healthy human brain] would you put an LLM with a 10M token context window?
- deleted 6mo ago
  [deleted]
boxed 6mo ago
> We don't even know what the pre-requisites for consciousness are so we have no way of knowing. Imo we don't even have a definition of the word that we agree on.
- pydry 6mo ago
  We're pretty clear on the distinction between a conscious and an unconscious human. We might not clearly understand the diff between the two states but we can certainly point to it and go "it's that".
  - freedomben 6mo ago
    I'm not sure it's that clear. What about a person who is on drugs to the point they clearly don't know what reality is happening around them, but they are able to speak and move and such? I'm not sure I'd call that conscious, but by most definitions it is.
    - collyw 6mo ago
      Indeed, doing a first aid course we were pointed out that sleeping is different from being unconscious. You can wake someone from sleep pretty quickly. You can't bring an unconscious person back in the same way.
    - Jtarii 6mo ago
      You would just say that they have an altered experience of consciousness from the norm.
  - agnosticmantis 6mo ago
    Now discuss whether a bonobo, a dog, a cat, a mouse, an ant, a bacterium is conscious. And you’ll find it’s not as clear cut.
    - amanaplanacanal 6mo ago
      I'm pretty sure the mammals are conscious the same way I am, in that they experience qualia the same way I do. Insects and bacteria, I suspect not, but how could I tell? There is no way to prove that other humans experience consciousness, really.
  - Jtarii 6mo ago
    >We're pretty clear on the distinction between a conscious and an unconscious human. You are using unconscious as a synonym for asleep, which is not the same thing as having no conscious experience due to dreams. We are clear on the distinction between a dead human and an alive human however.
    - 6mo ago
- qsera 6mo ago
  Ability to feel pain or pleasure is a good indicator I think..
  - echoangle 6mo ago
    And how do you define pain and pleasure? Do insects feel pain?
    - retsibsi 6mo ago
      > And how do you define pain and pleasure? They're not reducible, but I don't know if that means we don't have definitions; we can describe them well enough that most people (who aren't p-zombies or playing the sceptical philosopher role) know pretty well what we mean. All of our definitions have to bottom out somewhere... > Do insects feel pain? Nobody (except the insects) can know for sure. Our inability to know whether X is true doesn't imply X is meaningless, though.
      - echoangle 6mo ago
        But how can X be a good indicator for something I want to determine if I can’t measure X either?
        retsibsi 6mo ago
        > But how can X be a good indicator for something I want to determine if I can’t measure X either? In the comment that started this subthread, qsera was responding to someone who said "Imo we don't even have a definition of [consciousness]". If qsera meant that we can measure consciousness in terms of pleasure and pain, then of course I agree that they were just pushing the problem back a step. But I don't think that's what they meant.
    - qsera 6mo ago
      > Do insects feel pain? Yes, I think so. Because they show behavior that is consistent with being in a state of pain. Despite what consciousness really is, I think evolution found a way to tap into that, by causing pain, or by registering pain on the consciousness by some unknown mechanism, for behaviors that are not beneficial to the organism that hosts the respective consciousness... So I think if an organism that evolved here can display painful behavior, then it should really feel pain.
- mrandish 6mo ago
  > we don't even have a definition of the word that we agree on. Indeed, for any in-depth discussion of LLMs and consciousness to be productive, clearly defining terms and scope is essential. The Stanford Encyclopedia of Philosophy is an excellent resource: https://plato.stanford.edu/entries/consciousness/ https://plato.stanford.edu/entries/consciousness/
KaiserPro 6mo ago
> LLMs have emergent behavior that is reminiscent of language forming brains, Indeed, but then we need to prove that they are not "chinese box" conscious. Which is hard, because it might be that the thing running the chinese box is conscious, but can only communicate in a way it doesn't understand
- mrandish 6mo ago
  Reference: Philosopher John Searle's influential "Chinese Room" thought experiment https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
jaybrendansmith 6mo ago
For consciousness, what is required is a 'master model' that is trained on grounded experience over time to understand, based on its goals, what inputs to focus on. Without this, even an LLM with continuity over time could not even understand what to keep in its context window, let alone what questions to answer or actions to take. You need the entire thing for consciousness, you cannot fake it, because what an LLM does now is simply instantiate a new 'self' every time it is asked a question or at least within a context window. Humans arguably do that every morning when they wake up, but they have this master model that is trained throughout their lifetime on what is important and what is not to their goals.

throwuxiytayq 6mo ago

Clive Wearing's memory lasts for less than 30 seconds, so he has no memory of being awake before now. He is permanently in a state of feeling like he has just woken up, observing his surroundings for the first time. Clive Wearing's mind has no time continuity and basically zero memory integration. Is he not conscious? There's interviews with the guy. Where on the scale [No mind <-> Clive Wearing <-> Healthy human brain] would you put an LLM with a 10M token context window?

deleted 6mo ago
[deleted]

boxed 6mo ago

> We don't even know what the pre-requisites for consciousness are so we have no way of knowing. Imo we don't even have a definition of the word that we agree on.

pydry 6mo ago
We're pretty clear on the distinction between a conscious and an unconscious human. We might not clearly understand the diff between the two states but we can certainly point to it and go "it's that".
- freedomben 6mo ago
  I'm not sure it's that clear. What about a person who is on drugs to the point they clearly don't know what reality is happening around them, but they are able to speak and move and such? I'm not sure I'd call that conscious, but by most definitions it is.
  - collyw 6mo ago
    Indeed, doing a first aid course we were pointed out that sleeping is different from being unconscious. You can wake someone from sleep pretty quickly. You can't bring an unconscious person back in the same way.
  - Jtarii 6mo ago
    You would just say that they have an altered experience of consciousness from the norm.
- agnosticmantis 6mo ago
  Now discuss whether a bonobo, a dog, a cat, a mouse, an ant, a bacterium is conscious. And you’ll find it’s not as clear cut.
  - amanaplanacanal 6mo ago
    I'm pretty sure the mammals are conscious the same way I am, in that they experience qualia the same way I do. Insects and bacteria, I suspect not, but how could I tell? There is no way to prove that other humans experience consciousness, really.
- Jtarii 6mo ago
  >We're pretty clear on the distinction between a conscious and an unconscious human. You are using unconscious as a synonym for asleep, which is not the same thing as having no conscious experience due to dreams. We are clear on the distinction between a dead human and an alive human however.
  - 6mo ago
qsera 6mo ago
Ability to feel pain or pleasure is a good indicator I think..
- echoangle 6mo ago
  And how do you define pain and pleasure? Do insects feel pain?
  - retsibsi 6mo ago
    > And how do you define pain and pleasure? They're not reducible, but I don't know if that means we don't have definitions; we can describe them well enough that most people (who aren't p-zombies or playing the sceptical philosopher role) know pretty well what we mean. All of our definitions have to bottom out somewhere... > Do insects feel pain? Nobody (except the insects) can know for sure. Our inability to know whether X is true doesn't imply X is meaningless, though.
    - echoangle 6mo ago
      But how can X be a good indicator for something I want to determine if I can’t measure X either?
      - retsibsi 6mo ago
        > But how can X be a good indicator for something I want to determine if I can’t measure X either? In the comment that started this subthread, qsera was responding to someone who said "Imo we don't even have a definition of [consciousness]". If qsera meant that we can measure consciousness in terms of pleasure and pain, then of course I agree that they were just pushing the problem back a step. But I don't think that's what they meant.
  - qsera 6mo ago
    > Do insects feel pain? Yes, I think so. Because they show behavior that is consistent with being in a state of pain. Despite what consciousness really is, I think evolution found a way to tap into that, by causing pain, or by registering pain on the consciousness by some unknown mechanism, for behaviors that are not beneficial to the organism that hosts the respective consciousness... So I think if an organism that evolved here can display painful behavior, then it should really feel pain.
mrandish 6mo ago
> we don't even have a definition of the word that we agree on. Indeed, for any in-depth discussion of LLMs and consciousness to be productive, clearly defining terms and scope is essential. The Stanford Encyclopedia of Philosophy is an excellent resource: https://plato.stanford.edu/entries/consciousness/ https://plato.stanford.edu/entries/consciousness/

KaiserPro 6mo ago

> LLMs have emergent behavior that is reminiscent of language forming brains, Indeed, but then we need to prove that they are not "chinese box" conscious. Which is hard, because it might be that the thing running the chinese box is conscious, but can only communicate in a way it doesn't understand

mrandish 6mo ago
Reference: Philosopher John Searle's influential "Chinese Room" thought experiment https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

jaybrendansmith 6mo ago

For consciousness, what is required is a 'master model' that is trained on grounded experience over time to understand, based on its goals, what inputs to focus on. Without this, even an LLM with continuity over time could not even understand what to keep in its context window, let alone what questions to answer or actions to take. You need the entire thing for consciousness, you cannot fake it, because what an LLM does now is simply instantiate a new 'self' every time it is asked a question or at least within a context window. Humans arguably do that every morning when they wake up, but they have this master model that is trained throughout their lifetime on what is important and what is not to their goals.