6 ms·

Sorry, but introspection is not possible in LLMs. They don't have the hardware to have metaphysical dualism and therefore no mind to mentally be aware of.

amanaplanacanal 2mo ago
I suspect the article is bullshit, but what hardware are you thinking of?
- rmunn 2mo ago
  I'm not GP, but the brain is a fantastically complex machine at the cellular level. It's not just electrical activity in neurons, there are also chemical neurotransmitters that play a significant role in emotion, memory, and so on. We understand more about them than we did a hundred years ago, but there's still so much about them where our understanding is limited: for example, we still don't know how the brain produces consciousness (and, in fact, what "consciousness" actually is is widely disputed). But it's pretty indisputable that the brain is an essential component of consciousness: getting your brain knocked around inside your skull (a concussion) usually produces unconsciousness (and sometimes worse), for example. Now, maybe consciousness is produced exclusively through the electrical action of neurons, in which case computers have the necessary machinery to simulate it. But if neurotransmitters, or other things that we haven't even discovered yet, are vital parts of the process that produces consciousness (and I'm not the only one to strongly suspect they are), then computer hardware, lacking anything that even remotely resembles the action of neurotransmitter chemicals, will be unable to produce consciousness.
  - kelseyfrog 2mo ago
    People will make absolutely absurd claims about my own interiority when I say I don't have a feeling of consciousness as if they have epistemic authority over people's inner world. When I ask people, "When did you first remember feeling conscious?" and receive nonsense replies - mostly people respond with their first memories and make no reference to the feeling of consciousness. Consciousness, like God, is often explained via gaps. Quantum effects, holography, emergency, any number of hard-to-understand ideas from science. All of these don't paint a great picture for the idea of consciousness. There's no way to measure it. At least with other sense we can set up discriminative tests - I can use my feeling of the qualia of color to distinguish objects. I can use the qualia of smell to distinguish chemicals. It honestly doesn't feel like it exists at times, it's so elusive. This is turning into a rant, but make it make sense.
    - andrewflnr 2mo ago
      Subjective experience (what most people mean when they say "consciousness") is basically prior to logic, language, etc. "Making sense" just doesn't have a way to come to grips with it. Despite this, for most of us anyway, it plainly exists. This is why it's The Hard Problem. We can neither ignore it nor make headway against it. It's no surprise that most attempts at talking about it are, as you've noticed, a disaster. At the risk of making absurd claims about your interiority: you might be overthinking it. It's conceivable that you're an honest philosophical zombie in a world of conscious humans, but more likely whatever you do experience is the same thing everyone else is talking about, and you just don't resonate with their overly grandiose descriptions.
      - kelseyfrog 2mo ago
        Thank you for the generous and kind response. Part of me wonders if, like alexithymia[1], aphantasia, anauralia[2] and anendophasia[3], there exists the inability to recognize one's own conscious experience. We would never know. 1. https://en.wikipedia.org/wiki/Alexithymia https://en.wikipedia.org/wiki/Alexithymia 2. https://www.anauralia.com/anauralia https://www.anauralia.com/anauralia 3. https://pubmed.ncbi.nlm.nih.gov/38728320/ https://pubmed.ncbi.nlm.nih.gov/38728320/
        andrewflnr 2mo ago
        It seems more likely when you put it that way. Thanks for the interesting links regardless. Deeper cuts than the average aphantasia reference.
    - rmunn 2mo ago
      As andrewfinr mentioned, it seems like you might be working with a different definition of "consciousness" than most people are using, hence why their responses appear to be nonsense. Certainly when I talked about consciousness and the brain, I was going by the most basic, simple definition: awareness. Are you awake and perceiving the world? Then you're conscious. Are you asleep or knocked out? Then you're unconscious. Now, self-awareness (the ability to reason about one's own consciousness) is one meta step beyond being conscious. Animals have awareness, experience the world through their senses, and so on. But to date, there is no evidence that I'm aware of that animals can reason about their own experience of the world. Learn from it, yes: many animals are capable of remembering past experiences and learning from them (if I push this lever I receive food, and so on). But reasoning about it, thinking about the process of thinking and so on? So far it appears that that ability is unique to human beings. (And contra the paper's authors, I don't believe that the LLMs are doing that. The reasons why would be complex and this comment is long enough already, so I'll defer the long explanation unless anyone actually cares to hear it). EDIT to add this P.S.: I realized on re-reading my first comment that I wasn't entirely consistent in how I used the term "consciousness". Specifically, when I said that we're not entirely sure how the brain produces consciousness, I was thinking more of self-awareness than of awareness. Sure, we also don't fully understand the mechanics of how the brain takes nerve signals from our retinas and turns that into "sight", but we understand quite a bit of the process. But how the brain produces the sense of self, the sense that "I" exist as a separate being from "you" and all that goes along with that? That's still a deep mystery, and we have (if anything) only the most surface-level understanding of that process. In the rest of my comment I was using the most simple definition of consciousness, as I mentioned... but in that one sentence I slipped into the more meta definition. Which goes to illustrate how hard these things can be to talk about: any time there's more than one definition for a single word, confusion can arise. I was trying to stick to one definition and I failed; imagine how confusing it can be to have a group discussion where multiple people are mentally thinking of different definitions of the word? There's a good reason why most philosophy texts I read (I took a philosophy minor in college, though I haven't kept up on it) spend the first few chapters carefully defining their terms, only launching into their actual argument once the terms have been defined.
- mikestorrent 2mo ago
  Well, homomorphic encryption is becoming a thing, and secure enclave/TPM chips have been a thing for ages. What would it take to give some kind of AI hardware the ability to have "private thoughts", i.e. cryptographically impossible to go and look at, just like human thoughts are (for most intents and purposes)? Could we let it spend some time on introspection - burning thinking tokens into the black box? I'm not saying we _should_, just putting the idea out into the aether...
  - blooalien 2mo ago
    Now, build an "AI chip" that can never stop "thinking"; even when it's in "sleep mode" it "dreams". Always active. Run it with a monstrous amount of RAM and GPU arrays for active "thinking" and a long term memory store (massive SSD disk array) and use the "dream time" to sort, organize, and prune that memory. Put it into a hybrid constant training/inference state so it can do both/either; constantly updating it's own model weights. Maybe even a tool harness to allow it to update it's own software/request hardware upgrades? Maybe even give it tools to train and implement new/other model types it might need/want? How soon do we get real AGI/ASI? How soon after "The Matrix"/"Terminator"-like reality (or for that matter "Star Trek"-like reality if the AI decides on a benevolent existence)?
GPerson 2mo ago
They’re just going to define terms so that eventually they discover consciousness and then play dumb long enough for us all to get screwed (see Jacob Tsimerman’s scenario 1 if you prefer your alarmism to come from a prestigious source).
Kim_Bruning 2mo ago
> They don't have the hardware to have metaphysical dualism and therefore no mind to mentally be aware of. Somehow this sentence is self-contradictory, but I can't quite lay my finger on the exact spot.
- andrewflnr 2mo ago
  The contradiction is the notion of having hardware to support metaphysical dualism. It's an oxymoron. I think what they're actually trying to say is that the physical reality of LLMs doesn't leave any metaphysical space for dualism, which is required in their view for "mind". Ed: or maybe not, based on their other comments.

amanaplanacanal 2mo ago

I suspect the article is bullshit, but what hardware are you thinking of?

rmunn 2mo ago
I'm not GP, but the brain is a fantastically complex machine at the cellular level. It's not just electrical activity in neurons, there are also chemical neurotransmitters that play a significant role in emotion, memory, and so on. We understand more about them than we did a hundred years ago, but there's still so much about them where our understanding is limited: for example, we still don't know how the brain produces consciousness (and, in fact, what "consciousness" actually is is widely disputed). But it's pretty indisputable that the brain is an essential component of consciousness: getting your brain knocked around inside your skull (a concussion) usually produces unconsciousness (and sometimes worse), for example. Now, maybe consciousness is produced exclusively through the electrical action of neurons, in which case computers have the necessary machinery to simulate it. But if neurotransmitters, or other things that we haven't even discovered yet, are vital parts of the process that produces consciousness (and I'm not the only one to strongly suspect they are), then computer hardware, lacking anything that even remotely resembles the action of neurotransmitter chemicals, will be unable to produce consciousness.
- kelseyfrog 2mo ago
  People will make absolutely absurd claims about my own interiority when I say I don't have a feeling of consciousness as if they have epistemic authority over people's inner world. When I ask people, "When did you first remember feeling conscious?" and receive nonsense replies - mostly people respond with their first memories and make no reference to the feeling of consciousness. Consciousness, like God, is often explained via gaps. Quantum effects, holography, emergency, any number of hard-to-understand ideas from science. All of these don't paint a great picture for the idea of consciousness. There's no way to measure it. At least with other sense we can set up discriminative tests - I can use my feeling of the qualia of color to distinguish objects. I can use the qualia of smell to distinguish chemicals. It honestly doesn't feel like it exists at times, it's so elusive. This is turning into a rant, but make it make sense.
  - andrewflnr 2mo ago
    Subjective experience (what most people mean when they say "consciousness") is basically prior to logic, language, etc. "Making sense" just doesn't have a way to come to grips with it. Despite this, for most of us anyway, it plainly exists. This is why it's The Hard Problem. We can neither ignore it nor make headway against it. It's no surprise that most attempts at talking about it are, as you've noticed, a disaster. At the risk of making absurd claims about your interiority: you might be overthinking it. It's conceivable that you're an honest philosophical zombie in a world of conscious humans, but more likely whatever you do experience is the same thing everyone else is talking about, and you just don't resonate with their overly grandiose descriptions.
    - kelseyfrog 2mo ago
      Thank you for the generous and kind response. Part of me wonders if, like alexithymia[1], aphantasia, anauralia[2] and anendophasia[3], there exists the inability to recognize one's own conscious experience. We would never know. 1. https://en.wikipedia.org/wiki/Alexithymia https://en.wikipedia.org/wiki/Alexithymia 2. https://www.anauralia.com/anauralia https://www.anauralia.com/anauralia 3. https://pubmed.ncbi.nlm.nih.gov/38728320/ https://pubmed.ncbi.nlm.nih.gov/38728320/
      - andrewflnr 2mo ago
        It seems more likely when you put it that way. Thanks for the interesting links regardless. Deeper cuts than the average aphantasia reference.
  - rmunn 2mo ago
    As andrewfinr mentioned, it seems like you might be working with a different definition of "consciousness" than most people are using, hence why their responses appear to be nonsense. Certainly when I talked about consciousness and the brain, I was going by the most basic, simple definition: awareness. Are you awake and perceiving the world? Then you're conscious. Are you asleep or knocked out? Then you're unconscious. Now, self-awareness (the ability to reason about one's own consciousness) is one meta step beyond being conscious. Animals have awareness, experience the world through their senses, and so on. But to date, there is no evidence that I'm aware of that animals can reason about their own experience of the world. Learn from it, yes: many animals are capable of remembering past experiences and learning from them (if I push this lever I receive food, and so on). But reasoning about it, thinking about the process of thinking and so on? So far it appears that that ability is unique to human beings. (And contra the paper's authors, I don't believe that the LLMs are doing that. The reasons why would be complex and this comment is long enough already, so I'll defer the long explanation unless anyone actually cares to hear it). EDIT to add this P.S.: I realized on re-reading my first comment that I wasn't entirely consistent in how I used the term "consciousness". Specifically, when I said that we're not entirely sure how the brain produces consciousness, I was thinking more of self-awareness than of awareness. Sure, we also don't fully understand the mechanics of how the brain takes nerve signals from our retinas and turns that into "sight", but we understand quite a bit of the process. But how the brain produces the sense of self, the sense that "I" exist as a separate being from "you" and all that goes along with that? That's still a deep mystery, and we have (if anything) only the most surface-level understanding of that process. In the rest of my comment I was using the most simple definition of consciousness, as I mentioned... but in that one sentence I slipped into the more meta definition. Which goes to illustrate how hard these things can be to talk about: any time there's more than one definition for a single word, confusion can arise. I was trying to stick to one definition and I failed; imagine how confusing it can be to have a group discussion where multiple people are mentally thinking of different definitions of the word? There's a good reason why most philosophy texts I read (I took a philosophy minor in college, though I haven't kept up on it) spend the first few chapters carefully defining their terms, only launching into their actual argument once the terms have been defined.
mikestorrent 2mo ago
Well, homomorphic encryption is becoming a thing, and secure enclave/TPM chips have been a thing for ages. What would it take to give some kind of AI hardware the ability to have "private thoughts", i.e. cryptographically impossible to go and look at, just like human thoughts are (for most intents and purposes)? Could we let it spend some time on introspection - burning thinking tokens into the black box? I'm not saying we _should_, just putting the idea out into the aether...
- blooalien 2mo ago
  Now, build an "AI chip" that can never stop "thinking"; even when it's in "sleep mode" it "dreams". Always active. Run it with a monstrous amount of RAM and GPU arrays for active "thinking" and a long term memory store (massive SSD disk array) and use the "dream time" to sort, organize, and prune that memory. Put it into a hybrid constant training/inference state so it can do both/either; constantly updating it's own model weights. Maybe even a tool harness to allow it to update it's own software/request hardware upgrades? Maybe even give it tools to train and implement new/other model types it might need/want? How soon do we get real AGI/ASI? How soon after "The Matrix"/"Terminator"-like reality (or for that matter "Star Trek"-like reality if the AI decides on a benevolent existence)?

GPerson 2mo ago

They’re just going to define terms so that eventually they discover consciousness and then play dumb long enough for us all to get screwed (see Jacob Tsimerman’s scenario 1 if you prefer your alarmism to come from a prestigious source).

Kim_Bruning 2mo ago

> They don't have the hardware to have metaphysical dualism and therefore no mind to mentally be aware of. Somehow this sentence is self-contradictory, but I can't quite lay my finger on the exact spot.

andrewflnr 2mo ago
The contradiction is the notion of having hardware to support metaphysical dualism. It's an oxymoron. I think what they're actually trying to say is that the physical reality of LLMs doesn't leave any metaphysical space for dualism, which is required in their view for "mind". Ed: or maybe not, based on their other comments.