5 ms·

You might as well be asking a tape recorder why it said something. Why are we confusing the situation with non-nonsensical comparisons? There is no internal m

by jayd16 6mo ago

You might as well be asking a tape recorder why it said something. Why are we confusing the situation with non-nonsensical comparisons? There is no internal monologue with which to have introspection (beyond what the AI companies choose to hide as a matter of UX or what have you). There is no "I was feeling upset when I said/did that" unless it's in the context. There is no ghost in the machine that we cannot see before asking. Even if a model is able to come up with a narrative, it's simply that. Looking at the log and telling you a story.

vidarh 6mo ago
Sperry's experiments makes it quite clear that the comparison is not nonsensical: humans can't reliably tell why we do things either. It is not imbuing AI with anything more to recognise that. Rather pointing out that when we seek to imply the gap is so huge we often overestimate our own abilities.
- abcde666777 6mo ago
  Slight pushback - I think there's still a lot more consistency and coherence in a human's recollection of their motives than an LLM. Sometimes I think we're too eager to compare ourselves to them.
  - vidarh 6mo ago
    We have pretty much evidence to support that human recollection includes the right data to be able to ascertain why we actually did something.
- fluoridation 6mo ago
  Humans at least have a mental state that only they are privy to to work from, and not just their words and actions. The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to.
  - lmm 6mo ago
    > Humans at least have a mental state that only they are privy to to work from Maybe. How do you tell? What would you expect to be different if they didn't? > The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to. Insight is not solely a function of available input information. Arguably being able to search and extract the relevant parts is a far more important part of having insights.
    - fluoridation 6mo ago
      >Maybe. How do you tell? What would you expect to be different if they didn't? I think you're asking how I would know if other people were P-zombies. That's an inappropriate question because I didn't talk about subjective experience, just about internal state. There's no question about whether other people have internal states. I can show someone a piece of information in such a way that only they see it and then ask them to prove that they know it such that I can be certain to an arbitrarily high degree that their report is correct. Unvoiced thoughts are trickier to prove, but quite often they leave their mark in the person's voiced thoughts. >Insight is not solely a function of available input information. Arguably being able to search and extract the relevant parts is a far more important part of having insights. LLMs are notoriously bad at judging relevance. I've noticed quite often if you ask a somewhat vague question they try to cold-read you by throwing various guesses to see which one you latch onto. They're very bad at interpreting novel metaphors, for example.
      - lmm 6mo ago
        > I didn't talk about subjective experience, just about internal state. There's no question about whether other people have internal states. I can show someone a piece of information in such a way that only they see it and then ask them to prove that they know it such that I can be certain to an arbitrarily high degree that their report is correct. Well, sure, but that much is equally true for an LLM with a scratchpad or what have you. (I guess you could say that the user should have access to the LLM's scratchpad and therefore be just as able to understand the state as the LLM itself, but as we move towards the LLM using its own state vectors that's less and less true in practice). I agree that a human may have a mood or secret knowledge or what have you in a way that an LLM wouldn't, but if all you're positing is access to some inert but hidden state then that feels like a Toaster-Enhanced Turing Machine.
- jayd16 6mo ago
  It is non-sensical because you're simply bringing in comparisons without anything linking the two. You might as well be talking about how oranges, and bicycles think as well as that is just as relevant as how humans think in this discussion. In fact, talking about "thinking" at all is already the wrong direction to go down when trying to triage an incident like this. "Do not anthropomorphize the lawnmower" applies to AI as much as Larry Ellison.
  - vidarh 6mo ago
    The thing linking the two is that neither are able to accurately introspect and explain the actual reason why they made a decision. If thinking is the wrong direction to go down, then it is also the wrong direction to go down when talking about humans.
    - jayd16 6mo ago
      If your plane fails to fly and humans can't fly then we should be looking at the musculature of humans when working on the plane?

by jayd16 6mo ago

vidarh 6mo ago

Sperry's experiments makes it quite clear that the comparison is not nonsensical: humans can't reliably tell why we do things either. It is not imbuing AI with anything more to recognise that. Rather pointing out that when we seek to imply the gap is so huge we often overestimate our own abilities.

abcde666777 6mo ago
Slight pushback - I think there's still a lot more consistency and coherence in a human's recollection of their motives than an LLM. Sometimes I think we're too eager to compare ourselves to them.
- vidarh 6mo ago
  We have pretty much evidence to support that human recollection includes the right data to be able to ascertain why we actually did something.
fluoridation 6mo ago
Humans at least have a mental state that only they are privy to to work from, and not just their words and actions. The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to.
- lmm 6mo ago
  > Humans at least have a mental state that only they are privy to to work from Maybe. How do you tell? What would you expect to be different if they didn't? > The LLM literally cannot possibly have a deeper insight into the root cause than the user, because it can only work from the information that the user has access to. Insight is not solely a function of available input information. Arguably being able to search and extract the relevant parts is a far more important part of having insights.
  - fluoridation 6mo ago
    >Maybe. How do you tell? What would you expect to be different if they didn't? I think you're asking how I would know if other people were P-zombies. That's an inappropriate question because I didn't talk about subjective experience, just about internal state. There's no question about whether other people have internal states. I can show someone a piece of information in such a way that only they see it and then ask them to prove that they know it such that I can be certain to an arbitrarily high degree that their report is correct. Unvoiced thoughts are trickier to prove, but quite often they leave their mark in the person's voiced thoughts. >Insight is not solely a function of available input information. Arguably being able to search and extract the relevant parts is a far more important part of having insights. LLMs are notoriously bad at judging relevance. I've noticed quite often if you ask a somewhat vague question they try to cold-read you by throwing various guesses to see which one you latch onto. They're very bad at interpreting novel metaphors, for example.
    - lmm 6mo ago
      > I didn't talk about subjective experience, just about internal state. There's no question about whether other people have internal states. I can show someone a piece of information in such a way that only they see it and then ask them to prove that they know it such that I can be certain to an arbitrarily high degree that their report is correct. Well, sure, but that much is equally true for an LLM with a scratchpad or what have you. (I guess you could say that the user should have access to the LLM's scratchpad and therefore be just as able to understand the state as the LLM itself, but as we move towards the LLM using its own state vectors that's less and less true in practice). I agree that a human may have a mood or secret knowledge or what have you in a way that an LLM wouldn't, but if all you're positing is access to some inert but hidden state then that feels like a Toaster-Enhanced Turing Machine.
jayd16 6mo ago
It is non-sensical because you're simply bringing in comparisons without anything linking the two. You might as well be talking about how oranges, and bicycles think as well as that is just as relevant as how humans think in this discussion. In fact, talking about "thinking" at all is already the wrong direction to go down when trying to triage an incident like this. "Do not anthropomorphize the lawnmower" applies to AI as much as Larry Ellison.
- vidarh 6mo ago
  The thing linking the two is that neither are able to accurately introspect and explain the actual reason why they made a decision. If thinking is the wrong direction to go down, then it is also the wrong direction to go down when talking about humans.
  - jayd16 6mo ago
    If your plane fails to fly and humans can't fly then we should be looking at the musculature of humans when working on the plane?