4 ms·

Why do you believe that humans have access to an “internal thought process”? I.e. what do you think is different about an agent’s narration of a thought process

by antonvs 6mo ago

Why do you believe that humans have access to an “internal thought process”? I.e. what do you think is different about an agent’s narration of a thought process vs. a human’s? I suspect you’re making assumptions that don’t hold up to scrutiny.

deleted 6mo ago
[deleted]
fc417fc802 6mo ago
I made no such claim and I don't understand what direct relevance you believe the human thought process has to the issue at hand. You appear to be defaulting to the assumption that LLMs and humans have comparable thought processes. I don't think it's on me to provide evidence to the contrary but rather on you to provide evidence for such a seemingly extraordinary position. For an example of a difference, consider that inserting arbitrary placeholder tokens into the output stream improves the quality of the final result. I don't know about you but if I simply repeat "banana banana banana" to myself my output quality doesn't magically increase.
- DiogenesKynikos 6mo ago
  Given that LLMs can speak basically any language and answer almost any arbitrary question much like a human would, the claim that LLMs have comparable (not identical) thought processes to humans does not seem extraordinary at all.
- antonvs 6mo ago
  > I don't understand what direct relevance you believe the human thought process has to the issue at hand. You're the one who raised it. Perhaps you should clarify what you mean by "isn't real" - do you believe a human narrating their thought process is saying something that's more real? Someone else replied to your comment asking essentially the same question, perhaps better phrased: > What would be different if it was "real"? What makes you think that when humans "narrate" "their" "internal thought process", it's any more "real"?
  - fc417fc802 6mo ago
    No, I did not raise it. I said that X is false. You responded with "why do you think Y is true" and now you ask "do you believe that Y is true" neither of which is relevant to X being true or false. Humans and LLMs are not the same thing. The colloquial term for this is whataboutism. What do I mean by isn't real? Exactly what I said originally. It's a roleplay of something that sounds plausible as opposed to what actually happened. There is obviously some process that is producing the output. The thinking trace is not a representation of that underlying process. Rather the thinking trace is an adjacent output of that same process.
yladiz 6mo ago
Are you legitimately arguing that humans don’t have an internal thought process in some way?
- vidarh 6mo ago
  They're arguing that we have no evidence that humans have access to our underlying thoughts any more than the models do.
  - yladiz 6mo ago
    What does that mean though, to “have access to our underlying thoughts”? Humans can obviously mentally do things that are impossible for a language model to do, because it’s trivial to show that humans do not need language to do mental tasks, and this includes things related to thought, so I don’t really get what is being argued in the first place.
    - antonvs 6mo ago
      > it’s trivial to show that humans do not need language to do mental tasks LLMs don't need language to do mental tasks, either. Their input and output is language - like humans - but in between, the high-dimensional vector representations (often loosely called latent space) are not language in any meaningful sense. LLMs can benefit from "thinking out loud" much as humans can. The issue is not whether the supposed "thoughts" are actually representative on any "internal" thoughts, but rather that explicating the problem in more detail can help reach better conclusions. One point I was making is that the idea that humans are doing something "special" (or in the OP comment's terms, "real") in this area isn't well-supported, in fact there's plenty of evidence against it.
      - yladiz 6mo ago
        My point is that language is not a requirement for humans to perform mental tasks absolutely. It is a fundamental requirement of a large language model.

by antonvs 6mo ago

deleted 6mo ago

[deleted]

fc417fc802 6mo ago

I made no such claim and I don't understand what direct relevance you believe the human thought process has to the issue at hand. You appear to be defaulting to the assumption that LLMs and humans have comparable thought processes. I don't think it's on me to provide evidence to the contrary but rather on you to provide evidence for such a seemingly extraordinary position. For an example of a difference, consider that inserting arbitrary placeholder tokens into the output stream improves the quality of the final result. I don't know about you but if I simply repeat "banana banana banana" to myself my output quality doesn't magically increase.

DiogenesKynikos 6mo ago
Given that LLMs can speak basically any language and answer almost any arbitrary question much like a human would, the claim that LLMs have comparable (not identical) thought processes to humans does not seem extraordinary at all.
antonvs 6mo ago
> I don't understand what direct relevance you believe the human thought process has to the issue at hand. You're the one who raised it. Perhaps you should clarify what you mean by "isn't real" - do you believe a human narrating their thought process is saying something that's more real? Someone else replied to your comment asking essentially the same question, perhaps better phrased: > What would be different if it was "real"? What makes you think that when humans "narrate" "their" "internal thought process", it's any more "real"?
- fc417fc802 6mo ago
  No, I did not raise it. I said that X is false. You responded with "why do you think Y is true" and now you ask "do you believe that Y is true" neither of which is relevant to X being true or false. Humans and LLMs are not the same thing. The colloquial term for this is whataboutism. What do I mean by isn't real? Exactly what I said originally. It's a roleplay of something that sounds plausible as opposed to what actually happened. There is obviously some process that is producing the output. The thinking trace is not a representation of that underlying process. Rather the thinking trace is an adjacent output of that same process.

yladiz 6mo ago

Are you legitimately arguing that humans don’t have an internal thought process in some way?

vidarh 6mo ago
They're arguing that we have no evidence that humans have access to our underlying thoughts any more than the models do.
- yladiz 6mo ago
  What does that mean though, to “have access to our underlying thoughts”? Humans can obviously mentally do things that are impossible for a language model to do, because it’s trivial to show that humans do not need language to do mental tasks, and this includes things related to thought, so I don’t really get what is being argued in the first place.
  - antonvs 6mo ago
    > it’s trivial to show that humans do not need language to do mental tasks LLMs don't need language to do mental tasks, either. Their input and output is language - like humans - but in between, the high-dimensional vector representations (often loosely called latent space) are not language in any meaningful sense. LLMs can benefit from "thinking out loud" much as humans can. The issue is not whether the supposed "thoughts" are actually representative on any "internal" thoughts, but rather that explicating the problem in more detail can help reach better conclusions. One point I was making is that the idea that humans are doing something "special" (or in the OP comment's terms, "real") in this area isn't well-supported, in fact there's plenty of evidence against it.
    - yladiz 6mo ago
      My point is that language is not a requirement for humans to perform mental tasks absolutely. It is a fundamental requirement of a large language model.