4 ms·

Are you legitimately arguing that humans don’t have an internal thought process in some way?

by yladiz 6mo ago

vidarh 6mo ago
They're arguing that we have no evidence that humans have access to our underlying thoughts any more than the models do.
- yladiz 6mo ago
  What does that mean though, to “have access to our underlying thoughts”? Humans can obviously mentally do things that are impossible for a language model to do, because it’s trivial to show that humans do not need language to do mental tasks, and this includes things related to thought, so I don’t really get what is being argued in the first place.
  - antonvs 6mo ago
    > it’s trivial to show that humans do not need language to do mental tasks LLMs don't need language to do mental tasks, either. Their input and output is language - like humans - but in between, the high-dimensional vector representations (often loosely called latent space) are not language in any meaningful sense. LLMs can benefit from "thinking out loud" much as humans can. The issue is not whether the supposed "thoughts" are actually representative on any "internal" thoughts, but rather that explicating the problem in more detail can help reach better conclusions. One point I was making is that the idea that humans are doing something "special" (or in the OP comment's terms, "real") in this area isn't well-supported, in fact there's plenty of evidence against it.
    - yladiz 6mo ago
      My point is that language is not a requirement for humans to perform mental tasks absolutely. It is a fundamental requirement of a large language model.
      - fc417fc802 6mo ago
        That's a meaningless argument of definition. Replace the language input and output with something else and it's no longer termed an LLM. It's like saying that a "human who writes with right hand" fundamentally requires his right hand in order to write anything because without it he is no longer a "human who writes with right hand" despite that he is still writing (now with his left hand).
        yladiz 6mo ago
        I’m not sure I follow. A language model fundamentally needs language to operate, and humans do not. Am I missing something from your point?
    - fc417fc802 6mo ago
      > LLMs can benefit from "thinking out loud" much as humans can. The two processes aren't equivalent. An LLM that fills the thinking trace with a meaningless placeholder token will still exhibit improved performance. There are also regularly things in the thinking trace that don't match the final output if you look closely but on the surface they appear convincing. It's largely a trained performance. If you go in with the erroneous expectation that it accurately reflects the underlying thought process then you're likely to come away with faulty conclusions.

Are you legitimately arguing that humans don’t have an internal thought process in some way?

by yladiz 6mo ago

Are you legitimately arguing that humans don’t have an internal thought process in some way?

They're arguing that we have no evidence that humans have access to our underlying thoughts any more than the models do.

yladiz 6mo ago
What does that mean though, to “have access to our underlying thoughts”? Humans can obviously mentally do things that are impossible for a language model to do, because it’s trivial to show that humans do not need language to do mental tasks, and this includes things related to thought, so I don’t really get what is being argued in the first place.
- antonvs 6mo ago
  > it’s trivial to show that humans do not need language to do mental tasks LLMs don't need language to do mental tasks, either. Their input and output is language - like humans - but in between, the high-dimensional vector representations (often loosely called latent space) are not language in any meaningful sense. LLMs can benefit from "thinking out loud" much as humans can. The issue is not whether the supposed "thoughts" are actually representative on any "internal" thoughts, but rather that explicating the problem in more detail can help reach better conclusions. One point I was making is that the idea that humans are doing something "special" (or in the OP comment's terms, "real") in this area isn't well-supported, in fact there's plenty of evidence against it.
  - yladiz 6mo ago
    My point is that language is not a requirement for humans to perform mental tasks absolutely. It is a fundamental requirement of a large language model.
    - fc417fc802 6mo ago
      That's a meaningless argument of definition. Replace the language input and output with something else and it's no longer termed an LLM. It's like saying that a "human who writes with right hand" fundamentally requires his right hand in order to write anything because without it he is no longer a "human who writes with right hand" despite that he is still writing (now with his left hand).
      - yladiz 6mo ago
        I’m not sure I follow. A language model fundamentally needs language to operate, and humans do not. Am I missing something from your point?
  - fc417fc802 6mo ago
    > LLMs can benefit from "thinking out loud" much as humans can. The two processes aren't equivalent. An LLM that fills the thinking trace with a meaningless placeholder token will still exhibit improved performance. There are also regularly things in the thinking trace that don't match the final output if you look closely but on the surface they appear convincing. It's largely a trained performance. If you go in with the erroneous expectation that it accurately reflects the underlying thought process then you're likely to come away with faulty conclusions.