3 ms·

You are seriously telling me that humans predicting word for word when they speak?

by maaanu 3y ago

adam_arthur 3y ago
A system that "predicts the next token" in such a way that it is indistinguishable from a human, is just like a human in practice yes. How does a human decide which word to use in your mind? Magic? No, it's a logically based biological/neurological process through which at the end of it, you've decided on a word. They are both forms of computing that can produce largely indistinguishable output... doesn't matter that one is biological and the other isn't
- RhodesianHunter 3y ago
  It's quite possible a human forms a full representation of a coherent thought before translating that thought into words, meaning no token by token prediction.
  - adam_arthur 3y ago
    And digital audio will never match analog. Can anybody tell the difference anymore? If a machine produces highly similar output does it matter? LLM's clearly exhibit behavior of having implicitly learned systems, which is the valuable part of intelligence. Humans infer systems through text too, by the way.
    - RhodesianHunter 3y ago
      We're not at the point where people can't tell the difference though. A human will often say "I don't know". The machine will outright lie using it's best probabilistic guess.
  - doctor_eval 3y ago
    It’s also possible that translating a coherent thought into words is done on a token by token basis. I’m surely not the only one who sometimes can’t “find the right word” in the middle of a sentence when trying to describe a thought or idea.
    - RhodesianHunter 3y ago
      Right, but that would still be functionally quite different from what an LLM does, given that it has no full thought and is entirely predictive by token.
stevenhuang 3y ago
Actually yes, architecturally that's the essence of predictive coding. It's among the leading theories in neuroscience for how our brains work https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding

You are seriously telling me that humans predicting word for word when they speak?

by maaanu 3y ago

You are seriously telling me that humans predicting word for word when they speak?

A system that "predicts the next token" in such a way that it is indistinguishable from a human, is just like a human in practice yes. How does a human decide which word to use in your mind? Magic? No, it's a logically based biological/neurological process through which at the end of it, you've decided on a word. They are both forms of computing that can produce largely indistinguishable output... doesn't matter that one is biological and the other isn't

RhodesianHunter 3y ago
It's quite possible a human forms a full representation of a coherent thought before translating that thought into words, meaning no token by token prediction.
- adam_arthur 3y ago
  And digital audio will never match analog. Can anybody tell the difference anymore? If a machine produces highly similar output does it matter? LLM's clearly exhibit behavior of having implicitly learned systems, which is the valuable part of intelligence. Humans infer systems through text too, by the way.
  - RhodesianHunter 3y ago
    We're not at the point where people can't tell the difference though. A human will often say "I don't know". The machine will outright lie using it's best probabilistic guess.
- doctor_eval 3y ago
  It’s also possible that translating a coherent thought into words is done on a token by token basis. I’m surely not the only one who sometimes can’t “find the right word” in the middle of a sentence when trying to describe a thought or idea.
  - RhodesianHunter 3y ago
    Right, but that would still be functionally quite different from what an LLM does, given that it has no full thought and is entirely predictive by token.

stevenhuang 3y ago

Actually yes, architecturally that's the essence of predictive coding. It's among the leading theories in neuroscience for how our brains work https://en.wikipedia.org/wiki/Predictive_coding https://en.wikipedia.org/wiki/Predictive_coding