4 ms·

To me LLMs seem to most closely resemble the regions of the brain used for converting speech to abstract thought and vice-versa, because LLMs are very good at g

by Salgat 2y ago

To me LLMs seem to most closely resemble the regions of the brain used for converting speech to abstract thought and vice-versa, because LLMs are very good at generating natural language and knowing the flow of speech. An LLM is similar to if you took the the Wernicke's and Broca's Areas and stuck a regression between them. The problem is that the regression in the middle is just a brute force of the entire world's knowledge instead of a real thought.

randcraw 2y ago
I think the major lessons from the success of LLMs are two: 1) the astonishing power of a largely trivial association engine based only on the semantic categories inferred by word2vec, and 2) that so much of the communication abilities of the human mind require so little rational thought (since LLMs demonstrate essentially none of the skills in Kahneman's and Tversky's System 2 thinking (logic, circumspection, self-correction, reflection, etc). I guess this also disproves Minsky's 'Society of Mind' conjecture - a large part of human cognition (System 1) does not require the complex interaction of heterogeneous mental components.
- GeneralMayhem 2y ago
  What makes this tough is that LLMs can show logical thinking and self-correction when specifically prompted (e.g. "think step by step", "double-check and then correct your work"). It seems unlikely that they can truthfully self-reflect, but I don't think it's strictly impossible.
  - deleted 2y ago
    [deleted]
  - Terr_ 2y ago
    > LLMs can show logical thinking and self-correction The same way they "show" sadness or contrition or excitement? We need to be careful with our phrasing here: LLMs can be prompted to provide you associated phrases that usually seem to fit with the rest of the word-soup, but whether the model is actually demonstrating "logical thinking" or "self-correction" is a Chinese Room problem [0]. (Or else a "No, it doesn't, I can tell because I checked the code.") [0] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
- autoexec 2y ago
  > that so much of the communication abilities of the human mind require so little rational thought Beyond that fact that human minds created them, I doubt that LLMs can tell us anything about the abilities of the human mind or what language in humans requires. The most we can learn from LLMs about ourselves will be in how we react to them, or more broadly, what the datasets LLMs use show about who we are.

by Salgat 2y ago

randcraw 2y ago

I think the major lessons from the success of LLMs are two: 1) the astonishing power of a largely trivial association engine based only on the semantic categories inferred by word2vec, and 2) that so much of the communication abilities of the human mind require so little rational thought (since LLMs demonstrate essentially none of the skills in Kahneman's and Tversky's System 2 thinking (logic, circumspection, self-correction, reflection, etc). I guess this also disproves Minsky's 'Society of Mind' conjecture - a large part of human cognition (System 1) does not require the complex interaction of heterogeneous mental components.

GeneralMayhem 2y ago
What makes this tough is that LLMs can show logical thinking and self-correction when specifically prompted (e.g. "think step by step", "double-check and then correct your work"). It seems unlikely that they can truthfully self-reflect, but I don't think it's strictly impossible.
- deleted 2y ago
  [deleted]
- Terr_ 2y ago
  > LLMs can show logical thinking and self-correction The same way they "show" sadness or contrition or excitement? We need to be careful with our phrasing here: LLMs can be prompted to provide you associated phrases that usually seem to fit with the rest of the word-soup, but whether the model is actually demonstrating "logical thinking" or "self-correction" is a Chinese Room problem [0]. (Or else a "No, it doesn't, I can tell because I checked the code.") [0] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
autoexec 2y ago
> that so much of the communication abilities of the human mind require so little rational thought Beyond that fact that human minds created them, I doubt that LLMs can tell us anything about the abilities of the human mind or what language in humans requires. The most we can learn from LLMs about ourselves will be in how we react to them, or more broadly, what the datasets LLMs use show about who we are.