4 ms·

I think ChatGPT makes quite clear that understanding language and understanding underlying concepts expressed in language are different things. While it seems t

I think ChatGPT makes quite clear that understanding language and understanding underlying concepts expressed in language are different things. While it seems to be competent in processing and producing language, it appears to have no underlying concept of logic, physics, society, etc. We shouldn't conflate different kinds of understanding. There's no need to say "it's either shuffling tokens blindly or showing intelligence".

janalsncm 3y ago
> it appears to have no underlying concept of logic, physics, society, etc. Sure. ChatGPT is quite limited and gives incorrect and contradictory responses at times. And most importantly (imo) it is unable to update an interpretable knowledge base as real life facts change. But the fact that a particular instance of LLM has this unwanted behavior is not categorical evidence that all LLMs will show the behavior. GPT2 is pretty pathetic compared to GPT3, and ChatGPT is even more impressive. I’m agnostic as to whether statistical language models can ever overcome their current limitations, but given previous emergent behavior I wouldn’t rule it out. GPT4’s context window is something like 32k tokens. Someday that may be orders of magnitude larger, and it may be possible to fit an up to date copy of all of Wikipedia inside (or a condensed version), as well as the current conversation. It might seem crazy but it’s foreseeable.
- namaria 3y ago
  > given previous emergent behavior I wouldn’t rule it out. I find this an interesting turn of phrase, because "emergent behavior" is often used by those holding a reductionist view to hand wave away complexity that cannot be reduced to the analyzed elements. In a way it makes me think of Greek sophists, who would cogently argue for two opposing view points and making a convincing case for both. It's way easier to make a verbal case about something than to prove it true, or false.
perfmode 3y ago
language is just concepts. understanding language and understanding concepts is the same thing. to think one can understand language and not understand concepts is to not understand language.
- namaria 3y ago
  > understanding language and understanding concepts is the same thing It most definitely is not. Are you not aware of non-verbal modes of cognition? I can recall and process physical processes, complex emotional experiences, without using language. I'm willing to bet you, and everyone else, can too. > to think one can understand language and not understand concepts is to not understand language. This is circular logic. We have material evidence of a system that can competently handle language, while falling short of understanding even basic underlying arithmetic, logic, etc.
  - zogrodea 3y ago
    Corroborating the first point with an example. "There is a story that Buddha once, at the climax of a philosophical discussion, broke into gesture-language as an Oxford philosopher may break into Greek: he took a flower in his hand, and looked at it; one of his disciples smiled, and the master said to him, ‘You have understood me.’
    - namaria 3y ago
      If any philosophy ever convinced me there is cognition beyond language, it's Buddhism. I really take it to heart. Whenever I'm experiencing psychic pain I usually realize I'm trying to hold on to some belief or feeling related to my ego and if I can convince myself it's not important for me to hold on to that, the pain melts away...
      - zogrodea 3y ago
        I’m sorry to hear of your pain but it’s good to hear you’re managing to cope with it. Buddhism is an interesting philosophy definitely. It had an influence on David Hume’s bundle theory (that an object is just a collection of properties, instead of having a single substance that persists as the object’s properties change). That recently reminded me of structural vs physical/referential equality in programming. It’s kind of weird come to think of it that an atheist like Hume was so heavily influenced by “religion” come to think of it (some ideas like his version of Occasionalism were influenced by Islamic philosophers too). Didn’t notice the before.

janalsncm 3y ago

> it appears to have no underlying concept of logic, physics, society, etc. Sure. ChatGPT is quite limited and gives incorrect and contradictory responses at times. And most importantly (imo) it is unable to update an interpretable knowledge base as real life facts change. But the fact that a particular instance of LLM has this unwanted behavior is not categorical evidence that all LLMs will show the behavior. GPT2 is pretty pathetic compared to GPT3, and ChatGPT is even more impressive. I’m agnostic as to whether statistical language models can ever overcome their current limitations, but given previous emergent behavior I wouldn’t rule it out. GPT4’s context window is something like 32k tokens. Someday that may be orders of magnitude larger, and it may be possible to fit an up to date copy of all of Wikipedia inside (or a condensed version), as well as the current conversation. It might seem crazy but it’s foreseeable.

namaria 3y ago
> given previous emergent behavior I wouldn’t rule it out. I find this an interesting turn of phrase, because "emergent behavior" is often used by those holding a reductionist view to hand wave away complexity that cannot be reduced to the analyzed elements. In a way it makes me think of Greek sophists, who would cogently argue for two opposing view points and making a convincing case for both. It's way easier to make a verbal case about something than to prove it true, or false.

perfmode 3y ago

language is just concepts. understanding language and understanding concepts is the same thing. to think one can understand language and not understand concepts is to not understand language.

namaria 3y ago
> understanding language and understanding concepts is the same thing It most definitely is not. Are you not aware of non-verbal modes of cognition? I can recall and process physical processes, complex emotional experiences, without using language. I'm willing to bet you, and everyone else, can too. > to think one can understand language and not understand concepts is to not understand language. This is circular logic. We have material evidence of a system that can competently handle language, while falling short of understanding even basic underlying arithmetic, logic, etc.
- zogrodea 3y ago
  Corroborating the first point with an example. "There is a story that Buddha once, at the climax of a philosophical discussion, broke into gesture-language as an Oxford philosopher may break into Greek: he took a flower in his hand, and looked at it; one of his disciples smiled, and the master said to him, ‘You have understood me.’
  - namaria 3y ago
    If any philosophy ever convinced me there is cognition beyond language, it's Buddhism. I really take it to heart. Whenever I'm experiencing psychic pain I usually realize I'm trying to hold on to some belief or feeling related to my ego and if I can convince myself it's not important for me to hold on to that, the pain melts away...
    - zogrodea 3y ago
      I’m sorry to hear of your pain but it’s good to hear you’re managing to cope with it. Buddhism is an interesting philosophy definitely. It had an influence on David Hume’s bundle theory (that an object is just a collection of properties, instead of having a single substance that persists as the object’s properties change). That recently reminded me of structural vs physical/referential equality in programming. It’s kind of weird come to think of it that an atheist like Hume was so heavily influenced by “religion” come to think of it (some ideas like his version of Occasionalism were influenced by Islamic philosophers too). Didn’t notice the before.