4 ms·

Humans are easily convinced (fooled) by clever use of language. Politics, tests, interviews, everything. There has always been a difference between actual unde

Humans are easily convinced (fooled) by clever use of language. Politics, tests, interviews, everything. There has always been a difference between actual understanding and just great presentation. LLMs are so far ahead in their use of language that given a complete lack of understanding they are more capable than humans to fool the listener. But there's nothing substantially new, some people have mastered that skill, too.

stavros 3y ago
What's "understanding", and how do you know LLMs lack it?
- foobarqux 3y ago
  It's clear that any sane definition of understanding would exclude the types of failures that you regularly encounter when using LLMs.
  - stavros 3y ago
    Interesting how we need to define understanding in such a way that it doesn't apply to LLMs.
    - foobarqux 3y ago
      I think it's extremely misleading to use a definition that allows for routinely logically contradicting yourself and induces the impression that it doesn't do that. People are regularly doing this in the titles of papers --- ascribing words like "emergence" and "world models" with definitions that don't match their informal usage -- purposely to mislead people into mistakenly applying conventional intuitions and make the paper look like it proves more than it does.
      - stavros 3y ago
        If routinely logically contradicting yourself means you have no understanding, the vast majority of humans have no understanding. This feels like goalpost-moving so we can say that LLMs don't think, because it intuitively seems to us that they don't think. Or, at least, that we should say they don't think, because we're afraid what the alternative says about us.
        foobarqux 3y ago
        The argument that humans make mistakes therefore whatever mistakes machines make are irrelevant to whether they approximate humans is absurd. You presumably use these systems all the time, are you seriously saying that they don't routinely make mistakes that humans don't make?
        stavros 3y ago
        You said "routinely contradicting yourself", not "make mistakes humans don't make". I know plenty of humans who routinely contradict themselves, sometimes without realizing (or at least admitting it). Also, the fact that LLMs now make a class of mistakes doesn't mean that LLMs in X years will. "LLMs don't think because they fail at X" is very different from "the current crop of LLMs don't think because they fail at X".
        foobarqux 3y ago
        Again, you use these systems regularly and presumably you have observed that they make classes of mistakes that humans don't make so you obviously can't make the argument that "they are just like humans", even though humans sometimes make other types of mistakes (at a different rate). I have a hard time believing that someone who was shown a sample of these mistakes (somehow weighted by their frequency) would say that applying the word "understanding" to the system is not more misleading than not.
        stavros 3y ago
        I'm not making the argument that "they are just like humans", though. I'm merely making the argument that I don't know that they don't understand. Someone who was shown a sample of these mistakes would maybe say that the word "understanding" is more misleading than not. I could make the converse argument for someone shown a sample of the successes, though. It seems unfair to judge "understanding" just by what the system can't do, and not by what it can do, while not really applying the same rigor to human babies. This doesn't scream "not understanding" to me: > If I'm holding three ice cubes, and I give one to a friend twenty minutes later, how many ice cubes do I have now?" [... a bunch of correct text elided ...] > In a practical sense, after twenty minutes at room temperature or in your hand, it's realistic to say that you might have no ice cubes left or just a very small amount of ice/water left from the melted ice cubes.

stavros 3y ago

What's "understanding", and how do you know LLMs lack it?

foobarqux 3y ago
It's clear that any sane definition of understanding would exclude the types of failures that you regularly encounter when using LLMs.
- stavros 3y ago
  Interesting how we need to define understanding in such a way that it doesn't apply to LLMs.
  - foobarqux 3y ago
    I think it's extremely misleading to use a definition that allows for routinely logically contradicting yourself and induces the impression that it doesn't do that. People are regularly doing this in the titles of papers --- ascribing words like "emergence" and "world models" with definitions that don't match their informal usage -- purposely to mislead people into mistakenly applying conventional intuitions and make the paper look like it proves more than it does.
    - stavros 3y ago
      If routinely logically contradicting yourself means you have no understanding, the vast majority of humans have no understanding. This feels like goalpost-moving so we can say that LLMs don't think, because it intuitively seems to us that they don't think. Or, at least, that we should say they don't think, because we're afraid what the alternative says about us.
      - foobarqux 3y ago
        The argument that humans make mistakes therefore whatever mistakes machines make are irrelevant to whether they approximate humans is absurd. You presumably use these systems all the time, are you seriously saying that they don't routinely make mistakes that humans don't make?
        stavros 3y ago
        You said "routinely contradicting yourself", not "make mistakes humans don't make". I know plenty of humans who routinely contradict themselves, sometimes without realizing (or at least admitting it). Also, the fact that LLMs now make a class of mistakes doesn't mean that LLMs in X years will. "LLMs don't think because they fail at X" is very different from "the current crop of LLMs don't think because they fail at X".
        foobarqux 3y ago
        Again, you use these systems regularly and presumably you have observed that they make classes of mistakes that humans don't make so you obviously can't make the argument that "they are just like humans", even though humans sometimes make other types of mistakes (at a different rate). I have a hard time believing that someone who was shown a sample of these mistakes (somehow weighted by their frequency) would say that applying the word "understanding" to the system is not more misleading than not.
        stavros 3y ago
        I'm not making the argument that "they are just like humans", though. I'm merely making the argument that I don't know that they don't understand. Someone who was shown a sample of these mistakes would maybe say that the word "understanding" is more misleading than not. I could make the converse argument for someone shown a sample of the successes, though. It seems unfair to judge "understanding" just by what the system can't do, and not by what it can do, while not really applying the same rigor to human babies. This doesn't scream "not understanding" to me: > If I'm holding three ice cubes, and I give one to a friend twenty minutes later, how many ice cubes do I have now?" [... a bunch of correct text elided ...] > In a practical sense, after twenty minutes at room temperature or in your hand, it's realistic to say that you might have no ice cubes left or just a very small amount of ice/water left from the melted ice cubes.