5 ms·

At least 40 years, these models just string words together without understanding their meaning or relationship

by ilostmyshoes 4y ago

FeepingCreature 4y ago
Seems like an overreach. It clearly has some understanding, just not a very deep one.
- ilostmyshoes 4y ago
  Understanding basic grammar isn't even close to understanding scientific concepts and knowing how to connect them
  - tiborsaas 4y ago
    It took an astrophysics exam and did quite well in my opinion. It's worth watching this video: https://www.youtube.com/watch?v=K0cmmKPklp4 https://www.youtube.com/watch?v=K0cmmKPklp4 For human students, the median score was 75.6%, ChatGPT performed slightly worse than a Columbia non-science major undergraduate, 73.9%. Some of the errors was just messing up the arithmetics but it got the reasoning right.
    - beepbooptheory 4y ago
      Yeah but that test is for humans, its like saying a tiger is a pretty good wrestler because it did well in the wrestling ring, instead of just understanding that a tiger is big and brutal, which just happens to coincide with the sport. The tiger needs its own sport/test.
      - tiborsaas 4y ago
        I'm not sure what your analogy proves, I wouldn't want to fight a tiger with bare hands. I agree, that humans and machines are not the same, but when it comes to intelligence, we are are in the same sport: answering questions in plain text. The exam was a reply for this part: > isn't even close to understanding scientific concepts and knowing how to connect them That test in my book definitely proves that it can understand (or let's say process and create abstractions) scientific concepts and can place them on a knowledge graph.
        beepbooptheory 4y ago
        I'm just saying its not the same test for the model vs the human. The human has to remember stuff, understand stuff in their limited human way of understanding, which the model does not. So what is really the utility of "testing" it? You don't take tests to become a better test taker, to prove you can pass the test. A test is more a tool to use to try and gauge our understanding of things, its not something itself. You're logic here is extremely flawed.
        tiborsaas 4y ago
        We obviously work in different ways, but how it comes up with the answer doesn't really matter. > So what is really the utility of "testing" it? Eventually these models will be integrated into everyday products so we need to learn how trustworthy they are. These tests are written in natural language, the same way everybody interacts with them.
        beepbooptheory 4y ago
        It absolutely matters if you're going to say it passing a human test speaks to anything at all! If humans had a hardcoded multiplication table in their brain, it would be silly to "test" them on their multiplication ability, it would be pointless. Test's are useful precisely to gauge human retention/understanding, but they are even then a crude tool as I think anyone whose taken a test would agree. It borders on absurd to take it seriously for a model. Its like applauding the calculator for doing math. If you really do think this stuff is important, show it on its own terms. Make tests suited for the model with rigorous expectations tuned for a computer, not a human. Then you can start showing some impressive things. I just urge you to see the basic flaw in your conceptions here. I don't want to rob you of your enthusiasm, just better direct it.
        FeepingCreature 4y ago
        If you wanted to hire people to multiply numbers, it would be very worthwhile to test the functioning of the hardcoded multiplication table, or whatever other implementation they had. Nobody is saying that GPT-3 is smart. But it is surprisingly capable.
    - WaitWaitWha 4y ago
      from your reference video (thanks): > 7:36 The exam is structured with multiple choice questions at the start and then some freeform short answers towards the end. Imagine if you could look up all material in the last 10 years published about astrophysics (or insert any major topic here) near instantly, the question's details are also presented for ingestion, then make prediction what is and is not based on weighing the frequency and potentially the source of information, and then respond in relatively coherent sentences. There is no astrophysics in this.
      - tiborsaas 4y ago
        ChatGPT adds explanations why it picked a certain answer. But you are right in a sense, this checks for existing knowledge, It probably couldn't come up with a solution to an open problem.
        FeepingCreature 4y ago
        Minor correction: ChatGPT does not add explanations why it picked a certain answer; ChatGPT adds explanations that it thinks fit with the answer it has already given. The what predates the why. That's why, if you want ChatGPT to actually think about a problem, you have to get it to give the explanation first, gradually, and only then output the answer. That way, GPT can use the rationale it has given in the prediction of the answer that fits best. If it has to give the answer first, it's already committed. If a human has to answer a question, they follow a process like this: - parse the question - compute a rationale, and evaluate it to gain an answer - output the answer first, because it's the most useful - output the cached rationale. But GPT trains left to right, and it cannot see the first two steps as they're not in the text. From its perspective, most humans produce answers and explanations simultaneously, instantaneously and intuitively. This makes it almost impossible for GPT to learn how to think. It has to rely on the very few samples where humans give the rationale first, and then the answer. And because this is rare, you have to really push GPT to actually use its scant cognitive abilities. ("Let's think about it step by step.")
- scotty79 4y ago
  It's a language model, so it's good at generating language. Any congruency with knowledge of what the language expresses is purely accidental side effect.
  - FeepingCreature 4y ago
    Surely human language use (GPT-3's training domain) has some connection to reality, however faint.
    - scotty79 4y ago
      Yeah. That connection is the source of this accidental side effect.
      - FeepingCreature 4y ago
        That seems a stretch to call an "accidental side effect" though. It's an effect exactly inasmuch as human language use is inextricable from reality. Put otherwise, GPT doesn't interact with reality directly, but mediated through our language. But we also don't experience reality directly, so it's hard to see how much of a hindrance that would pose. Hell, much of our model of the world is just as rooted in language as GPT's. The part of my model of reality of which I have direct sensory experience is tiny.
        scotty79 4y ago
        > Put otherwise, GPT doesn't interact with reality directly, but mediated through our language. Yes, but the human language is garbage. So while passing throught that filter without maintaining the highest degree of precision barely any of the actual reality remains. Truth sounds exactly the same as a lie. The actual language that decently captures reality is math. Nothing less is even barely sufficient to produce almost anything but nonsense. Nonsense nicely sounding to human ears but still nonsense.
  - MayeulC 4y ago
    Not necessarily. Training pushes language models towards generalization and "understanding" of concepts. It certainly "knows" some stuff. However it can't reason about it. There have been some articles on HN recently on how accurately LMs build world models.

At least 40 years, these models just string words together without understanding their meaning or relationship

FeepingCreature 4y ago

Seems like an overreach. It clearly has some understanding, just not a very deep one.

ilostmyshoes 4y ago
Understanding basic grammar isn't even close to understanding scientific concepts and knowing how to connect them
- tiborsaas 4y ago
  It took an astrophysics exam and did quite well in my opinion. It's worth watching this video: https://www.youtube.com/watch?v=K0cmmKPklp4 https://www.youtube.com/watch?v=K0cmmKPklp4 For human students, the median score was 75.6%, ChatGPT performed slightly worse than a Columbia non-science major undergraduate, 73.9%. Some of the errors was just messing up the arithmetics but it got the reasoning right.
  - beepbooptheory 4y ago
    Yeah but that test is for humans, its like saying a tiger is a pretty good wrestler because it did well in the wrestling ring, instead of just understanding that a tiger is big and brutal, which just happens to coincide with the sport. The tiger needs its own sport/test.
    - tiborsaas 4y ago
      I'm not sure what your analogy proves, I wouldn't want to fight a tiger with bare hands. I agree, that humans and machines are not the same, but when it comes to intelligence, we are are in the same sport: answering questions in plain text. The exam was a reply for this part: > isn't even close to understanding scientific concepts and knowing how to connect them That test in my book definitely proves that it can understand (or let's say process and create abstractions) scientific concepts and can place them on a knowledge graph.
      - beepbooptheory 4y ago
        I'm just saying its not the same test for the model vs the human. The human has to remember stuff, understand stuff in their limited human way of understanding, which the model does not. So what is really the utility of "testing" it? You don't take tests to become a better test taker, to prove you can pass the test. A test is more a tool to use to try and gauge our understanding of things, its not something itself. You're logic here is extremely flawed.
        tiborsaas 4y ago
        We obviously work in different ways, but how it comes up with the answer doesn't really matter. > So what is really the utility of "testing" it? Eventually these models will be integrated into everyday products so we need to learn how trustworthy they are. These tests are written in natural language, the same way everybody interacts with them.
        beepbooptheory 4y ago
        It absolutely matters if you're going to say it passing a human test speaks to anything at all! If humans had a hardcoded multiplication table in their brain, it would be silly to "test" them on their multiplication ability, it would be pointless. Test's are useful precisely to gauge human retention/understanding, but they are even then a crude tool as I think anyone whose taken a test would agree. It borders on absurd to take it seriously for a model. Its like applauding the calculator for doing math. If you really do think this stuff is important, show it on its own terms. Make tests suited for the model with rigorous expectations tuned for a computer, not a human. Then you can start showing some impressive things. I just urge you to see the basic flaw in your conceptions here. I don't want to rob you of your enthusiasm, just better direct it.
        FeepingCreature 4y ago
        If you wanted to hire people to multiply numbers, it would be very worthwhile to test the functioning of the hardcoded multiplication table, or whatever other implementation they had. Nobody is saying that GPT-3 is smart. But it is surprisingly capable.
  - WaitWaitWha 4y ago
    from your reference video (thanks): > 7:36 The exam is structured with multiple choice questions at the start and then some freeform short answers towards the end. Imagine if you could look up all material in the last 10 years published about astrophysics (or insert any major topic here) near instantly, the question's details are also presented for ingestion, then make prediction what is and is not based on weighing the frequency and potentially the source of information, and then respond in relatively coherent sentences. There is no astrophysics in this.
    - tiborsaas 4y ago
      ChatGPT adds explanations why it picked a certain answer. But you are right in a sense, this checks for existing knowledge, It probably couldn't come up with a solution to an open problem.
      - FeepingCreature 4y ago
        Minor correction: ChatGPT does not add explanations why it picked a certain answer; ChatGPT adds explanations that it thinks fit with the answer it has already given. The what predates the why. That's why, if you want ChatGPT to actually think about a problem, you have to get it to give the explanation first, gradually, and only then output the answer. That way, GPT can use the rationale it has given in the prediction of the answer that fits best. If it has to give the answer first, it's already committed. If a human has to answer a question, they follow a process like this: - parse the question - compute a rationale, and evaluate it to gain an answer - output the answer first, because it's the most useful - output the cached rationale. But GPT trains left to right, and it cannot see the first two steps as they're not in the text. From its perspective, most humans produce answers and explanations simultaneously, instantaneously and intuitively. This makes it almost impossible for GPT to learn how to think. It has to rely on the very few samples where humans give the rationale first, and then the answer. And because this is rare, you have to really push GPT to actually use its scant cognitive abilities. ("Let's think about it step by step.")
scotty79 4y ago
It's a language model, so it's good at generating language. Any congruency with knowledge of what the language expresses is purely accidental side effect.
- FeepingCreature 4y ago
  Surely human language use (GPT-3's training domain) has some connection to reality, however faint.
  - scotty79 4y ago
    Yeah. That connection is the source of this accidental side effect.
    - FeepingCreature 4y ago
      That seems a stretch to call an "accidental side effect" though. It's an effect exactly inasmuch as human language use is inextricable from reality. Put otherwise, GPT doesn't interact with reality directly, but mediated through our language. But we also don't experience reality directly, so it's hard to see how much of a hindrance that would pose. Hell, much of our model of the world is just as rooted in language as GPT's. The part of my model of reality of which I have direct sensory experience is tiny.
      - scotty79 4y ago
        > Put otherwise, GPT doesn't interact with reality directly, but mediated through our language. Yes, but the human language is garbage. So while passing throught that filter without maintaining the highest degree of precision barely any of the actual reality remains. Truth sounds exactly the same as a lie. The actual language that decently captures reality is math. Nothing less is even barely sufficient to produce almost anything but nonsense. Nonsense nicely sounding to human ears but still nonsense.
- MayeulC 4y ago
  Not necessarily. Training pushes language models towards generalization and "understanding" of concepts. It certainly "knows" some stuff. However it can't reason about it. There have been some articles on HN recently on how accurately LMs build world models.