3 ms·

Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the sam

Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process. Humans themselves don't learn math by just talking to one another about it, rather they actually have to do it in order to internalize it. In other words, mathematics or arithmetic is not highly encoded in language. It's not that nobody can think of these tests, it's that they don't say what you imagine they do. A poor understanding of math is simply that...a poor understanding of math. General understanding is not binary. You can understand some things well and not understand others. That is one. 2, people really need to start doing these gotcha tests on GPT-4. It's just much better across the board. And has a much better understanding of arithmetic than chatGPT.

civilized 4y ago
The issue I'm pointing to here doesn't have anything to do with deep understanding of primes built over years of education etc etc. All I'm saying is that ChatGPT doesn't know that 2^2 = 4 means 2^2 * a = 4 * a for any a. ChatGPT is often good at understanding patterns involving the substitution of one string for another. So you might hope that it could do well in a case like this. But it doesn't really. It is aware of the laws of arithmetic and can explain them in the abstract but it can't apply them consistently in the real world. I look forward to seeing how GPT-4 does as well. I don't have ready access to it. Looks like I would have to pay to get ChatGPT 4. But I will go out on a limb and predict that it won't be hard to generate this kind of issue with the new version.
- famouswaffles 4y ago
  The point I'm making is that chatGPT doesn't know that because the model it has of arithmetic is faulty. It's really as simple as that. The primes thing was just an illustrative example. You want it to do large scale arithmetic with high accuracy? Describe arithmetic as an algorithm to be performed on two numbers. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 It's not about chatGPT not understanding anything. It's about chatGPT not understanding math very well. That bleeds into understanding of related concepts as well. LLMs build all these models of the world from the text they train on. Well not all if it is accurate.
  - civilized 4y ago
    I get what you're saying. I wonder if you are fully following through the implications. If GPT were generally intelligent, we shouldn't need to devote a special research project to teaching it math. We could just throw a math textbook at it, explanations and worked examples, and it would figure it out from there. Almost certainly its training data contains a great deal of such material already. That this doesn't work suggests its mental architecture is insufficient to grasp what it's been told. (Note that people are not advantaged with any specialized symbolic representation of numbers, like the integer data type a computer has. We manipulate numerical symbols as text, same as the AI.) It's all well and good that it can improve when a special effort is made, but it sounds like even with that special effort, it still doesn't show the level of competence one would expect from a human-level intelligence with access to virtually infinite, untiring silicon computational resources. GPT has access to abundant materials to learn the laws of arithmetic from, and it can tell you what they are (because it memorizes everything) but it isn't really understanding what it's learned. That points to a shortcoming of the architecture that won't be solved by merely throwing more data at it.
    - famouswaffles 4y ago
      Are humans not generally intelligent ?. Since when has been the answer to "not good at math" been chuck a textbook at it ? You would have limited success doing this with people. Do people not explain things they don't fully understand? Understanding is not binary. This is kind of problem I keep seeing. Expectations and post shifting have grown so much that a significant chunk of the human population wouldn't even pass so called General Intelligence requirements.
      - civilized 4y ago
        I learned arithmetic laws from a textbook and practice problems. I didn't need a teacher gesticulating or any fancy multimodal stuff. It's symbolic manipulation. What is the machine missing that I was given? There's no post shifting. The research community has been setting itself realistically attainable benchmarks. Now that the research community has made a lot of progress against its benchmarks, we have hype, claims of general intelligence. Which attracts people like me, who compare the hype to actual performance. And as I said elsewhere, the performance of GPT on the questions I posed is only comparable to a human with a severe traumatic brain injury.

civilized 4y ago

The issue I'm pointing to here doesn't have anything to do with deep understanding of primes built over years of education etc etc. All I'm saying is that ChatGPT doesn't know that 2^2 = 4 means 2^2 * a = 4 * a for any a. ChatGPT is often good at understanding patterns involving the substitution of one string for another. So you might hope that it could do well in a case like this. But it doesn't really. It is aware of the laws of arithmetic and can explain them in the abstract but it can't apply them consistently in the real world. I look forward to seeing how GPT-4 does as well. I don't have ready access to it. Looks like I would have to pay to get ChatGPT 4. But I will go out on a limb and predict that it won't be hard to generate this kind of issue with the new version.

famouswaffles 4y ago
The point I'm making is that chatGPT doesn't know that because the model it has of arithmetic is faulty. It's really as simple as that. The primes thing was just an illustrative example. You want it to do large scale arithmetic with high accuracy? Describe arithmetic as an algorithm to be performed on two numbers. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 It's not about chatGPT not understanding anything. It's about chatGPT not understanding math very well. That bleeds into understanding of related concepts as well. LLMs build all these models of the world from the text they train on. Well not all if it is accurate.
- civilized 4y ago
  I get what you're saying. I wonder if you are fully following through the implications. If GPT were generally intelligent, we shouldn't need to devote a special research project to teaching it math. We could just throw a math textbook at it, explanations and worked examples, and it would figure it out from there. Almost certainly its training data contains a great deal of such material already. That this doesn't work suggests its mental architecture is insufficient to grasp what it's been told. (Note that people are not advantaged with any specialized symbolic representation of numbers, like the integer data type a computer has. We manipulate numerical symbols as text, same as the AI.) It's all well and good that it can improve when a special effort is made, but it sounds like even with that special effort, it still doesn't show the level of competence one would expect from a human-level intelligence with access to virtually infinite, untiring silicon computational resources. GPT has access to abundant materials to learn the laws of arithmetic from, and it can tell you what they are (because it memorizes everything) but it isn't really understanding what it's learned. That points to a shortcoming of the architecture that won't be solved by merely throwing more data at it.
  - famouswaffles 4y ago
    Are humans not generally intelligent ?. Since when has been the answer to "not good at math" been chuck a textbook at it ? You would have limited success doing this with people. Do people not explain things they don't fully understand? Understanding is not binary. This is kind of problem I keep seeing. Expectations and post shifting have grown so much that a significant chunk of the human population wouldn't even pass so called General Intelligence requirements.
    - civilized 4y ago
      I learned arithmetic laws from a textbook and practice problems. I didn't need a teacher gesticulating or any fancy multimodal stuff. It's symbolic manipulation. What is the machine missing that I was given? There's no post shifting. The research community has been setting itself realistically attainable benchmarks. Now that the research community has made a lot of progress against its benchmarks, we have hype, claims of general intelligence. Which attracts people like me, who compare the hype to actual performance. And as I said elsewhere, the performance of GPT on the questions I posed is only comparable to a human with a severe traumatic brain injury.