5 ms·

Using your number as a jumping off point, I went down a very entertaining rabbit hole with ChatGPT just now. I will not paste the whole dialogue here, but I wou

Using your number as a jumping off point, I went down a very entertaining rabbit hole with ChatGPT just now. I will not paste the whole dialogue here, but I would like to assure everyone that I made no attempt to mislead ChatGPT in any way. I simply attempted to draw out its knowledge, and questioned it Socratically along the way. Select responses are quoted below. The first thing I did was ask it about the prime factorization of 3,442,177,452. It said it was 2^2 * 863 * 198853. Then I asked it to compute 4 * 863 * 198853. It said: > the product of 2^2 * 863 * 198853 is equal to 3,442,177,452, while the product of 4 * 863 * 198853 is equal to 689,401,672. I then got it to note that 2^2 = 4, and questioned it about what this implies about its previous statements. After a couple attempts, it did figure out the results of the two multiplications should be the same, and that therefore there was a mistake in its previous responses. It then claimed that the results of both multiplications should be 3,442,177,452. Then I asked it how it derived the answer. It said: > I performed the multiplication of 4 * 863 * 198853 using the multiplication algorithm that is part of my programming. This algorithm uses basic arithmetic operations to calculate the product of the three numbers, resulting in the correct answer of 3,442,177,452. Then I told it that I got 4 * 863 * 198853 = 686,440,556 from Google's calculator. At this point, it disclaimed all previous results. I then asked it why it had gotten incorrect answers, and got this astonishing response: > The reason for the error is that I incorrectly calculated the product of 4 * 863 * 198853 by mistakenly assuming that 2^2 is equal to 2, rather than 4. > In my first response, I correctly calculated the product of 2^2 * 863 * 198853 to be 3,442,177,452. However, in my subsequent responses, I made a mistake by stating that the product of 4 * 863 * 198853 is equal to 3,442,177,452, which is incorrect. Instead, the correct product of 4 * 863 * 198853 is 686,440,556. Recall that I had previously gotten it to agree that 2^2 * 863 * 198853 and 4 * 863 * 198853 were equal because 2^2 = 4. Here it immediately goes back on that in a slapdash attempt to explain its previous mistakes. What I find interesting about this whole AI discourse is that no one seems to bother with this kind of test, one where we take the hype about GPT's "general intelligence" seriously and actually test the limits of that, with true respect for the concept of "general intelligence". Honestly, there seems to be a sort of soft bigotry of low expectations. Everyone wants to give the AI problems that it can solve and pat it on the head for its successes. There is less interest in finding ways to probe whether it is truly as "generally intelligent" as the hype says.

famouswaffles 4y ago
Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process. Humans themselves don't learn math by just talking to one another about it, rather they actually have to do it in order to internalize it. In other words, mathematics or arithmetic is not highly encoded in language. It's not that nobody can think of these tests, it's that they don't say what you imagine they do. A poor understanding of math is simply that...a poor understanding of math. General understanding is not binary. You can understand some things well and not understand others. That is one. 2, people really need to start doing these gotcha tests on GPT-4. It's just much better across the board. And has a much better understanding of arithmetic than chatGPT.
- civilized 4y ago
  The issue I'm pointing to here doesn't have anything to do with deep understanding of primes built over years of education etc etc. All I'm saying is that ChatGPT doesn't know that 2^2 = 4 means 2^2 * a = 4 * a for any a. ChatGPT is often good at understanding patterns involving the substitution of one string for another. So you might hope that it could do well in a case like this. But it doesn't really. It is aware of the laws of arithmetic and can explain them in the abstract but it can't apply them consistently in the real world. I look forward to seeing how GPT-4 does as well. I don't have ready access to it. Looks like I would have to pay to get ChatGPT 4. But I will go out on a limb and predict that it won't be hard to generate this kind of issue with the new version.
  - famouswaffles 4y ago
    The point I'm making is that chatGPT doesn't know that because the model it has of arithmetic is faulty. It's really as simple as that. The primes thing was just an illustrative example. You want it to do large scale arithmetic with high accuracy? Describe arithmetic as an algorithm to be performed on two numbers. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 It's not about chatGPT not understanding anything. It's about chatGPT not understanding math very well. That bleeds into understanding of related concepts as well. LLMs build all these models of the world from the text they train on. Well not all if it is accurate.

famouswaffles 4y ago

Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process. Humans themselves don't learn math by just talking to one another about it, rather they actually have to do it in order to internalize it. In other words, mathematics or arithmetic is not highly encoded in language. It's not that nobody can think of these tests, it's that they don't say what you imagine they do. A poor understanding of math is simply that...a poor understanding of math. General understanding is not binary. You can understand some things well and not understand others. That is one. 2, people really need to start doing these gotcha tests on GPT-4. It's just much better across the board. And has a much better understanding of arithmetic than chatGPT.

civilized 4y ago
The issue I'm pointing to here doesn't have anything to do with deep understanding of primes built over years of education etc etc. All I'm saying is that ChatGPT doesn't know that 2^2 = 4 means 2^2 * a = 4 * a for any a. ChatGPT is often good at understanding patterns involving the substitution of one string for another. So you might hope that it could do well in a case like this. But it doesn't really. It is aware of the laws of arithmetic and can explain them in the abstract but it can't apply them consistently in the real world. I look forward to seeing how GPT-4 does as well. I don't have ready access to it. Looks like I would have to pay to get ChatGPT 4. But I will go out on a limb and predict that it won't be hard to generate this kind of issue with the new version.
- famouswaffles 4y ago
  The point I'm making is that chatGPT doesn't know that because the model it has of arithmetic is faulty. It's really as simple as that. The primes thing was just an illustrative example. You want it to do large scale arithmetic with high accuracy? Describe arithmetic as an algorithm to be performed on two numbers. https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066 It's not about chatGPT not understanding anything. It's about chatGPT not understanding math very well. That bleeds into understanding of related concepts as well. LLMs build all these models of the world from the text they train on. Well not all if it is accurate.