7 ms·

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to t

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to the square root (which I think is correct but irrelevant), and even though it gave a non zero remainder when dividing the two numbers (I did not check if the remainder is correct). Then it gave a (wrong) factorisation, not even using 58657. It's cool and it will probably be able to get this right one day but it's a big goal to miss.

Etheryte 4y ago
This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.
- foldr 4y ago
  Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.
  - zetalyrae 4y ago
    This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.
    - saurik 4y ago
      Yeah... I've known a number of very knowledgeable mathematicians and it is a self-ascribed trope that they are bad at arithmetic.
    - foldr 4y ago
      Sure, but the system should then be able to show its working and explain how it derived the incorrect result. If there is other evidence that ChatGPT 'understands' the concept of a prime number, then let's see it. If wrong answers still count because humans sometimes make mistakes, then I guess it won’t be too difficult to construct an impressive mathematical AI. It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities. Remember that something as simple as ELIZA was perfectly capable of fooling humans who were predisposed to believe it was intelligent.
      - rain1 4y ago
        > the system should then be able to show its working The system fundamentally cannot do this. You can make it generate text that is like what someone would say when asked to show their working, but that's a different thing. > It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities. I agree. I am seeing a bit too much over-optimistic predictions about these things. And many of these predictions are stated as fact.
      - ben_w 4y ago
        > then I guess it won’t be too difficult to construct an impressive mathematical AI. Yes, we've had Wolfram Alpha for ages. For me, the biggest problem Wolfram Alpha is that it often doesn't understand the questions, and while I also sometimes get that with ChatGPT, the latter is much much better. I've not had a chance to play with the plug-in that connects GPT to Wolfram Alpha. > It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities. This is an excellent and important point. I think people who treat it as already being superhuman in the depth (not merely breadth) of each skill are nearly as wrong as those who treat it as merely a souped-up autocomplete. I've only played with 3 and 3.5 so far, not 4, my impression is that it's somewhere between "a noob" and "genius with Alzheimer's". A noob at everything at the same time, which is weird because asking a human in English to write JavaScript with all the comments in French and following that up with a request written in German for a description of Cartesian dualism written in Chinese is not something that any human would be expected to do well at, but it can do moderately well at. Edit: I should probably explicitly add that by my usage of "noob", most people aren't even that good at most things. I might be able to say 你好 if you are willing to overlook a pronunciation so poor it could easily be 尼好 or 拟好 (both of which I only found out about by asking ChatGPT), so I'm sub-noob at Chinese.
    - rain1 4y ago
      This is a good point. If you ask GPT about much more conceptual advanced mathematics it's actually very good at conversing about this. That said, it does 'hallucinate' falsehoods and it will stick with them once they have been said. etc. You have to double check everything it says if you are on unknown ground with it.
    - glenstein 4y ago
      Right, over the totality of things that it reasons about, to some degree it will make inroads to correctly answering these kinds of questions, and in some ways it'll make errors, and what's interesting, is it'll make errors because the way in which it's attempting to answer them bears a lot of the hallmarks that we associate with conceptual understanding, rather than the mechanical operations of a calculator which truly is blind but always correct. In a way, being wrong can be a better signal of something approximating understanding under the hood. It's like if it was shooting a basketball, and it takes numerous shots, and most of them go in but some of them go out, but even the ones that miss bear the hallmarks of proper shooting form that lead to correct answers. I do think that this specific moment we're going through, in early 2023, is producing some of the most fascinating, confidently incorrect misunderstandings of chatgpt, and I hope that someone is going through these comment sections and collecting them so that we can remark on them 5 to 10 years down the line. I suspect that in time these misunderstandings are going to be comparable to how boomers misunderstand computers and the internet.
      - doctor_eval 4y ago
        Yes, exactly this. I recently asked ChatGPT to write me some Go code. What it wrote was mostly fine, but it incorrectly used a method invocation on an object, instead of the correct code which was to pass the object to a function at the package level (aka static method). I think it's a real stretch to suggest that this could happen as a result of simply regurgitating strings it's seen before, because the string it emitted was not ever going to work. To me, it looked for all the world like the same kind of conceptual error that I have made in Go, and the only way I could see this working is if GPT had a (slightly faulty) model of the Go language, or of the particular (and lesser known) package I was asking about. It's felt more like it "forgot" - like me - how the package worked, so it followed its model instead. That error was WAY more interesting than the code.
        glenstein 4y ago
        Just on the point of forgetting, to pass the time while bored at work I told Chad GPT about my fantasy baseball team, and over the course of the conversation it forgot who my first baseman was but input the name of an entirely different first baseman who I had never mentioned, but who is a real person and who in fact was similar in a lot of ways to my actual first baseman. And sometimes when it would attempt to recall my lineup it would seem to 'forget' certain players even if it remembered them later.
    - civilized 4y ago
      But it's much worse than that for ChatGPT. A person would know that their mental math division of large numbers is unreliable, but if you ask ChatGPT about the reliability of its answers, it is uniformly confident about almost everything. It has no grasp of the difference between the things it "knows" which are true and the things it "knows" which may be false. And this is the error I notice people very consistently making when they evaluate the intelligence of ChatGPT and similar models. They marvel at its ability to produce impressive truths, but think nothing of its complete inability to distinguish these truths from similar-sounding falsehoods. This is another form of the rose-colored glasses, the confirmation bias we are seeing at the peak of the current hype cycle, reminiscent of when Blake Lemoine convinced himself that LaMDA was sentient. A decade ago, techies were dazzling executives with ML models that detected fraud or whatever as if by magic. But then, when the dazzling demos were tempered by the brutal objectivity and rigor of the precision-recall curve, a lot of these models didn't end up getting used in practice. Something similar will happen with ChatGPT. People will eventually have to admit what it is failing to do, and only then will we start up Gartner's fabled Slope of Enlightenment to the Plateau of Productivity.
      - zetalyrae 4y ago
        The failures don't refute the successes. Anything and anyone can fail, you don't get intelligent output by chance. If it makes mistakes we wouldn't make, on obvious things, it is because it is an alien form of intellingece. RLHF and the tokenizer together explain many of the more common failure modes.
        civilized 4y ago
        Nobody is saying the intelligent output is by chance. This is a machine that is fed terabytes of intelligent inputs and is able to produce intelligent outputs. So one explanation of its producing intelligent outputs is that it's basically regurgitating what it was fed. The way to test that, of course, is to give it problems that it hasn't seen. Unfortunately, because GPT has seen so much, giving it problems it definitely hasn't seen is itself now a hard problem. What's been shown so far: OpenAI's benchmarking is not always rigorous or appropriate, and GPT's performance is brittle and highly sensitive to the problem phrasing [1]. I agree with the article that GPT's training enables it to access meaningful conceptual abstractions. But there is clearly quite a lot that's still missing. For now, people are too excited to care. But when they try to deliver on their promises, the gaps will still be there. Hopefully at that point we will embark towards a new level of understanding. [1] https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks https://aisnakeoil.substack.com/p/gpt-4-and-professional-ben...
  - rain1 4y ago
    > Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems There is a causality inversion here. The only reason people know it can't do this is because they have tried and seen it cannot do this. We do not have very precise bounds a-priori what GPT can and cannot do. We only learn them from black box testing.
    - freehorse 4y ago
      This is not completely true, though. We expected GPT and LLMs in general to be bad in tasks that are not language-focused, exactly because they are large LANGUAGE models. Mathematics is one of them. Though it is also true that one may not have expected that GPT would be so bad as it is in extrapolating, nobody should have expected it to any good in any involved mathematical argumentation.
  - bmacho 4y ago
    Or, it has a deep conceptual understanding of its input data, but, it works similar to humans, and during its reasoning it pulls out some facts from memory or hand-waving, which can be wrong. No mathematician would make this exact mistake because it looks ridiculous, but for example I made similar mistakes multiple times, claiming low numbers prime or not prime wrongly during a reasoning. Especially when working at night.
  - rhn_mk1 4y ago
    It's not deriving an understanding of this particular concept. Which is unsurprising given that it was not trained to be good at maths. That in no way precludes deriving deep understanding of some other concepts. Even humans are not expected to be able to learn anything we throw at them.
    - wizofaus 4y ago
      The question is whether an LLM can ever be trained to be good at maths. Currently LLMs can string together tokens that roughly equate to words in order to form sentences that carry particular semantic value. But how much change to the underlying technology would be required to give them the ability to string together digits to represent numbers and then numbers and operators in order to represent equations with particular semantics? I'd suggest it's already able to explain what most common mathematical operations "mean", it just doesn't have the wherewithal to perform them (though it can certainly do a good job of faking it for lower-valued integers that are likely to have occurred in its training data).
      - rhn_mk1 4y ago
        To some limited extent, they are already good at maths: https://ar5iv.labs.arxiv.org/html/2201.02177 https://ar5iv.labs.arxiv.org/html/2201.02177 This paper makes me believe it's less about the lack of ability to understand maths, but the power to learn more of it.
        wizofaus 4y ago
        Is that paper discussing LLMs though? That some sort of neural network can be trained to perform maths doesn't seem surprising and I would expect eventually it'll be a standard expectation they can do so even without being fed masses of training data (ideally no more than a human needs). It's the path from ChatGPT-4 to such a network I'm curious about.
  - barking_biscuit 4y ago
    >Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data. I don't think it really makes the case that you think it makes. Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process. Humans themselves don't learn math by just talking to one another about it, rather they actually have to do it in order to internalize it. I suspect it's simply a case of we haven't sent ChatGPT to highschool yet. There will be some particular trick in terms of training methodology that trains the network to perform that more specialized analysis that doesn't simply emerge out of 'predict the next token from all this random internet text' as it's likely severely underrepresented in the training data in the first place, but I think it ought to be perfectly possible.
    - famouswaffles 4y ago
      You can increase the accuracy of arithmetic by simply describing arithmetic (addition, subtraction etc) as an algorithm to be performed on the numbers. 98.5% accuracy on addition arithmetic this way with GPT-3 https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066
- freehorse 4y ago
  Interfacing with APIs does not solve any fundamental issues about what LLMs "understand" or not though. At most they will be more accurate in arithmetic tasks, but that's it.
  - meghan_rain 4y ago
    Did you "understand" that 58657 is/is not a factor of 3442177452? Or did you use a calculator to verify? Because that would be exactly what ChatGPT with plugins does.
- ccppurcell 4y ago
  I wasn't asking for complex mathematics, just for the basic understanding that an even number greater than 2 can't be prime.
zetalyrae 4y ago
There is no reason to expect it to be good at mental arithmetic.
rain1 4y ago
I think what would be even better than the next GPT being able to get that question right, is for it to correctly identify that it cannot solve this problem itself.
civilized 4y ago
Using your number as a jumping off point, I went down a very entertaining rabbit hole with ChatGPT just now. I will not paste the whole dialogue here, but I would like to assure everyone that I made no attempt to mislead ChatGPT in any way. I simply attempted to draw out its knowledge, and questioned it Socratically along the way. Select responses are quoted below. The first thing I did was ask it about the prime factorization of 3,442,177,452. It said it was 2^2 * 863 * 198853. Then I asked it to compute 4 * 863 * 198853. It said: > the product of 2^2 * 863 * 198853 is equal to 3,442,177,452, while the product of 4 * 863 * 198853 is equal to 689,401,672. I then got it to note that 2^2 = 4, and questioned it about what this implies about its previous statements. After a couple attempts, it did figure out the results of the two multiplications should be the same, and that therefore there was a mistake in its previous responses. It then claimed that the results of both multiplications should be 3,442,177,452. Then I asked it how it derived the answer. It said: > I performed the multiplication of 4 * 863 * 198853 using the multiplication algorithm that is part of my programming. This algorithm uses basic arithmetic operations to calculate the product of the three numbers, resulting in the correct answer of 3,442,177,452. Then I told it that I got 4 * 863 * 198853 = 686,440,556 from Google's calculator. At this point, it disclaimed all previous results. I then asked it why it had gotten incorrect answers, and got this astonishing response: > The reason for the error is that I incorrectly calculated the product of 4 * 863 * 198853 by mistakenly assuming that 2^2 is equal to 2, rather than 4. > In my first response, I correctly calculated the product of 2^2 * 863 * 198853 to be 3,442,177,452. However, in my subsequent responses, I made a mistake by stating that the product of 4 * 863 * 198853 is equal to 3,442,177,452, which is incorrect. Instead, the correct product of 4 * 863 * 198853 is 686,440,556. Recall that I had previously gotten it to agree that 2^2 * 863 * 198853 and 4 * 863 * 198853 were equal because 2^2 = 4. Here it immediately goes back on that in a slapdash attempt to explain its previous mistakes. What I find interesting about this whole AI discourse is that no one seems to bother with this kind of test, one where we take the hype about GPT's "general intelligence" seriously and actually test the limits of that, with true respect for the concept of "general intelligence". Honestly, there seems to be a sort of soft bigotry of low expectations. Everyone wants to give the AI problems that it can solve and pat it on the head for its successes. There is less interest in finding ways to probe whether it is truly as "generally intelligent" as the hype says.
perryizgr8 4y ago
If you ask that question to humans most wouldn't be able to solve it. Even with access to a calculator. Not being able to do it just means it's not good at maths.
sdifnhiono 4y ago
These LLMs are trained to predict text. "Fancy autocomplete" is not just a dismissal. It is the fundamental nature of these things. It is an autocomplete tool hacked together to emulate an assistant. "I don't know" is clearly a truer answer than the nonsense ChatGPT made up, but that doesn't mean it's a better answer according to whatever non-goals this machine has. As always, anthropomorphize an LLM at your peril. These things are not people. No one understands how they work.

Etheryte 4y ago

This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.

foldr 4y ago
Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.
- zetalyrae 4y ago
  This does not follow. You can know a lot about, say, number theory, and still make elementary arithmetic errors.
  - saurik 4y ago
    Yeah... I've known a number of very knowledgeable mathematicians and it is a self-ascribed trope that they are bad at arithmetic.
  - foldr 4y ago
    Sure, but the system should then be able to show its working and explain how it derived the incorrect result. If there is other evidence that ChatGPT 'understands' the concept of a prime number, then let's see it. If wrong answers still count because humans sometimes make mistakes, then I guess it won’t be too difficult to construct an impressive mathematical AI. It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities. Remember that something as simple as ELIZA was perfectly capable of fooling humans who were predisposed to believe it was intelligent.
    - rain1 4y ago
      > the system should then be able to show its working The system fundamentally cannot do this. You can make it generate text that is like what someone would say when asked to show their working, but that's a different thing. > It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities. I agree. I am seeing a bit too much over-optimistic predictions about these things. And many of these predictions are stated as fact.
    - ben_w 4y ago
      > then I guess it won’t be too difficult to construct an impressive mathematical AI. Yes, we've had Wolfram Alpha for ages. For me, the biggest problem Wolfram Alpha is that it often doesn't understand the questions, and while I also sometimes get that with ChatGPT, the latter is much much better. I've not had a chance to play with the plug-in that connects GPT to Wolfram Alpha. > It's very tempting to give these systems the benefit of the doubt, but that tends to lead to hugely inflated conclusions about their capabilities. This is an excellent and important point. I think people who treat it as already being superhuman in the depth (not merely breadth) of each skill are nearly as wrong as those who treat it as merely a souped-up autocomplete. I've only played with 3 and 3.5 so far, not 4, my impression is that it's somewhere between "a noob" and "genius with Alzheimer's". A noob at everything at the same time, which is weird because asking a human in English to write JavaScript with all the comments in French and following that up with a request written in German for a description of Cartesian dualism written in Chinese is not something that any human would be expected to do well at, but it can do moderately well at. Edit: I should probably explicitly add that by my usage of "noob", most people aren't even that good at most things. I might be able to say 你好 if you are willing to overlook a pronunciation so poor it could easily be 尼好 or 拟好 (both of which I only found out about by asking ChatGPT), so I'm sub-noob at Chinese.
  - rain1 4y ago
    This is a good point. If you ask GPT about much more conceptual advanced mathematics it's actually very good at conversing about this. That said, it does 'hallucinate' falsehoods and it will stick with them once they have been said. etc. You have to double check everything it says if you are on unknown ground with it.
  - glenstein 4y ago
    Right, over the totality of things that it reasons about, to some degree it will make inroads to correctly answering these kinds of questions, and in some ways it'll make errors, and what's interesting, is it'll make errors because the way in which it's attempting to answer them bears a lot of the hallmarks that we associate with conceptual understanding, rather than the mechanical operations of a calculator which truly is blind but always correct. In a way, being wrong can be a better signal of something approximating understanding under the hood. It's like if it was shooting a basketball, and it takes numerous shots, and most of them go in but some of them go out, but even the ones that miss bear the hallmarks of proper shooting form that lead to correct answers. I do think that this specific moment we're going through, in early 2023, is producing some of the most fascinating, confidently incorrect misunderstandings of chatgpt, and I hope that someone is going through these comment sections and collecting them so that we can remark on them 5 to 10 years down the line. I suspect that in time these misunderstandings are going to be comparable to how boomers misunderstand computers and the internet.
    - doctor_eval 4y ago
      Yes, exactly this. I recently asked ChatGPT to write me some Go code. What it wrote was mostly fine, but it incorrectly used a method invocation on an object, instead of the correct code which was to pass the object to a function at the package level (aka static method). I think it's a real stretch to suggest that this could happen as a result of simply regurgitating strings it's seen before, because the string it emitted was not ever going to work. To me, it looked for all the world like the same kind of conceptual error that I have made in Go, and the only way I could see this working is if GPT had a (slightly faulty) model of the Go language, or of the particular (and lesser known) package I was asking about. It's felt more like it "forgot" - like me - how the package worked, so it followed its model instead. That error was WAY more interesting than the code.
      - glenstein 4y ago
        Just on the point of forgetting, to pass the time while bored at work I told Chad GPT about my fantasy baseball team, and over the course of the conversation it forgot who my first baseman was but input the name of an entirely different first baseman who I had never mentioned, but who is a real person and who in fact was similar in a lot of ways to my actual first baseman. And sometimes when it would attempt to recall my lineup it would seem to 'forget' certain players even if it remembered them later.
  - civilized 4y ago
    But it's much worse than that for ChatGPT. A person would know that their mental math division of large numbers is unreliable, but if you ask ChatGPT about the reliability of its answers, it is uniformly confident about almost everything. It has no grasp of the difference between the things it "knows" which are true and the things it "knows" which may be false. And this is the error I notice people very consistently making when they evaluate the intelligence of ChatGPT and similar models. They marvel at its ability to produce impressive truths, but think nothing of its complete inability to distinguish these truths from similar-sounding falsehoods. This is another form of the rose-colored glasses, the confirmation bias we are seeing at the peak of the current hype cycle, reminiscent of when Blake Lemoine convinced himself that LaMDA was sentient. A decade ago, techies were dazzling executives with ML models that detected fraud or whatever as if by magic. But then, when the dazzling demos were tempered by the brutal objectivity and rigor of the precision-recall curve, a lot of these models didn't end up getting used in practice. Something similar will happen with ChatGPT. People will eventually have to admit what it is failing to do, and only then will we start up Gartner's fabled Slope of Enlightenment to the Plateau of Productivity.
    - zetalyrae 4y ago
      The failures don't refute the successes. Anything and anyone can fail, you don't get intelligent output by chance. If it makes mistakes we wouldn't make, on obvious things, it is because it is an alien form of intellingece. RLHF and the tokenizer together explain many of the more common failure modes.
      - civilized 4y ago
        Nobody is saying the intelligent output is by chance. This is a machine that is fed terabytes of intelligent inputs and is able to produce intelligent outputs. So one explanation of its producing intelligent outputs is that it's basically regurgitating what it was fed. The way to test that, of course, is to give it problems that it hasn't seen. Unfortunately, because GPT has seen so much, giving it problems it definitely hasn't seen is itself now a hard problem. What's been shown so far: OpenAI's benchmarking is not always rigorous or appropriate, and GPT's performance is brittle and highly sensitive to the problem phrasing [1]. I agree with the article that GPT's training enables it to access meaningful conceptual abstractions. But there is clearly quite a lot that's still missing. For now, people are too excited to care. But when they try to deliver on their promises, the gaps will still be there. Hopefully at that point we will embark towards a new level of understanding. [1] https://aisnakeoil.substack.com/p/gpt-4-and-professional-benchmarks https://aisnakeoil.substack.com/p/gpt-4-and-professional-ben...
- rain1 4y ago
  > Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems There is a causality inversion here. The only reason people know it can't do this is because they have tried and seen it cannot do this. We do not have very precise bounds a-priori what GPT can and cannot do. We only learn them from black box testing.
  - freehorse 4y ago
    This is not completely true, though. We expected GPT and LLMs in general to be bad in tasks that are not language-focused, exactly because they are large LANGUAGE models. Mathematics is one of them. Though it is also true that one may not have expected that GPT would be so bad as it is in extrapolating, nobody should have expected it to any good in any involved mathematical argumentation.
- bmacho 4y ago
  Or, it has a deep conceptual understanding of its input data, but, it works similar to humans, and during its reasoning it pulls out some facts from memory or hand-waving, which can be wrong. No mathematician would make this exact mistake because it looks ridiculous, but for example I made similar mistakes multiple times, claiming low numbers prime or not prime wrongly during a reasoning. Especially when working at night.
- rhn_mk1 4y ago
  It's not deriving an understanding of this particular concept. Which is unsurprising given that it was not trained to be good at maths. That in no way precludes deriving deep understanding of some other concepts. Even humans are not expected to be able to learn anything we throw at them.
  - wizofaus 4y ago
    The question is whether an LLM can ever be trained to be good at maths. Currently LLMs can string together tokens that roughly equate to words in order to form sentences that carry particular semantic value. But how much change to the underlying technology would be required to give them the ability to string together digits to represent numbers and then numbers and operators in order to represent equations with particular semantics? I'd suggest it's already able to explain what most common mathematical operations "mean", it just doesn't have the wherewithal to perform them (though it can certainly do a good job of faking it for lower-valued integers that are likely to have occurred in its training data).
    - rhn_mk1 4y ago
      To some limited extent, they are already good at maths: https://ar5iv.labs.arxiv.org/html/2201.02177 https://ar5iv.labs.arxiv.org/html/2201.02177 This paper makes me believe it's less about the lack of ability to understand maths, but the power to learn more of it.
      - wizofaus 4y ago
        Is that paper discussing LLMs though? That some sort of neural network can be trained to perform maths doesn't seem surprising and I would expect eventually it'll be a standard expectation they can do so even without being fed masses of training data (ideally no more than a human needs). It's the path from ChatGPT-4 to such a network I'm curious about.
- barking_biscuit 4y ago
  >Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data. I don't think it really makes the case that you think it makes. Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process. Humans themselves don't learn math by just talking to one another about it, rather they actually have to do it in order to internalize it. I suspect it's simply a case of we haven't sent ChatGPT to highschool yet. There will be some particular trick in terms of training methodology that trains the network to perform that more specialized analysis that doesn't simply emerge out of 'predict the next token from all this random internet text' as it's likely severely underrepresented in the training data in the first place, but I think it ought to be perfectly possible.
  - famouswaffles 4y ago
    You can increase the accuracy of arithmetic by simply describing arithmetic (addition, subtraction etc) as an algorithm to be performed on the numbers. 98.5% accuracy on addition arithmetic this way with GPT-3 https://arxiv.org/abs/2211.09066 https://arxiv.org/abs/2211.09066
freehorse 4y ago
Interfacing with APIs does not solve any fundamental issues about what LLMs "understand" or not though. At most they will be more accurate in arithmetic tasks, but that's it.
- meghan_rain 4y ago
  Did you "understand" that 58657 is/is not a factor of 3442177452? Or did you use a calculator to verify? Because that would be exactly what ChatGPT with plugins does.
ccppurcell 4y ago
I wasn't asking for complex mathematics, just for the basic understanding that an even number greater than 2 can't be prime.

zetalyrae 4y ago

There is no reason to expect it to be good at mental arithmetic.

rain1 4y ago

I think what would be even better than the next GPT being able to get that question right, is for it to correctly identify that it cannot solve this problem itself.

civilized 4y ago

Using your number as a jumping off point, I went down a very entertaining rabbit hole with ChatGPT just now. I will not paste the whole dialogue here, but I would like to assure everyone that I made no attempt to mislead ChatGPT in any way. I simply attempted to draw out its knowledge, and questioned it Socratically along the way. Select responses are quoted below. The first thing I did was ask it about the prime factorization of 3,442,177,452. It said it was 2^2 * 863 * 198853. Then I asked it to compute 4 * 863 * 198853. It said: > the product of 2^2 * 863 * 198853 is equal to 3,442,177,452, while the product of 4 * 863 * 198853 is equal to 689,401,672. I then got it to note that 2^2 = 4, and questioned it about what this implies about its previous statements. After a couple attempts, it did figure out the results of the two multiplications should be the same, and that therefore there was a mistake in its previous responses. It then claimed that the results of both multiplications should be 3,442,177,452. Then I asked it how it derived the answer. It said: > I performed the multiplication of 4 * 863 * 198853 using the multiplication algorithm that is part of my programming. This algorithm uses basic arithmetic operations to calculate the product of the three numbers, resulting in the correct answer of 3,442,177,452. Then I told it that I got 4 * 863 * 198853 = 686,440,556 from Google's calculator. At this point, it disclaimed all previous results. I then asked it why it had gotten incorrect answers, and got this astonishing response: > The reason for the error is that I incorrectly calculated the product of 4 * 863 * 198853 by mistakenly assuming that 2^2 is equal to 2, rather than 4. > In my first response, I correctly calculated the product of 2^2 * 863 * 198853 to be 3,442,177,452. However, in my subsequent responses, I made a mistake by stating that the product of 4 * 863 * 198853 is equal to 3,442,177,452, which is incorrect. Instead, the correct product of 4 * 863 * 198853 is 686,440,556. Recall that I had previously gotten it to agree that 2^2 * 863 * 198853 and 4 * 863 * 198853 were equal because 2^2 = 4. Here it immediately goes back on that in a slapdash attempt to explain its previous mistakes. What I find interesting about this whole AI discourse is that no one seems to bother with this kind of test, one where we take the hype about GPT's "general intelligence" seriously and actually test the limits of that, with true respect for the concept of "general intelligence". Honestly, there seems to be a sort of soft bigotry of low expectations. Everyone wants to give the AI problems that it can solve and pat it on the head for its successes. There is less interest in finding ways to probe whether it is truly as "generally intelligent" as the hype says.

perryizgr8 4y ago

If you ask that question to humans most wouldn't be able to solve it. Even with access to a calculator. Not being able to do it just means it's not good at maths.

sdifnhiono 4y ago

These LLMs are trained to predict text. "Fancy autocomplete" is not just a dismissal. It is the fundamental nature of these things. It is an autocomplete tool hacked together to emulate an assistant. "I don't know" is clearly a truer answer than the nonsense ChatGPT made up, but that doesn't mean it's a better answer according to whatever non-goals this machine has. As always, anthropomorphize an LLM at your peril. These things are not people. No one understands how they work.