5 ms·

Last time I checked ChatGPT was unable to solve basic elementary school math problems.

by mik1998 4y ago

ben_w 4y ago
I just asked it to give me 20 and then solve them. Either it is overestimating what elementary school teaches, or I'm underestimating the same: ~~~~ Sure! Here are the solutions to the problems: 1. Solve for x: 3x + 4 = 10 Solution: x = 2 2. Simplify: (5 + 3) * 2 Solution: 16 3. Convert: 3/4 cups to tablespoons Solution: 6 tablespoons 4. Round: 8.6 to the nearest whole number Solution: 9 5. Find the area of a rectangle with length 5 and width 3 Solution: 15 6. Find the perimeter of a triangle with sides of length 4, 5, and 6 Solution: 15 7. Solve for y: x + y = 8, x = 3 Solution: y = 5 8. Simplify: (6 - 2)^2 Solution: 16 9. Find the volume of a cube with side length 5 Solution: 125 10. Find the circumference of a circle with radius 3 Solution: 18.85 11. Simplify: 1/2 + 1/3 Solution: 5/6 12. Find the slope of the line with points (2, 3) and (4, 5) Solution: 1 13. Find the midpoint of the line segment with endpoints (6, 8) and (10, 12) Solution: (8, 10) 14. Solve for x: 5x - 3 = 2x + 7 Solution: x = 4 15. Find the roots of the equation x^2 + 3x + 2 = 0 Solution: x = -1 and -2 16. Find the median of the set {1, 3, 5, 7, 9} Solution: 5 17. Find the mean of the set {2, 4, 6, 8, 10} Solution: 6 18. Solve for x: log_2(x) = 3 Solution: x = 8 19. Solve for x: sin(x) = 0.5 Solution: x = 30 degrees or x = 330 degrees 20. Find the inverse of the function f(x) = 2x + 3 Solution: f^(-1)(x) = (x - 3)/2 ~~~~ Mistakes: 3, 5, 14, 19 (half points) Answer to 9 would probably be accepted in a school exam, even though it's rounded to 2 decimals and the question didn't specifically ask for that. So, depending on how severely you mark it, 75%-82.5%.
- mik1998 4y ago
  The problem with language models is that there's no thought behind what they say. It's just pattern recognition. If you ask it an original convoluted but simple natural language question, it will spit out nonsense. There's no thought behind what it sees - when it sees 5x - 3 = 2x + 7, it doesn't think that the x represents the variable, you need to plug in a number such that both sides are equal etc. It just sees the symbols and looks up the answer in it's memory. There's zero intelligence here. It's like a hypothetical student who simply memorizes all definitions without understanding at all. You can ask it for the definition of a linear system of equations, and you can ask it how to solve one, but there is zero understanding or thought there, in spite of apparent knowledge. In a way, it reminds me of Feynman's anecdote about Brazilian universities.
  - ben_w 4y ago
    Goalpost shifting aside, that presumes a definition of "thought". Perhaps these models are different from us in this regard (I'd assume so simply because it had to read more tokens than the total number of times an average single synapse in a human brain will fire in a lifetime), but I don't think humanity collectively yet has a concept of what it means to think that's good enough to prove the absence. Can submarines swim? And does it really matter?
    - mik1998 4y ago
      I'm not shifting goalposts, simply elaborating on your results. You already showed that ChatGPT cannot handle elementary school math problems.
      - ben_w 4y ago
        Which of those do you think were elementary grade? Because I'm fairly sure those are GCSE level[0]. In fact, why don't you suggest 20 "basic elementary school math problems", if you suggest 20 that are at that level, I'll put them in and post what it says; no cherry picking results, but I will grade it. Also, when I was at school, 75%-82.5% was a good score, and looking up the current system, that percentage range is in the top three grades (out of 9) at that level, and can be top grade depending on year and exam board. That is why I say you are moving the goalposts. [0] UK secondary schools are different from US highschools: https://en.wikipedia.org/wiki/General_Certificate_of_Secondary_Education https://en.wikipedia.org/wiki/General_Certificate_of_Seconda...
        mik1998 4y ago
        I haven't been in elementary school for some time, but I do remember linear equations being taught. Maybe except the sine question, since trigonometry is taught later. Getting 3/4th of simple elementary school questions right indicates that someone is really bad at math. These are not difficult questions. I don't have time to come up with 20 exercises, but I can suggest a few that I think the AI may have trouble with. All of these are trivial single-variable linear equations that a smart elementary school student should have no problem with. (Note that the numbers are intentionally scrambled; this does not increase the difficulty of the problem): 1. Solve for x: 113(6x - 45) = 92x 2. Multiply 13.4a - 18b by 18b + 13.4a 3. Four girls bought bus tickets. Anne bought seventeen 20-minute tickets and paid $323. Marianne bought twenty-eight 75-minute tickets and paid $784. Alice bought nineteen 20-minute tickets and eight 75-minute tickets, and Alex bought seven thousand eighty five 20-minute tickets and ninety six 75-minute tickets. How much did Alex pay? 4. An old man walked fifty nine kilometres in four hours through a flat field. He drove a car to France for nine hours, and then proceeded to walk for eighteen hours through a flat field at the same pace. How many kilometres did the old man walk in France? 5. Alice bought seventy-nine pens and twenty-six notebooks. The arithmetic mean of the cost of these articles was $9.31. The sum of the cost of the notebooks was $112.762. How much did a pen cost?

Last time I checked ChatGPT was unable to solve basic elementary school math problems.

I just asked it to give me 20 and then solve them. Either it is overestimating what elementary school teaches, or I'm underestimating the same: ~~~~ Sure! Here are the solutions to the problems: 1. Solve for x: 3x + 4 = 10 Solution: x = 2 2. Simplify: (5 + 3) * 2 Solution: 16 3. Convert: 3/4 cups to tablespoons Solution: 6 tablespoons 4. Round: 8.6 to the nearest whole number Solution: 9 5. Find the area of a rectangle with length 5 and width 3 Solution: 15 6. Find the perimeter of a triangle with sides of length 4, 5, and 6 Solution: 15 7. Solve for y: x + y = 8, x = 3 Solution: y = 5 8. Simplify: (6 - 2)^2 Solution: 16 9. Find the volume of a cube with side length 5 Solution: 125 10. Find the circumference of a circle with radius 3 Solution: 18.85 11. Simplify: 1/2 + 1/3 Solution: 5/6 12. Find the slope of the line with points (2, 3) and (4, 5) Solution: 1 13. Find the midpoint of the line segment with endpoints (6, 8) and (10, 12) Solution: (8, 10) 14. Solve for x: 5x - 3 = 2x + 7 Solution: x = 4 15. Find the roots of the equation x^2 + 3x + 2 = 0 Solution: x = -1 and -2 16. Find the median of the set {1, 3, 5, 7, 9} Solution: 5 17. Find the mean of the set {2, 4, 6, 8, 10} Solution: 6 18. Solve for x: log_2(x) = 3 Solution: x = 8 19. Solve for x: sin(x) = 0.5 Solution: x = 30 degrees or x = 330 degrees 20. Find the inverse of the function f(x) = 2x + 3 Solution: f^(-1)(x) = (x - 3)/2 ~~~~ Mistakes: 3, 5, 14, 19 (half points) Answer to 9 would probably be accepted in a school exam, even though it's rounded to 2 decimals and the question didn't specifically ask for that. So, depending on how severely you mark it, 75%-82.5%.

mik1998 4y ago
The problem with language models is that there's no thought behind what they say. It's just pattern recognition. If you ask it an original convoluted but simple natural language question, it will spit out nonsense. There's no thought behind what it sees - when it sees 5x - 3 = 2x + 7, it doesn't think that the x represents the variable, you need to plug in a number such that both sides are equal etc. It just sees the symbols and looks up the answer in it's memory. There's zero intelligence here. It's like a hypothetical student who simply memorizes all definitions without understanding at all. You can ask it for the definition of a linear system of equations, and you can ask it how to solve one, but there is zero understanding or thought there, in spite of apparent knowledge. In a way, it reminds me of Feynman's anecdote about Brazilian universities.
- ben_w 4y ago
  Goalpost shifting aside, that presumes a definition of "thought". Perhaps these models are different from us in this regard (I'd assume so simply because it had to read more tokens than the total number of times an average single synapse in a human brain will fire in a lifetime), but I don't think humanity collectively yet has a concept of what it means to think that's good enough to prove the absence. Can submarines swim? And does it really matter?
  - mik1998 4y ago
    I'm not shifting goalposts, simply elaborating on your results. You already showed that ChatGPT cannot handle elementary school math problems.
    - ben_w 4y ago
      Which of those do you think were elementary grade? Because I'm fairly sure those are GCSE level[0]. In fact, why don't you suggest 20 "basic elementary school math problems", if you suggest 20 that are at that level, I'll put them in and post what it says; no cherry picking results, but I will grade it. Also, when I was at school, 75%-82.5% was a good score, and looking up the current system, that percentage range is in the top three grades (out of 9) at that level, and can be top grade depending on year and exam board. That is why I say you are moving the goalposts. [0] UK secondary schools are different from US highschools: https://en.wikipedia.org/wiki/General_Certificate_of_Secondary_Education https://en.wikipedia.org/wiki/General_Certificate_of_Seconda...
      - mik1998 4y ago
        I haven't been in elementary school for some time, but I do remember linear equations being taught. Maybe except the sine question, since trigonometry is taught later. Getting 3/4th of simple elementary school questions right indicates that someone is really bad at math. These are not difficult questions. I don't have time to come up with 20 exercises, but I can suggest a few that I think the AI may have trouble with. All of these are trivial single-variable linear equations that a smart elementary school student should have no problem with. (Note that the numbers are intentionally scrambled; this does not increase the difficulty of the problem): 1. Solve for x: 113(6x - 45) = 92x 2. Multiply 13.4a - 18b by 18b + 13.4a 3. Four girls bought bus tickets. Anne bought seventeen 20-minute tickets and paid $323. Marianne bought twenty-eight 75-minute tickets and paid $784. Alice bought nineteen 20-minute tickets and eight 75-minute tickets, and Alex bought seven thousand eighty five 20-minute tickets and ninety six 75-minute tickets. How much did Alex pay? 4. An old man walked fifty nine kilometres in four hours through a flat field. He drove a car to France for nine hours, and then proceeded to walk for eighteen hours through a flat field at the same pace. How many kilometres did the old man walk in France? 5. Alice bought seventy-nine pens and twenty-six notebooks. The arithmetic mean of the cost of these articles was $9.31. The sum of the cost of the notebooks was $112.762. How much did a pen cost?