4 ms·

> show me a 4th grade math problem they can't handle Sure. "8 7 6 5 4 3 2 1 - add minus signs and parenthesis to get 31." P.S. There is an answer online and

> show me a 4th grade math problem they can't handle Sure. "8 7 6 5 4 3 2 1 - add minus signs and parenthesis to get 31." P.S. There is an answer online and some LLMs will just copy it verbatim. This doesn't count.

simonw 4mo ago
Whoa, 4th grade math problems got hard! I'm not sure how I'd tackle that one myself.
- simonw 4mo ago
  GPT-5.5 found a solution only after assuming that you're allowed to concatenate numbers together e.g. 8 7 becomes 87 (it complained at first that it was "under-specified") - using Python it brute-forced a solution (actually finding 13): https://chatgpt.com/share/6a1db54f-7ab8-8333-9218-86a469c284bc https://chatgpt.com/share/6a1db54f-7ab8-8333-9218-86a469c284... Are you sure this is 4th grade level?
  - minimaxir 4mo ago
    I questioned OP's "there is an answer online" claim so I checked and the only source found for the original question was a 5th grade Russian school for mathematics. https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%5B%221kj6r9fOIshFmCwcNpQDcfEFnFdYezjkm%22%5D,%22action%22:%22open%22,%22userId%22:%22110194568010056725318%22,%22resourceKeys%22:%7B%7D%7D&usp=sharing https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...
  - MichaelNolan 4mo ago
    Apparently there is a way to solve this without brute forcing all the combinations. It has to do with looking at how many even an odd numbers there are, and taking into account the goal number is odd. And then thinking through the combinations [even-even=even, even-odd=odd,…] Though this is obviously not something I would expect a 4th grader to solve.
dwohnitmok 4mo ago
[dead]
DiogenesKynikos 4mo ago
> 4th grade math problem And it turns out to be an extremely difficult problem given to Russian math prodigies, which requires one to bend the rules and turn "8 7" into "87".
- otabdeveloper4 4mo ago
  It's a standard "Russian math" problem. There's boatloads more where that came from, and none of them are solved by LLMs.
sanderjd 4mo ago
It's very funny how you chose an example that is both not 4th grade level math and also something the frontier LLMs are much more likely to be able to solve than nearly any 4th grader. This is a counterexample to your argument, not evidence for your claim. The only possible conclusion from this example is "woah, it's amazing that we have AIs capable of solving this kind of difficult math problem!", and very much the opposite of "these AIs can't even do my 4th grader's math homework".

simonw 4mo ago

Whoa, 4th grade math problems got hard! I'm not sure how I'd tackle that one myself.

simonw 4mo ago
GPT-5.5 found a solution only after assuming that you're allowed to concatenate numbers together e.g. 8 7 becomes 87 (it complained at first that it was "under-specified") - using Python it brute-forced a solution (actually finding 13): https://chatgpt.com/share/6a1db54f-7ab8-8333-9218-86a469c284bc https://chatgpt.com/share/6a1db54f-7ab8-8333-9218-86a469c284... Are you sure this is 4th grade level?
- minimaxir 4mo ago
  I questioned OP's "there is an answer online" claim so I checked and the only source found for the original question was a 5th grade Russian school for mathematics. https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%5B%221kj6r9fOIshFmCwcNpQDcfEFnFdYezjkm%22%5D,%22action%22:%22open%22,%22userId%22:%22110194568010056725318%22,%22resourceKeys%22:%7B%7D%7D&usp=sharing https://aistudio.google.com/app/prompts?state=%7B%22ids%22:%...
- MichaelNolan 4mo ago
  Apparently there is a way to solve this without brute forcing all the combinations. It has to do with looking at how many even an odd numbers there are, and taking into account the goal number is odd. And then thinking through the combinations [even-even=even, even-odd=odd,…] Though this is obviously not something I would expect a 4th grader to solve.

dwohnitmok 4mo ago

[dead]

DiogenesKynikos 4mo ago

> 4th grade math problem And it turns out to be an extremely difficult problem given to Russian math prodigies, which requires one to bend the rules and turn "8 7" into "87".

otabdeveloper4 4mo ago
It's a standard "Russian math" problem. There's boatloads more where that came from, and none of them are solved by LLMs.

sanderjd 4mo ago

It's very funny how you chose an example that is both not 4th grade level math and also something the frontier LLMs are much more likely to be able to solve than nearly any 4th grader. This is a counterexample to your argument, not evidence for your claim. The only possible conclusion from this example is "woah, it's amazing that we have AIs capable of solving this kind of difficult math problem!", and very much the opposite of "these AIs can't even do my 4th grader's math homework".