4 ms·
Checking the arithmetic in every paper published seems like an good use case for LLMs. Has someone built a better version than uploading a PDF to ChatGPT and as
by tlb 9mo ago
Checking the arithmetic in every paper published seems like an good use case for LLMs. Has someone built a better version than uploading a PDF to ChatGPT and asking it to check the arithmetic?
- ironbound 9mo agoLLM's are why we're in this mess, they can't do math or count r's
- brookst 9mo agoMore accurate to say they can’t see r’s. They process language but not letters.
- gordonhart 9mo agoModern reasoning models are actually pretty good at arithmetic and almost certainly would have caught this error if asked. Source: we benchmark this sort of stuff at my company and for the past year or so frontier models with a modest reasoning budget typically succeed at arithmetic problems (except for multiplication/division problems with many decimal places, which this isn't).
- RobotToaster 9mo agoInteresting, how have you found they have been performing at more complex things like calculus and analysis?
- speedgoose 9mo agoIt’s on the front page of HN once in a while.
- nine_k 9mo agoAn LLM usually has a powerful digital computer right in its disposal, and could use it as a tool to do precise calculations.
- UqWBcuFx6NV4r 9mo agoYes, yes. We’ve all seen the same screenshots. Very funny. Those of us who don’t base our technical understandings on memes are well aware of the tooling at the disposal of all modern reasoning models gives them the capability to do such things. Please don’t bring the culture war here.
- literalAardvark 9mo agoThey can't do math? ChatGPT 5.2 has recently been churning through unsolved Erdös problems. I think right now one is partially validated by a pro and the other one I know of is "ai-solved" but not verified. As in: we're the ones who can't quite keep up. https://arxiv.org/abs/2601.07421 https://arxiv.org/abs/2601.07421 And the only reason they can't count Rs is that we don't show them Rs due to a performance optimization.
- ironbound 9mo agoYou can feed it the Hodge Conjecture for all I care, the current algorithms are a joke and without real breakthroughs your just generating left to right text with billions in hardware.
- literalAardvark 9mo agoGuess frontier math and programming are just left to right text then.