3 ms·
> Tell ChatGPT to multiply 129348723423 and 2987892342424 and it'll probably get it wrong because nowhere on Reddit is that exact question for it to copy. But w
by simianwords 9mo ago
> Tell ChatGPT to multiply 129348723423 and 2987892342424 and it'll probably get it wrong because nowhere on Reddit is that exact question for it to copy. But what's interesting is it'll tend to get the first and large digits correct (more often than not) but the middle is just noise.
People have no idea how capable LLM's are and confidently write these kind of things.
- jmyeet 9mo agoThis is a known problem and an active area of research [1][2][3][4]. [1]: https://arxiv.org/html/2505.15623v1 https://arxiv.org/html/2505.15623v1 [2]: https://medium.com/@adnanmasood/why-large-language-models-struggle-with-mathematical-reasoning-3dc8e9f964ae https://medium.com/@adnanmasood/why-large-language-models-st... [3]: https://www.reachcapital.com/resources/thought-leadership/why-llms-are-bad-at-math-and-how-they-can-be-better/ https://www.reachcapital.com/resources/thought-leadership/wh... [4]: https://mathoverflow.net/questions/502120/examples-for-the-use-of-ai-and-especially-llms-in-notable-mathematical-developme https://mathoverflow.net/questions/502120/examples-for-the-u...
- simianwords 9mo agothe research does't capture the fact that LLM's can easily multiply these results. I mean it literally won gold in IMO, Putnam. Take 10,000 such multiplications. I'm sure not even a single one would be incorrect with GPT 5.2 (thinking). Want a wager?