4 ms·
Wait, what do you mean "LLMs are still absolutely useless at actual maths computation"? I rely on them constantly for maths (linear algebra, multivariable calc,
by tptacek 7mo ago
Wait, what do you mean "LLMs are still absolutely useless at actual maths computation"? I rely on them constantly for maths (linear algebra, multivariable calc, stat) --- literally thousands of problems run through GPT5 over the last 12 months, and to my recollection zero failures. But maybe you're thinking of something more specific?
- deleted 7mo ago[deleted]
- schneems 7mo agoThey are bad at math. But they are good at writing code and as an optimization some providers have it secretly write code to answer the problem, run it and give you the answer without telling you what it did in the middle part.
- avaer 7mo agoSomeone should tell the mathematicians if they use a calculator or a whiteboard or heavens forbid a computer they are "bad at math".
- deleted 7mo ago[deleted]
- schneems 7mo ago1) That's not related to chain of thought I was replying to. Someone asked about the "bad at math" and pointed out "but it seems good to me" so I added the color of why that might be the case. Your retort seems to imply I'm making an argument that because something uses tools for a job it cannot be good at the thing it's using a tool for. Which is not the case. 2) If you have something to say, just say it. Don't put words in my mouth and then argue with a thing I didn't say.
- tptacek 7mo agoRight, but your narrative was incorrect and based on faulty premises, which you haven't acknowledged. That's fine, except you're still pressing the argument. Can you please present a reasonable maths problem that I can bounce off GPT so we can see it fail? I can give you many hundreds of relatively complex problems, none of which have appeared in a textbook, that GPT has not only solved, but critiqued my own crappy solutions for. I'm only asking you for one counterpoint.
- schneems 7mo ago> your narrative was incorrect and based on faulty premises I am referring to specific, documented behavior of LLMs. Google it.
- tptacek 7mo agoGoogle any plausibly reasonable math problem, and even the terrible LLM that powers the Google search page will almost certainly solve it correctly for you. I don't need to reconstruct my argument axiomatically from folk beliefs.
- schneems 7mo agoYou seem to have misunderstood my comment. I'm happy to accept the fault for poor communication. But you're making it hard. You're signaling that any clarifications on my behalf will be treated as further arguments instead of some sort of shared desire to hear one another. I don't care to continue.
- tempaccount5050 7mo agoAre they bad at math? Or are they bad at arithmetic?
- tptacek 7mo agoWhat would I do to demonstrate that they are bad at math? If by "maths" we mean things like working out a double integral for a joint probability problem, or anything simpler than that, GPT5 has been flawless.
- schneems 7mo agoSearch the topic. It is historically documented. It might no longer be true though. A way to test might be running an open model locally, directly (without a harness) where you could be sure it's not going through a translation layer. I think these days it might have this tool call behavior built in, but I think back in the day it was treated more like a magic trick. Without it, it behaved similar to "how many r's are in strawberry" for simple math.
- tptacek 7mo agoIt is wildly not true. The request is for some reasonable math problem a model like GPT or Claude will fail at. I'm not going to set up a local model or some harness for it; I'm just going to copy/paste it into ChatGPT and watch it solve it. Propose a problem, if you think I'm wrong about this. Seems simple.
- schneems 7mo ago> wildly not true Source? Did you search anything like I suggested or no?
- tptacek 7mo agoMy argument: you can take basically any undergraduate collegiate math problem, right now, and it's likely that even the dumb LLM on the Google search page will solve, and nearly certain that frontier models will. Your argument: "it is possible to Google for people claiming LLMs can't do math".
- jasonfarnon 7mo agoWhat tier are you using? I have run lots of problems and am very impressed, but I find stupid errors a lot more frequently than that, e.g., arithmetic errors buried in a derivation or a bad definition, say 1/15 times. I would love to get zero failures out of thousands of (what sounds like college-level math) posed problems.
- tptacek 7mo agoI have a standard OpenAI/ChatGPT Pro account; GPT5 is my daily driver for math, and Claude for code.
- deleted 7mo ago[deleted]
- cuttothechase 7mo agocalc, stat etc from a text book is something they would naturally be good at but I don't think book based computations thats in the training set and its extrapolations is what is at question here. They are not great at playing chess as well - computational as well as analytic.
- tptacek 7mo agoI think this is wrong and a category error (none of the problems I've given it are in a textbook; they're virtually all randomized), but, try this: just give me a problem to hand off to GPT5, and we'll see how it does. Further evidence for the faultiness of your claim, if you don't want to take me up on that: I had problems off to GPT5 to check my own answers. None of the dumb mistakes I make or missed opportunities for simplification are in the book, and, again: it's flawless at pointing out those problems, despite being primed with a prompt suggesting I'm pretty sure I have the right answers.
- ButlerianJihad 7mo agoI only have rudimentary understanding of calculus, trigonometry, Google Sheets, and astronomy, but I was able to construct an accurate spreadsheet for astrometry calculations by using Grok and Gemini (both free, no subscription, just my personal account) to surface the formulas for measuring the distance between 2-3 points on the celestial sphere. The LLMs assisted me in also writing functions to convert DMS/HMS coordinates to decimal, and work in radians as well. I found and fixed bugs I wrote into the formulas and spreadsheets, and the LLMs were not my sole reference, but once the LLM mentioned the names of concepts and functions, I used Wikipedia for the general gist of things, and I appreciated the LLMs' relevant explanations that connected these disciplines together. I did this on March 14, 2026
- Drupon 7mo ago[flagged]
- pfdietz 7mo agoPerhaps learning how to get AI to solve your problems is the most important lesson to learn now? The rest seems like the current equivalent of learning cursive.
- tptacek 7mo agoNone of this makes any sense. I'm not in school and I'm not a teacher. It's just a random attempted drive-by dunk that faceplanted.
- Drupon 7mo agoNo, learning math is still a valuable skill. You still learned how addition worked growing up even though calculators existed, right? You learned to read even though screen readers exist?
- adrianN 7mo agoImo you learn to add not for the skill of adding, but for the skill of understanding and applying algorithms. Adding is just a simple example.