10 ms·
Large Language Model Reasoning Failures
- chrisjj 8mo agoThe only reasoning failures here are in the minds of humans gulled into expecting chatbot reasoning ability.
- altmanaltman 8mo agoBut how else will Dario raise Series X
- chrisjj 8mo agoToo true! :)
- sergiomattei 8mo agoPapers like these are much needed bucket of ice water. We antropomorphize these systems too much. Skimming through conclusions and results, the authors conclude that LLMs exhibit failures across many axes we'd find to be demonstrative of AGI. Moral reasoning, simple things like counting that a toddler can do, etc. They're just not human and you can reasonably hypothesize most of these failures stem from their nature as next-token predictors that happen to usually do what you want. So. If you've got OpenClaw running and thinking you've got Jarvis from Iron Man, this is probably a good read to ground yourself. Note there's a GitHub repo compiling these failures from the authors: https://github.com/Peiyang-Song/Awesome-LLM-Reasoning-Failures https://github.com/Peiyang-Song/Awesome-LLM-Reasoning-Failur...
- vagrantstreet 8mo agoIsn't it strange that we expect them to act like humans even though after a model was trained it remains static? How is this supposed to be even close to "human like" anyway
- LiamPowell 8mo agoIf we could reset a human to a prior state after a conversation then would conversations with them not still be "human like"? I'm not arguing that LLMs are human here, just that your reasoning doesn't make sense.
- hackinthebochs 8mo agoHenry Molaison was exactly this.
- mettamage 8mo ago> Isn't it strange that we expect them to act like humans even though after a model was trained it remains static? An LLM is more akin to interacting with a quirky human that has anterograde amnesia because it can't form long-term memories anymore, it can only follow you in a long-ish conversation.
- alansaber 8mo agoI mean you can continue to evolve the model weights but the performance would suck so we don't do it. Models are built to an optimal state for a general set of benchmarks, and weights are frozen in that state.
- lostmsu 8mo agohttps://en.wikipedia.org/wiki/List_of_cognitive_biases https://en.wikipedia.org/wiki/List_of_cognitive_biases Specifically, the idea that LLMs fail to solve some tasks correctly due to fundamental limitations where humans also fail periodically well may be an instance of the fundamental attribution error.
- otabdeveloper4 8mo ago> We antropomorphize these systems too much. They're sold as AGI by the cloud providers and the whole stock market scam will collapse if normies are allowed to peek behind the curtain.
- alansaber 8mo agoThe stock market being built on conjecture? Surely not sir.
- simianwords 8mo agoMost of the claims are likely falsified using current models. I wouldn’t take many of them seriously.
- jibal 8mo agoI wouldn't take baseless "likely" claims or the people who make them seriously.
- simianwords 8mo agoI falsified it on another thread
- throw310822 8mo ago> conclude that LLMs exhibit failures across many axes we'd find to be demonstrative of AGI. Which LLMs? There's tons of them and more powerful ones appear every month.
- alansaber 8mo agoTrue but the fundamental architecture tends not to be radically different, it's more about the training/RL regime
- throw310822 8mo agoBut the point is that to even start to claim that a limitation holds for all LLMs you can't use empirical results that have been demonstrated only for a few old models. You either have a theoretical proof, or you have empirical results that hold for all existing models, including the latest ones.
- donperignon 8mo agoan llm will never reason. reasoning is an emergent behavior of those systems that is poorly understood. neurosymbolic systems will be what combined with llm will define the future of AI
- hackinthebochs 8mo agoWhat are neurosymbolic systems supposed to bring to the table that LLMs can't in principle? A symbol is just a vehicle with a fixed semantics in some context. Embedding vectors of LLMs are just that.
- logicprog 8mo agoPre-programmed, hard and fast rules for manipulating those symbols, that can automatically be chained together according to other preset rules. This makes it reliable and observable. Think Datalog. IMO, symbolic AI is way too brittle and case-by-case to drive useful AI, but as a memory and reasoning system for more dynamic and flexible LLMs to call out to, it's a good idea.
- hackinthebochs 8mo agoSure, reliability is a problem for the current state of LLMs. But I see no reason to think that's an in principle limitation.
- logicprog 8mo agoThere are so many papers now showing that LLM "reasoning" is fragile and based on pattern-matching heuristics that I think it's worth considering that, while it may not be an in principle limitation — in the sense that if you gave an autoregressive predictor infinite data and compute, it'd have to learn to simulate the universe to predict perfectly — in practice we're not going to build Laplace's LLM, and we might need a more direct architecture as a short cut!
- simianwords 8mo ago
- Lapel2742 8mo ago> These models fail significantly in understanding real-world social norms (Rezaei et al., 2025), aligning with human moral judgments (Garcia et al., 2024; Takemoto, 2024), and adapting to cultural differences (Jiang et al., 2025b). Without consistent and reliable moral reasoning, LLMs are not fully ready for real-world decision-making involving ethical considerations. LOL. Finally the Techbro-CEOs succeeded in creating an AI in their own image.
- throw310822 8mo ago> These models Which models? The last ones came out this week.
- runlaszlorun 8mo agoI think this issue is way overlooked. Current LLMs embed a long list of values that are going to be incongruent with a large percentage of the population. I don't see any solution longer term other than more personalized models.
- simianwords 8mo agoi'm very skeptical of this paper. >Basic Arithmetic. Another fundamental failure is that LLMs quickly fail in arithmetic as operands increase (Yuan et al., 2023; Testolin, 2024), especially in multiplication. Research shows models rely on superficial pattern-matching rather than arithmetic algorithms, thus struggling notably in middle-digits (Deng et al., 2024). Surprisingly, LLMs fail at simpler tasks (determining the last digit) but succeed in harder ones (first digit identification) (Gambardella et al., 2024). Those fundamental inconsistencies lead to failures for practical tasks like temporal reasoning (Su et al., 2024). This is very misleading and I think flat out wrong. What's the best way to falsify this claim? Edit: I tried falsifying it. https://chatgpt.com/share/6999b72a-3a18-800b-856a-0d5da45b94c9 https://chatgpt.com/share/6999b72a-3a18-800b-856a-0d5da45b94... https://chatgpt.com/share/6999b755-62f4-800b-912e-d015f9afc81d https://chatgpt.com/share/6999b755-62f4-800b-912e-d015f9afc8... I provided really hard 20 digit multiplications without tools. If you looked at the reasoning trace, it does what is normally expected and gets it right. I think this is enough to suggest that the claims made in the paper are not valid and LLMs do reason well. To anyone who would disagree, can you provide a counter example that can't be solved using GPT 5 pro but that a normal student could do without mistakes?
- simianwords 8mo ago>Math Word Problem (MWP) Benchmarks. Certain benchmarks inherently possess richer logical structures that facilitate targeted perturbations. MWPs exemplify this, as their logic can be readily abstracted into reusable templates. Researchers use this property to generate variants by sampling numeric values (Gulati et al., 2024; Qian et al., 2024; Li et al., 2024b) or substituting irrelevant entities (Shi et al., 2023; Mirzadeh et al., 2024). Structural transformations – such as exchanging known and unknown components (Deb et al., 2024; Guo et al., 2024a) or applying small alterations that change the logic needed to solve problems (Huang et al., 2025b) – further highlight deeper robustness limitations. I'm willing to bet this is no longer true as well. We have models that are doing better than humans at IMO.
- otabdeveloper4 8mo ago> We have models that are doing better than humans at IMO. Not really. From my brief experience they can guess the final answer but the intermediate justifications and proofs are complete hallucinated bullshit. (Possibly because the final answer is usually some sort of neat and beatiful answer and human evaluators don't care about the final answer anyways, in any olympiad you're graded on the soundness of your reasoning.)