4 ms·
This paper puts words to something I’ve noticed repeatedly with LLMs, particularly Qwen3.6. When I read its reasoning, it appears to recognise the mistake and t
by florianherrengt 2mo ago
This paper puts words to something I’ve noticed repeatedly with LLMs, particularly Qwen3.6. When I read its reasoning, it appears to recognise the mistake and then carry on as if it hadn’t noticed it at all.
> models often determine their answers based on implicit biases tied to question templates, then construct reasoning chains to justify their predetermined conclusions
> its reasoning was correct right until the final step (Yes/No answer)
- Georgelemental 2mo agoNatural intelligences do this too
- freejazz 2mo agoYeah and it's not great then either
- tyg13 2mo agoMust we always see this restated every time? It's getting a bit stale always seeing these kinds of comments on articles about LLM.
- cyanydeez 2mo agoYou think, "this problem" is qn LLM problem?
- deleted 2mo ago[deleted]
- uludag 2mo agoI can just immagine the response to a headline "LLM chooses mass death: thousands killed in horrific AI accident" being something like "lots of humans have caused mass death too."
- gopher_space 2mo agoIt's the grounded portion of a feedback loop searching for the 'why is this happening' thinking. I'd imagine most of my own comments in this area boil down to "GIGO" most of the time. It's a relevant comment in this instance because we're discussing concepts you need to be both trained and practiced in to reason about, and that our discipline has traditionally been blind to. Plenty of people working with LLM context issues who've never been exposed to the idea of 'subtext' or could tell you why it would matter to their direction of effort.
- 8note 2mo agocontinuing with the "we need open training data" thread how much of the training data had thinking traces that dont make sense to people as being actually a description of why the output should be that way?
- 8note 2mo agoor rather, its not productive to "we should do better with artificial intelligence"
- ethin 2mo agoI agree, and I very strongly dislike it, to be polite about it. It contributes absolutely nothing and is an excellent way of hand-waving away literally anything an AI model does. Saying "well people do this too" is a great way to rationalize away anything you can imagine that an AI model would be capable of, because "humans do it too so what's the big deal, guys?"
- Kim_Bruning 2mo agoMaybe we can think up a (personal) rule we can follow? A naked "natural intelligences do this too" might be a bit too short to be useful. But if we can add when/where, cite papers, or show ways in which the parallel operates, then it might be useful. Compare, eg, talking about a robot arm, and someone goes "a natural arm does this too". You can tell about the fact that it has the same degrees of freedom in the same places, or how this pertains to inverse kinematics, or etc... Same way here, "this happens to be how natural intelligences seem to solve this too! According to Foo, Bar, Baz et al (2026) the gadget is always twiddled beforehand in macaque apes. " or "Same for natural intelligence: I've noticed I use the same general algorithm myself. I've always considered this the correct way to do translation between languages". -- A more concrete example of a useful answer here. Natural intelligence does this too! When given the question "explain your reasoning" humans are indeed quite prone to post-hoc confabulation. [1] [1] https://home.csulb.edu/~cwallis/382/readings/482/nisbett%20saying%20more.pdf https://home.csulb.edu/~cwallis/382/readings/482/nisbett%20s... "Telling More Than We Can Know: Verbal Reports on Mental Processes" (this citation is quite old and may have been superseded, mostly just to illustrate how the rule might work)
- phailhaus 2mo agoNo they don't, human intelligence has the ability to form an internal model of itself, which allows it to "notice" its own mistakes and change.
- cyanydeez 2mo agoMany who watched the last decade knows just because its possible to noticed mistakes and change, its clearly not a reliable process.
- elictronic 2mo agoThe current political climate is well reasoned and intentional. It might not be yours or mine, however the system is working exactly as the ones paying for it have intended.
- delichon 2mo agoMy brother and I have been arguing about that all of our lives. He believes everything is intentional and it's just a matter of discovering who benefits. I see chaos that nobody intends or controls. His political landscape is a tapestry of conspiracy theories and mine is a fog of war. I think his is more comforting, since it admits a possibility of a rational, predictable world.
- dgellow 2mo agoIt’s intentional in the sense that actors are acting intentionally for their own benefit (or at least what they believe is beneficial) and following incentives. Not that there is a master planner who manipulates everything
- elictronic 2mo agoThe current admin seems to have quite a few long term plans they have been working towards. Project 2025, Maralago accords. So far the only major policy item the Trump admin seems to have not intended was the Iran War. Israel killing the intended replacement, Iran leveraging the straight of Hormuz, and dropping three Tomahawks on an elementary school really botched that one.
- ethin 2mo ago[flagged]
- paimapi 2mo ago[dead]
- flyingpumba 2mo agoFirst author here, surprised to see the paper in HN! :) When doing the paper we noticed that models are very good at generating post-hoc plausible CoT, which to me knowledge can happen quite often with relatively easy tasks. You might be interested in reading this other paper that came out after ours: https://arxiv.org/abs/2507.05246 https://arxiv.org/abs/2507.05246