4 ms·
At least with Codex, this has not been my experience at all. It still screws up sure, but in every case I can ask "why did you do this" and it can trace back wh
by _fat_santa 27d ago
At least with Codex, this has not been my experience at all. It still screws up sure, but in every case I can ask "why did you do this" and it can trace back what made it take that particular decision. Typically it's always that I either didn't specify the problem correctly or made a really dumb mistake (executing the task on the wrong project....did this one yesterday) or it's something within a skill file that instructs it (at which point I fixup the instructions).
Once in a blue moon it's actually the model making a material error in it's thinking and I have to go back and redo it.
- arnorhs 27d agoagreed to some extent. I think this parody still highlights what I feel is often the experience. It might not happen on a simple task such as changing a button color, but on more complicated things, this can definitely be exactly what it feels like.
- tarxzvf 27d agoModels hallucinate plausible answers to why they did things. It might be true and it might be complete fiction.
- taeric 27d agoI'm growing increasingly confident that this is how people often work, as well.
- skinfaxi 27d agoPeople don't make rational decisions that make rationalized decisions. Is there any thought to pulling your hand off a hot surface?
- autoexec 27d agoPeople do both. Some choices aren't worth the time and effort of detailed analysis and contemplation and some are basically instinctual, but there are plenty of times that choices are carefully considered and well reasoned before being made and acted on.
- nemomarx 27d agoI kind of want my computer systems to be more reliable and predictable than paying an intern to manage something and asking why they messed up
- jaggederest 27d agoAt this point it very dramatically is more reliable and predictable than any human I've worked with. Do you know anyone who actually reads and adheres closely to all of the documentation every time it's changed?
- bluefirebrand 27d agoI don't know anyone who has that kind of time, no
- wccrawford 27d agoThat was my experience with Claude when my vibe-coded project was small. But now that I've been working on it a month and there's a lot of documentation, it's pretty clearly ignoring parts of the documentation and parts of the code. It will come up with some ridiculous statement about how something works, and I'll challenge it, and it'll admit I'm right. It definitely reads more documentation than any programmer I've ever worked with (myself included) but because it doesn't have a memory other than the documentation, it still makes mistakes like that. I haven't turned on "memory" or tried it with Codex, so I don't know how that'll change soon, though.
- jaggederest 27d agoYeah the biggest task these days that I do manually is curating the documentation. AGENTS.md in every major directory, and a variety of reference docs that are explicitly referenced in those files. # See DOC-ITEM-NAME DOC-ITEM-NAME.md When referencing documents, always use the exact syntax See <TAG> - this is enforced by a lint on precommit And those doc items are basically all of the values, architectural, strategic, and tactical items. It's a poor man's in-repo RAG but it's shockingly effective, especially if you keep them small. I may migrate some/all of them to skills over time, but I usually update them biweekly, and I only allow agents to make small edits or propose new notes. And typically I go through and delete or curate any agent edits before merge. Depending on language I've seen this scale past multiple millions of lines of code, as long as you pair it with all of the linting and tooling that you can possibly build.
- voakbasda 27d agoEvery time I hear someone complain about hallucinations, I laugh at the total lack of self awareness about our species. Humans are just as bad (now, probably worse) at telling the truth, whether due to intention or poor memory.
- lathiat 27d ago“Post Hoc Rationalisation” https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-rationalisation-reasoning-our-intuition-and-changing-our-minds/ https://www.patheos.com/blogs/tippling/2013/11/14/post-hoc-r...
- BurningFrog 27d agoI learned this from "The Elephant in the Brain", which I strongly recommend: https://amzn.to/4iSyLX8 https://amzn.to/4iSyLX8
- jameshart 27d agoI learned this from Dirk Gently’s Holistic Detective Agency, which I strongly recommend.
- BurningFrog 25d agoI read that book and strongly recommend it too! Don't remember that part though. It's been a few decades.
- supern0va 27d agoExactly. I am becoming increasingly convinced that this is actually just a part of how intelligence/cognition works.
- astrobe_ 27d agoBut is it really what we want, machines with the same defects as humans? I don't want a pocket calculator that make mistakes "sometimes" so I have to double-check the results, I want a pocket calculator that works (to those who want to argue that pocket calculators don't give the correct result for (1/3)*3: STFU).
- CookieCrisp 27d agoNo, but it makes sense to me that we’d need to go through this step to get where we want to go
- bmacho 27d agoNo. People have an inner monologue, partial results and ideas and they remember that. If they've worked some minutes/hours/weeks on something and you ask them why did they do that, they will either answer honestly and truthfully, lie, or say "I missed that/didn't seem important so I just chose something at random". None of these cases are similar to how AI works.
- yawnxyz 27d agoI think this is true with some people, but I don't think this holds true for some (or even most) people across the US (at least not all the folks I've worked with)
- croo 27d agoActually split brain experiments tells a different story. The left hemisphere actively confabulates, inventing plausible explanations for actions it didn’t initiate, suggesting that much of human self-narrative may be post-hoc storytelling.
- satvikpendem 27d agoIndeed. People literally make stuff up when their corpus callosum is severed.
- bmacho 27d agoAand what if their corpus callosum isn't severed?
- satvikpendem 26d agoThey also make stuff up.
- deleted 27d ago[deleted]
- bmacho 27d ago
- yonatan8070 27d agoAFAIK this really is true. I've seen some videos about patients who had the connecting part between the left and right halves of the brain cut as a (archaic) treatment for epilepsy. While it did help the epilepsy, their brain was essentially two brains controlling two halves of the body. With one controlling speech. There were experiments where one eye was shown some instruction text, the corresponding hand performed that instruction, and when asked why they dix that action, the speaking half just made up some plausible, yet completely wrong reason, just like an LLM.
- deleted 27d ago[deleted]
- Terr_ 27d agoI see this observation frequently, and I dislike how it often has the unsound subtext of: "Therefore something is going well or at least not too badly." If you build a robot where a pressurized hose leaks causing fluid to destroy part of the circuitry, we don't praise it as progress towards the human ideal of having brain aneurysms. A similar failure-path is not a reliable indicator of a similar success-path.
- krupan 26d agoAll people at least some times, absolutely yes. Isn't it awesome we built machine that does the exact same thing, but even faster and more often? /s
- theluketaylor 27d agoTrue, but even a hallucinated explanation of where things went wrong added to the context can force the model down a better path over the next few inputs.
- embedding-shape 27d agoYou can also literally tell them: "Here is your session ID: $ID, lookup the .jsonl session, trace exactly why this decision was being made, present evidence and concrete proof, no guessing or assumptions" and you'll get an evidence-based report without guesses.
- zamadatix 27d agoIt can always hallucinate said report results/evidence/proof just the same. This approach tends to help reduce the hallucination rate though. You can extend this further by using an adversarial agent trying to find mistakes in the other instance's logs in a loop where a 3rd neutral agent weighs the claims of the other two. This is also just another step in reducing error, it does not guarantee elimination of such errors. The latter is an impossible guarantee, even for humans.
- bmacho 27d agoAsk it to build you a simple and deterministic citation checking extension to your IDE that puts source in meta to citations. E.g. color citations green/red depending if they are valid.
- zamadatix 27d agoSure, you can always validate what it's saying yourself at any point & you can have it try to make manual verification an easier process to complete via methods such as the above.
- Kiro 27d agoThe point of the parent post is that the explanation shows they made the error themselves, so it's immediately validated.
- jameshart 27d agoTo test this, change the history in the context to indicate that the model did or recommended something completely different than it actually did, and then ask it to explain why. You’ll still get a plausible explanation.
- JeremyNT 27d agoYes. But although they can't know "why" a specific "wrong" answer was selected, the response is often still informative, and it can highlight real weaknesses in process or code structure that should be addressed anyway.
- malfist 27d agoWhen you've been perfectly precise in your spec and language, isn't that programming? Why use a stochastic goblin to do things in that case?
- blackenedgem 27d agoSee https://www.commitstrip.com/en/2016/08/25/a-very-comprehensive-and-precise-spec/ https://www.commitstrip.com/en/2016/08/25/a-very-comprehensi...
- throwaway6977 27d agoIt's just a lot faster at hammering it out than me pound for pound, and I can quickly rattle off via voice-to-text exactly what I want much faster than I can type all of the code (especially when across a few different files), in a huge majority of tasks I perform. It's also especially good at debugging by brute force quickly and at scale meaning e.g. it can start desperately bisecting diffs to find the source of a bug 10000% faster than I can.
- malfist 27d agoAnd then you get two blue buttons and a terms of service talking about chemical sales
- embedding-shape 27d agoFor me, typing "Create a new namespace with these enums, functions and traits, that should follow X, Y and Z constraints" is faster than typing all that code manually, and typing less is less straining on my hands/fingers.
- burntpineapple 27d ago[dead]
- Pannoniae 25d agothat would only be true with a perfectly expressive language with both high-level and low-level features, perfect support for any kind of metaprogramming and a godly optimiser. ofc we don't have that, so code is compressible, and compressible a lot. You can say what you need in English much shorter than in in any programming language
- applfanboysbgon 27d agoYesterday, with Astra Max, a very clear instruction to "remove the GUI editor pane and add <another component> to the existing sidebar" for a prototype I had it working on resulted in it deleting literally the entire GUI and building a new one from scratch, including the requested component and losing almost all other functionality of the application. This is fucking constant. I can't deny that this stupid tech saves time prototyping even with having to wrangle it, but it commits a fireable offense several times a day that no human would get away with and is obviously incapable of learning from mistakes in the way a human is. The only reason it's not fired is because it's a slave that works for no more than the cost to feed it.
- throwawayffffas 27d agoNo model acts like this in my experience, not fable, not opus, not k3, not gml, not qwen 3.8 either. Additionally the provided prompts are not what anyone who has used this things would say in either situation. Sure you can ask it to make one button blue and it can easily make all buttons blue, but they quickly backtrack if told to.
- barbazoo 27d agoThe game is fun because it's so obvious that any answer will just devolve into an even more unstable state when in reality I feel like it's pretty straight forward to correct it in the moment, if not permanently, to get what you actually need.
- reedlaw 27d agoCodex has the opposite problem. Instead of being overly proactive it's overly reticent. I have been preferring it lately, although my preference tend to switch every few months when a model or harness regresses horribly.
- pgwhalen 26d agoI see this as more of a cute historical artifact than anything. There was a time when models/harnesses behaved like this, but we are well past it for frontier (or not so frontier) models.