3 ms·
I've tried these with Claude various times and never get the wrong answer. I don't know why, but I am leaning they have stuff like "memory" turned on and possib
by kalcode 6mo ago
I've tried these with Claude various times and never get the wrong answer. I don't know why, but I am leaning they have stuff like "memory" turned on and possibly reusing sessions for everything? Only thing I think explains it to me.
If your always messing with the AI it might be making memories and expectations are being set. Or its the randomness. But I turned memories off, I don't like cross chats infecting my conversations context and I at worse it suggested "walk over and see if it is busy, then grab the car when line isn't busy".
- jorvi 6mo agoEven Gemini with no memory does hilarious things. Like, if you ask it how heavy the average man is, you usually get the right answer but occasionally you get a table that says: - 20-29: 190 pounds - 30-39: 375 pounds - 40-49: 750 pounds - 50-59: 4900 pounds Yet somehow people believe LLMs are on the cusp of replacing mathematicians, traders, lawyers and what not. At least for code you can write tests, but even then, how are you gonna trust something that can casually make such obvious mistakes?
- dyauspitr 6mo agoSo what? That might happen one out of 100 times. Even if it’s 1 in 10 who cares? Math is verifiable. You’ve just saved yourself weeks or months of work.
- icedchai 6mo agoYou don't think these errors compound? Generated code has 100's of little decisions. Yes, it "usually" works.
- dyauspitr 6mo agoNot in my experience. With a proper TDD framework it does better than most programmers at a company who anecdotally have a bug every 2-3 tasks.
- tranceylc 6mo agoThe kind of mistakes it makes are usually strange and inhuman though. Like getting hard parts correct while also getting something fundamental about the same problem wrong. And not in the “easy to miss or type wrong” way. I wish I had an example for you saved, but happens to me pretty frequently. Not only that but it also usually does testing incorrectly at a fundamental level, or builds tests around incorrect assumptions.
- icedchai 6mo agoI've seen LLMs implement "creative" workarounds. Example: Sonnet 4.5 couldn't figure out how to authenticate a web socket request using whatever framework I was experimenting with, so it decided to just not bother. Instead, it passed the username as part of the web socket request and blindly trusted that user was actually authenticated. The application looked like it worked. Tests did pass. But if you did a cursory examination of the code, it was all smoke and mirrors.
- bratwurst3000 6mo agoyes i wished i had safes some of my best examples too. One i had was super weird in chatgpt pro. It told me that after 30 years my interest would become negative and i would start loosing money. Didnt want to accept the error.
- russfink 6mo agoLLM’s: sometimes wrong but never in doubt.
- FeepingCreature 6mo agoErrors compounding is a meme. In iterated as well as verifiable domains, errors dilute instead of compounding because the llm has repeated chances to notice its failure.
- coldtea 6mo agoYes, just use random results. You’ve just saved yourself weeks or months of work of gathering actual results.
- nickjj 6mo agoYeah, ChatGPT's paid version is wildly inaccurate on very important and very basic things. I never got onboard with AI to begin with but nowadays I don't even load it unless I'm really stuck on something programming related.
- drnick1 6mo ago> how are you gonna trust something that can casually make such obvious mistakes? In many cases, a human can review the content generated, and still save a huge amount of time. LLMs are incredibly good at generating contracts, random business emails, and doing pointless homework for students.
- gf000 6mo agoAnd humans are incredibly bad at "skimming through this long text to check for errors", so this is not a happy pairing. As for the homework, there is obviously a huge category that is pointless. But it should not be that way, and the fundamental idea behind homework is sound and the only way something can be properly learnt is by doing exercises and thinking through it yourself.