5 ms·
We've been building out our agent [0], and we've found this to be the case. We actually dialed it back a bunch, because it feels _terrible_. Yes, you get more
by trjordan 1y ago
We've been building out our agent [0], and we've found this to be the case.
We actually dialed it back a bunch, because it feels _terrible_. Yes, you get more correct answers, but it's more akin to giving the agent anxiety. Especially with agents that have access to tools, they'll burn enormous amounts of time on tool calls, trying to get enough information to overcome a motivation that's essentially burned into its identity.
(We saw one conversation where it just browsed social media instead of looking at the code for like 5 minutes, which ... you know, I get it.)
It's been much more effective to make uncertainty or further exploration be part of the agents success criteria.
- BAD: "Critique your own thoughts" -> leads to the agent trying really hard to get it right, but still not willing to actually be wrong
- GOOD: "Expose where your thoughts are unsupported or could benefit from further information" -> leads to the agent producing high-quality results, with loose ends that the user can choose to incorporate, ignore, or correct.
That prompt, combined with dialing up the thinking (either with API or prompt tuning) works much better, because it's sidestepping the training and tuning that's implicitly encouraged it to sound correct at all times.
[0] https://tern.sh https://tern.sh, code migration AI
- djkddkkfkf 1y ago[flagged]
- searls 1y agoYeah this is a great comment and moves it forward. Will incorporate this into my personalization stanza. I agree, I think one reason this strategy has not helped me with Claude Code is that it just leads to endless spinning. Not just tool churn, but ten, twenty, thirty revisions of a file listing that should've just been tested and declared done much sooner. By the time Claude gets around to _executing the code_ it's so far gone it can't even get back to successful compilation. p.s. ChatGPT's hyperbole with this kind of personalization enabled is extremely embarrassing to read. It routinely opens paragraphs with things like "my skepticism is screaming" and I cringe pretty hard as I copy paste the working code underneath it.
- shinycode 1y agoAnxiety for AI ? I don’t follow all developments but it looks « weird » to me. Like AI could benefit from a psychologist or « psychology prompting » in its chain of thought like « don’t panic, you’re the best, you can do it » would have a positive outcome ? Pep talk for AI ?
- scotty79 1y agoIt's not that it gets anxiety. It's just that responses it starts to produce are similar to responses of anxious person. Shaky, unsure, indecisive, chaotic.
- byronic 1y agoYou're 100% right. This kind of prompt just alters the dice probabilities within the word bag. The OP here is ridiculous (as in 'worthy of ridicule') by means of ascribing intent to sentences with a different rhetorical flavor _that was prompted by the person putting in a prompt_. I am now fully of the opinion that LLM proponents should turn off their monitor to reveal the consciousness within the AI
- mrtesthah 1y agoClaude, is that you?
- hnhn34 1y agoI think you had a kneejerk reaction from taking GP's comment too literally. Happens to the best of us. They're saying the LLM is simulating an anxious human and needlessly burning tokens ($$$) as a result, not that the LLM literally has anxiety and needs digital Xanax.
- wyre 1y agoLLM’s typically think and answer questions like a human would, so if prompted to think about if it’s answers are correct or good enough, it’s going to overthink those things, like someone with anxiety would.
- DriverDaily 1y ago
- ForHackernews 1y agoThis comment reads like anthropomorphism. I don't believe an LLM can feel anything, anxiety included.
- natrius 1y agoYou know, it doesn't really seem like a mistake for people to anthropomorphize the thinking machines.
- ForHackernews 1y agoAttributing "thinking" is also a mistake. Anthropic have shown that the "thoughts" it produces to explain what it's "thinking" are (like all the rest of its output) just plausible text, unrelated to the actual node activation happening inside the model: https://transformer-circuits.pub/2025/attribution-graphs/biology.html#dives-cot https://transformer-circuits.pub/2025/attribution-graphs/bio... These tools don't have the capacity for introspection, and they are not doing anything that really resembles the thinking done by a human or an animal.
- librasteve 1y agoi think that we need to subset our definition of “thinking” on lines like - contextualising - packaging / assembling - parsing - recognizing / labeling - comparing / contrasting - analyzing / subsetting - checking - reasoning - introspecting in my view both human and LLMs perform most of these aspects of thinking in a similar way … I suspect that a large fraction of the human brain performs LLM like language manipulation for sure, today’s LLMs lack the last two on the list, and there is probably a rational debate to be had whether these can just emerge in the substrate provided by the LLM-like setting or whether the brain provides some hardwired additions to loop around focus selection and perceive-model-decide-act that will need to be grafted on to LLMs to achieve AGI
- scotty79 1y agoIt's just a shorthand. It's obviously not anxious. Just produces more anxious responses, like an anxious person would. It's no more anthropomorphizing than saying the computer is thinking when the hourglass is spinning.
- frays 1y agoUseful tips - "Expose where your thoughts are unsupported or could benefit from further information" is great. Adding it to my personalization.