4 ms·
In my case, where I see it most often is when the LLM has to rework something multiple times, and the feedback loop is vague (especially when all I have to give
by overgard 5mo ago
In my case, where I see it most often is when the LLM has to rework something multiple times, and the feedback loop is vague (especially when all I have to give it is "no error messages, but it's still broken"). It seems like after the third or fourth try it just kinda goes off the rails. I find that the one-shot quality tends to be a little better, if the slot machine happened to work correctly that time.
- verdverm 5mo agoYou shouldn't be using an LLM directly (web chat style). A proper harness allows an agent to see the errors itself and correct as needed. You can the correct it at higher, more meaningful levels.
- overgard 4mo agoMy experience is in doing this with Claude/Codex/OpenCode with a pretty rigorous setup (AGENTS.md/CLAUDE.md for specific subfolder rules, strict compile/test/lint rules. This isn't me copy-pasting from web chat.
- aleqs 4mo agoI've been building a general linter tool to help keep repos in a consistent and clean shape when working with AI [0]. You can define repo and file layout, structure, hygiene rules and have them checked in pre-commit, in ci, or manually. It also integrates and plays well with AGENTS.md - allowing exporting agent instructions from the alint rule config [1]. [0] https://github.com/asamarts/alint https://github.com/asamarts/alint [1] https://alint.org/agent-friendly-linter/ https://alint.org/agent-friendly-linter/