3 ms·
LLMs can create infinite worlds in the error message it’s receiving. It probably needs some outside signal to stop and re-assess. I don’t think LLMs have any ab
by coffeebeqn 2y ago
LLMs can create infinite worlds in the error message it’s receiving. It probably needs some outside signal to stop and re-assess. I don’t think LLMs have any ability to reason if they’re lost in their own world on their own. They’ll just keep creating new less and less coherent context for themselves
- davedx 2y agoFor sure - but if I'm paying for a tool like Devin then I'd expect the infrastructure around it to do things like stop it if it looks like that has happened. What you often see with agentic systems is that there's an agent whose role is to "orchestrate", and that's the kind of thing the orchestrator would do: every 10 minutes or so, check the output and elapsed time and decide if the "developer" agent needs a reality check.
- someothherguyy 2y agoIf you correct an LLM based agent coder, you are always right. Often, if you give it advice, it pretends like it understands you, then goes on to do something different from what it said it was going to do. Likewise, it will outright lie to you telling you it did things it didn't do. (In my experience)
- rsynnott 2y agoSo when people say these things are like junior developers, they really mean that they’re like the worst _stereotype_ of junior developers, then?
- deleted 2y ago[deleted]
- tobyhinloopen 2y agoYou can maybe have a supervisor AI agent trigger a retry / new approach
- nejsjsjsbsb 2y agoThey need impatience!