3 ms·
I guess I'm skeptical that this actually improves performance. I'm worried that the middle man, the tool outputs, can strip useful context that the agent actual
by sethcronin 7mo ago
I guess I'm skeptical that this actually improves performance. I'm worried that the middle man, the tool outputs, can strip useful context that the agent actually needs to diagnose.
- thebeas 7mo agoThat's why give the chance to the model to call expand() in case if it needs more context. We know it's counterintuitive, so we will add the benchmarks to the repo soon. Given our observations, the performance depends on the task and the model itself, most visible on long-running tasks
- fcarraldo 7mo agoHow does the model know it needs more context?
- thebeas 7mo agoWe provide the model with a tool, we call expand() that allows the model to get access to more context if needed by using it. We state this directly appended into the outputs so the model knows exactly where the lines were removed from.
- kingo55 7mo agoPresumably in much the same way it knows it needs to use to calls for reaching its objective.
- Zetaphor 7mo agoI'd argue not, as with tool calls it has available to it at all times a description of what each tool can be used for. There's plenty of intermediate but still important information that could be compacted away, and unless there was a logical reason to go looking for it the model doesn't know what it doesn't know.
- ivzak 7mo agoYou’re right - poor compression can cause that. But skipping compression altogether is also risky: once context gets too large, models can fail to use it properly even if the needed information is there. So the way to go is to compress without stripping useful context, and that’s what we are doing
- backscratches 7mo agoEdit your llm generated comment or at least make it output in a less annoying llm tone. It wastes our time.
- myrak 7mo ago[dead]