2 ms·
I have a tool wrapper that captures the output of anything and allows the LLM to query it later, to save on tokens. It “smartly” truncates the output (basically
by tobyhinloopen 2mo ago
I have a tool wrapper that captures the output of anything and allows the LLM to query it later, to save on tokens. It “smartly” truncates the output (basically like Node’s util.inspect) and allows the LLM to expand truncated content.
It basically is called like “capture some-cli” and it… captures the CLI output, outputting a subset of it + a handle to continue querying.
This for me solves the danger of a tool returning tons of content.
- layer8 2mo ago> captures the output of anything and allows the LLM to query it later Otherwise known as a “file”. ;)
- dezgeg 2mo agoDon't most harnesses already do that for bash commands?
- skyyr 2mo agoI’ve been trying headroom-ai for this reason. That project also reduces tokens in other clever ways. Have you looked into it?