26 ms·
This feels completely speculative: there's no measure of whether this approach is actually effective. Personally, I'm skeptical: - Having the agent look up th
by sheept 7mo ago
This feels completely speculative: there's no measure of whether this approach is actually effective.
Personally, I'm skeptical:
- Having the agent look up the JSON schemas and skills to use the CLI still dumps a lot of tokens into its context.
- Designing for AI agents over humans doesn't seem very future proof. Much of the world is still designed for humans, so the developers of agents are incentivized to make agents increasingly tolerate human design.
- This design is novel and may be fairly unfamiliar in the LLM's training data, so I'd imagine the agent would spend more tokens figuring this CLI out compared to a more traditional, human-centered CLI.
- gck1 7mo agoYeah, people seem to forget one of the L's in LLM stands for Language, and human language is likely the largest chunk in training data. A cli that is well designed for humans is well designed for agents too. The only difference is that you shouldn't dump pages of content that can pollute context needlessly. But then again, you probably shouldn't be dumping pages of content for humans either.
- Smaug123 7mo agoIt's not obvious that human language is or should be the largest amount of training data. It's much easier to generate training data from computers than from humans, and having more training data is very valuable. In paticular, for example, one could imagine creating a vast number of debugging problems, with logs and associated command outputs, and training on them.
- rkagerer 7mo agoI also feel like it's just a matter of time until someone cracks the nut of making agents better understand GUI's and more adept at using them. Is there progress happening in that trajectory?
- Imustaskforhelp 7mo ago> Is there progress happening in that trajectory? There was a recent Hackernews post which had a novel approach about making agents interact with GUI/computer-use https://news.ycombinator.com/item?id=47125014 https://news.ycombinator.com/item?id=47125014: The First Fully General Computer Action Model : https://si.inc/posts/fdm1/ https://si.inc/posts/fdm1/ Hope this helps
- magospietato 7mo agoSurely the skill for a cli tool is a couple of lines describing common usage, and a description of the help system?
- sheept 7mo agoSure, but the post itself brags, > gws ships 100+ SKILL.md files Which must altogether be hundreds of lines of YAML frontmatter polluting your context.
- justinwp 7mo agoYou don't need to install all of them.
- danw1979 7mo agoClaude Code, at least, will only load a SKILL.md file into context when it’s invoked by the user or LLM itself, i.e. in demand.
- sheept 7mo agoClaude will load the name and description of each enabled skill into context at startup[0]; the LLM needs to know what it can invoke, after all. It's negligible for a few skills, but a hundred skills will likely have some impact, e.g. deemphasizing other skills by adding noise. [0]: https://platform.claude.com/docs/en/agents-and-tools/agent-skills/overview https://platform.claude.com/docs/en/agents-and-tools/agent-s...
- danw1979 7mo agoGood point. Also, the author specifically mentions OpenClaw in the example skill frontmatter, so I'm wondering if their workflow even involves CC.
- mkobit 7mo agoYou could be thinking of the Claude tool search tool [0][1], which lets it do that on-demand loading. Skills (like another comment mentioned) have a progressive disclosure mechanism built-in. Agent Skills [2] appear to be the common format across most of the tools, but I have no idea if this actually has tool provider buy-in. [0]: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool https://platform.claude.com/docs/en/agents-and-tools/tool-us... [1]: https://www.anthropic.com/engineering/advanced-tool-use https://www.anthropic.com/engineering/advanced-tool-use [2]: : https://agentskills.io/home https://agentskills.io/home
- deleted 7mo ago[deleted]