2 ms·

And all those "names and one-line descriptions" live in an .md file and of course pollute your context. And then the model will randomly decide to load the .md

by troupo 3mo ago

And all those "names and one-line descriptions" live in an .md file and of course pollute your context. And then the model will randomly decide to load the .md file that is the "skill" randomly at any point. Again polluting the context. There's no magic.

CrompyBlompers 3mo ago
Most people seem to mean "Skills don't pollute the context window [with their entire contents]", they just elide the part in brackets. There is no magic and people should be more precise when discussing skills.
- NiloCK 3mo ago
  Yes, but I expect that the elided part is less important than people assume it is. Token count is a less important factor in context pollution than idea count. The worst of the rot factors are when models latch onto irrelevant information, or over-index on some vague idea/suggestion as if it was a hard direction, and then go off course. The names + one-line descriptions of 10 tools can do as much (or more!) to distract the focus and intentionality of an agent than a 30k token exhaustive API documentation of some tool.
simonw 3mo ago
I've seen implementations of a skill search, where instead of loading all descriptions into the initial prompt there's a tool the agent can call to search available skills and see if one might match their new task.
- troupo 3mo ago
  Search itself pollutes the context. With the need to maintain the tools, the search, the search results etc. in context. And wasting tokens while interpreting results.
majormajor 3mo ago
Is it so unusual to just ... not ... load the file with a description of them all automatically, but have different files listing different ones for different situations, and manually point to them in prompts?
- troupo 3mo ago
  Depends on how the harness injects those files. All harnesses these days auto-discover "skills" and read them anyway, and inject their descriptions, at the very least. Which leads to this anyway: https://news.ycombinator.com/item?id=49183906 https://news.ycombinator.com/item?id=49183906 But yeah. You could have files with skills outside of the auto-discovered skill tree, and invoke them manually.

by troupo 3mo ago

CrompyBlompers 3mo ago

Most people seem to mean "Skills don't pollute the context window [with their entire contents]", they just elide the part in brackets. There is no magic and people should be more precise when discussing skills.

NiloCK 3mo ago
Yes, but I expect that the elided part is less important than people assume it is. Token count is a less important factor in context pollution than idea count. The worst of the rot factors are when models latch onto irrelevant information, or over-index on some vague idea/suggestion as if it was a hard direction, and then go off course. The names + one-line descriptions of 10 tools can do as much (or more!) to distract the focus and intentionality of an agent than a 30k token exhaustive API documentation of some tool.

simonw 3mo ago

I've seen implementations of a skill search, where instead of loading all descriptions into the initial prompt there's a tool the agent can call to search available skills and see if one might match their new task.

troupo 3mo ago
Search itself pollutes the context. With the need to maintain the tools, the search, the search results etc. in context. And wasting tokens while interpreting results.

majormajor 3mo ago

Is it so unusual to just ... not ... load the file with a description of them all automatically, but have different files listing different ones for different situations, and manually point to them in prompts?

troupo 3mo ago
Depends on how the harness injects those files. All harnesses these days auto-discover "skills" and read them anyway, and inject their descriptions, at the very least. Which leads to this anyway: https://news.ycombinator.com/item?id=49183906 https://news.ycombinator.com/item?id=49183906 But yeah. You could have files with skills outside of the auto-discovered skill tree, and invoke them manually.