4 ms·
The example model in the documentation is 4o-mini, you might want to update that to a more recent model. As an aside, 4o-mini came out months before agent skil
by ssgodderidge 5mo ago
The example model in the documentation is 4o-mini, you might want to update that to a more recent model.
As an aside, 4o-mini came out months before agent skills were released… I’m curious how it performs with choosing to load skills in the first place?
- block_dagger 5mo agoThe skill is deterministically added to the prompt by the harness before the target model is invoked. There is no “choosing” to load a skill. You might be confusing skills with tools (MCP etc).
- ssgodderidge 5mo agoThe metadata is loaded by the harness, but the LLM still needs to choose to load the rest of the skill, no?
- block_dagger 5mo agoDefine “load.” It follows the instructions in the prompt - its natural behavior.
- hyperpape 5mo agoConcretely, it has to decide whether it is in a circumstance where that skill is useful, pull the instructions into the context and follow them.
- cassianoleal 5mo agoYep, and as with any other instructions, it can sometimes not pull the skill even if the trigger conditions are there.
- ssgodderidge 5mo agoI was using the term as you used in your comment. I believe the official term is "Activation" however: > Activation: When a task matches a skill’s description, the agent reads the full SKILL.md instructions into context.[1] > Full instructions load only when a task calls for them, so agents can keep many skills on hand with only a small context footprint. [1]: https://agentskills.io/home#how-do-agent-skills-work https://agentskills.io/home#how-do-agent-skills-work
- block_dagger 5mo agoAh, I misunderstood this, thanks for the link. You are correct. I was assuming this system worked like CLAUDE.md in that it was deterministically added to the context without the LLM choosing to add it. My mistake.
- albedoa 5mo agoYou are correct. I'm not sure what the parent is trying to say.
- cyanydeez 5mo agoit depends on the harness. opencode appears to prompt the models with tools and skills when answering questions.
- stingraycharles 5mo agoIt’s an artifact of the documentation being AI generated, they usually pick gpt4-era models, without giving it further thought. For Gemini it seems to always pick 2.5 despite 3.1 being the latest, Claude the 3.5-era models. Not sure what’s preventing AI labs on ensuring this stuff is refreshed during training.
- simonpure 5mo agoI was wondering the same and learned the model doesn't know about itself during training [0] [0] https://developers.googleblog.com/closing-the-knowledge-gap-with-agent-skills/ https://developers.googleblog.com/closing-the-knowledge-gap-...
- cyanydeez 5mo agothe model doesn't know itself, but all these larger models are generating a significant amount of synthetic data from the prior models, and the prior models are all context bloated renditions; you fill the KV cache with whatever alignment you want, and then generate synthetic data. That training on existing models is what brings out various other things about other models; then there's models that are just like snowballs, where you build one iteration, then you give it it's identity, then you train on that with the same synthetic generaiton. So a model could generation include at some point it's own name.
- stingraycharles 5mo agoI don’t think what you’re saying makes a lot of sense. You don’t “fill the KV cache with whatever alignment you want.” That doesn’t exist. The KV cache is an inference optimization, and is populated by running tokens through the model. Synthetic data is generated by other models, and yes this is often where identity propagates. I think with the snowballing you mean things like iterative self distillation? That’s definitely not done unsupervised, because of the risk of model collapse, and typically heavily curated and/or mixed with real data.