5 ms·

I'm not sure if I have the right mental model for a "skill". It's basically a context-management tool? Like a skill is a brief description of something, and if

by lacker 10mo ago

I'm not sure if I have the right mental model for a "skill". It's basically a context-management tool? Like a skill is a brief description of something, and if the model decides it wants the skill based on that description, then it pulls in the rest of whatever amorphous stuff the skill has, scripts, documents, what have you. Is this the right way to think about it?

canadiantim 10mo ago
I think it’s also important to think of skills in the context of tasks, so when you want an agent to perform a specialized task, then this is the context, the resources and scripts it needs to perform the task.
- hadlock 10mo ago
  I'm excited to use this with the Ghidra cli mode to rapidly decompile physics engines from various games. Do I want my flight simulator to behave like the Cessna like in flight simulator 3.0 in the air? Codex can already do that. Do I want the plane to handle like Yoshi from Mario Kart 64 when taxiing? It hasn't been done yet but Claude code is apparently pretty good at pulling apart n64 roms so that seems within the realm of possibility.
simonw 10mo ago
It's a folder with a markdown file in it plus optional additional reference files and executable scripts. The clever part is that the markdown file has a section in it like this: https://github.com/datasette/skill/blob/a63d8a2ddac9db8225ee3b2b62724932f6ca49c6/SKILL.md?plain=1#L1-L4 https://github.com/datasette/skill/blob/a63d8a2ddac9db8225ee... --- name: datasette-plugins description: "Writing Datasette plugins using Python and the pluggy plugin system. Use when Claude needs to: (1) Create a new Datasette plugin, (2) Implement plugin hooks like prepare_connection, register_routes, render_cell, etc., (3) Add custom SQL functions, (4) Create custom output renderers, (5) Add authentication or permissions logic, (6) Extend Datasette's UI with menus, actions, or templates, (7) Package a plugin for distribution on PyPI" --- On startup Claude Code / Codex CLI etc scan all available skills folders and extract just those descriptions into the context. Then, if you ask them to do something that's covered by a skill, they read the rest of that markdown file on demand before going ahead with the task.
- behnamoh 10mo ago
  why did this simple idea take so long to become available? I remember even in llama 2 days I was doing this stuff, and that model didn't even function call.
  - simonw 10mo ago
    Skills only work if you have a full blown code execution environment with a model that can run ls and cat and execute scripts and suchlike. The models are really good at driving those environments now which makes skills the right idea at the right time.
    - jstummbillig 10mo ago
      Why do you need code execution envs? Could the skill not just be a function over a business process, do a then b then c?
      - steilpass 10mo ago
        Turns out that basic shell commands are a really powerful for context management. And you get tools which run in shells for free. But yes. Other agent platforms will adopt this pattern.
        true2octave 10mo ago
        I prefer to provide CLIs to my agent I find it powerful how it can leverage and self-discover the best way to use a CLI and its parameters to achieve its goals It feels more powerful than providing pre-defined set functions as MCP that will have less flexibility as a CLI
    - deleted 10mo ago
      [deleted]
  - NiloCK 10mo ago
    I still don't really understand `skills` as ... anything? You said yourself that you've been doing this since llama 2 days - what do you mean by "become available"? It is useful in a user-education sense to communicate that it's good to actively document useful procedures like this, and it is likely a performance / utilization boost that the models are tuned or prompt-steered toward discovering this stuff in a conventional location. But honestly reading about skills mostly feels like reading: > # LLM provider has adopted a new paradigm: prompts > What's a prompt? > You tell the LLM what you'd like to do, and it tries to do it. OR, you could ask the LLM a question and it will answer to the best of its ability. Obviously I'm missing something.
jmalicki 10mo ago
Yes. I find these very useful for enforcing e.g. skills like debugging, committing code, make prs, responding to pr feedback from ai review agents, etc. without constantly polluting the context window. So when it's time to commit, make sure you run these checks, write a good commit message, etc. Debugging is especially useful since AI agents can often go off the rails and go into loops rewriting code - so it's in a skill I can push for "read the log messages. Inserting some more useful debug assertions to isolate the failure. Write some more unit tests that are more specific." Etc.
prescriptivist 10mo ago
Skills have a lot of uses, but one in particular I like is replacing one off MCP server usage. You can use (or write) an MCP server for you CI system and then add the instructions to your AGENTS.md to query the CI MCP for build results for the current branch. Then you need to find a way to distribute the MCP server so the rest of the team can use it or cook it into your dev environment setup. But all you really care about is one tool in the MCP server, the build result. Or... You can hack together a shell, python, whatever script that fetches build results from your CI server, dumps them to stdout in a semi structured format like markdown, then add a 10-15 line SKILL.md and you have the same functionality -- the skill just executes the one-off script and reads the output. You package the skill with the script, usually in a directory in the project you are working on, but you can also distribute them as plugins (bundles) that claud code can install from a "repository", which can just be a private git repo. It's a little UNIX-y in a way, little tools that pipe output to another tool and they are useful in a standalone context or in a chain of tools. Whereas MCP is a full blown RPC environment (that has it's uses, where appropriate).
- wiether 10mo ago
  How do you manage the credentials to requests your CI server in this case? They are hardcoded in the script associated to your SKILL?
  - true2octave 10mo ago
    Credentials are tied to the service principal of the user It’s straightforward for cloud services
marwamc 10mo ago
My understanding is this: A skill is made up of SKILL.md which is what tells claude how and when to use this skill. I'm a bit of a control freak so I'll usually explicitly direct claude to "load the wireframe-skill" and then do X. Now SKILL.md can have references to more finegrained behaviors or capabilities of our skill. My skills generally tend to have a reference/{workflows,tools,standards,testing-guide,routing,api-integration}.md. These references are what then gets "progressively loaded" into the context. Say I asked claude to use the wireframe-skill to create profileView mockup. While creating the wireframe, claude will need to figure out what API endpoints are available/relevant for the profileView and the response types etc. It's at this point that claude reads the references/api-integration.md file from the wireframe skill. After a while I found I didn't like the progressive loading so I usually direct claude to load all references in the skill before proceeding - this usually takes up maybe 20k to 30k tokens, but the accuracy and precision (imagined or otherwise ha!) is worth it for my use cases.
- kxrm 10mo ago
  > I'm a bit of a control freak so I'll usually explicitly direct claude to "load the wireframe-skill" and then do X. You shouldn't do this, it's generally considered bad practice. You should be optimizing your skill description. Often times if I am working with Claude Code and it doesn't load I skill, I ask it why it missed the skill. It will guide me to improving the skill description so that it is picked up properly next time. This iteration on skill description has allowed skills to stay out of context until they are needed rather predictably for me so far.
  - adastra22 10mo ago
    There are different ways to use the tool. If you chat with the model, you want it to naturally pick the right tool to use based on vibes and context so you don’t have to repeat yourself. If you are plugging a call it Claude code within a larger, structured workflow, you want the tool selection to be deterministic.
  - rane 10mo ago
    It's not enough. Sometimes skills just randomly won't be invoked.
delaminator 10mo ago
Claude Code is not very good at “remembering” its skills. Maybe they get compacted out of the context. But you can call upon them manually. I often do something like “using your Image Manipulation skill, make the icons from image.png” Or “use your web design skill to create a design for the front end” Tbh i do like that. I also get Claude to write its own skills. “Using what we learned about from this task, write a skill document called /whatever/using your writing skills skill” I have a GitHub template including my skills and commands, if you want to see them. https://github.com/lawless-m/claude-skills https://github.com/lawless-m/claude-skills
- Sammi 10mo ago
  I'm kinda confused about why this even is something that we need an extra feature for when it's basically already built in to the agentic development feature. I just keep a folder of md files and I add whatever one is relevant when it's relevant. It's kinda straight forward to do... Just like you I don't edit much in these files on my own. Mostly just ask the model to update an md file whenever I think we've figured out something new, so the learning sticks. I have files for test writing, backend route writing, db migration writing, frontend component writing etc. Whenever a section gets too big to live in agents.md it gets it's own file.
  - jorl17 10mo ago
    Because the concept of skills is not tied to code development :) Of course if that's what you're talking about, you are already very close to the "interface" that skills are presented in, and they are obvious (and perhaps not so useful) But think of your dad or grandma using a generic agent, and simply selecting that they want to have certain skills available to it. Don't even think of it as a chat interface. This is just some option that they set in their phone assistant app. Or, rather, it may be that they actually selected "Determine the best skills based on context", and the assistant has "skill packs" which it periodically determines it needs to enable based on key moments in the conversation or latest interactions. These are all workarounds for the problems of learning, memory...and, ultimately, limited context. But they for sure will be extremely useful.
  - 10mo ago
taytus 10mo ago
Easy, let me try to explain: You want to achieve X, so you ask your AI companion, "How do I do X?" Your companion thinks and tries a couple of things, and they eventually work. So you say, "You know what, next time, instead of figuring it out, just do this"... that is a skill. A recipe for how to do things.

by lacker 10mo ago

canadiantim 10mo ago

I think it’s also important to think of skills in the context of tasks, so when you want an agent to perform a specialized task, then this is the context, the resources and scripts it needs to perform the task.

hadlock 10mo ago
I'm excited to use this with the Ghidra cli mode to rapidly decompile physics engines from various games. Do I want my flight simulator to behave like the Cessna like in flight simulator 3.0 in the air? Codex can already do that. Do I want the plane to handle like Yoshi from Mario Kart 64 when taxiing? It hasn't been done yet but Claude code is apparently pretty good at pulling apart n64 roms so that seems within the realm of possibility.

simonw 10mo ago

It's a folder with a markdown file in it plus optional additional reference files and executable scripts. The clever part is that the markdown file has a section in it like this: https://github.com/datasette/skill/blob/a63d8a2ddac9db8225ee3b2b62724932f6ca49c6/SKILL.md?plain=1#L1-L4 https://github.com/datasette/skill/blob/a63d8a2ddac9db8225ee... --- name: datasette-plugins description: "Writing Datasette plugins using Python and the pluggy plugin system. Use when Claude needs to: (1) Create a new Datasette plugin, (2) Implement plugin hooks like prepare_connection, register_routes, render_cell, etc., (3) Add custom SQL functions, (4) Create custom output renderers, (5) Add authentication or permissions logic, (6) Extend Datasette's UI with menus, actions, or templates, (7) Package a plugin for distribution on PyPI" --- On startup Claude Code / Codex CLI etc scan all available skills folders and extract just those descriptions into the context. Then, if you ask them to do something that's covered by a skill, they read the rest of that markdown file on demand before going ahead with the task.

behnamoh 10mo ago
why did this simple idea take so long to become available? I remember even in llama 2 days I was doing this stuff, and that model didn't even function call.
- simonw 10mo ago
  Skills only work if you have a full blown code execution environment with a model that can run ls and cat and execute scripts and suchlike. The models are really good at driving those environments now which makes skills the right idea at the right time.
  - jstummbillig 10mo ago
    Why do you need code execution envs? Could the skill not just be a function over a business process, do a then b then c?
    - steilpass 10mo ago
      Turns out that basic shell commands are a really powerful for context management. And you get tools which run in shells for free. But yes. Other agent platforms will adopt this pattern.
      - true2octave 10mo ago
        I prefer to provide CLIs to my agent I find it powerful how it can leverage and self-discover the best way to use a CLI and its parameters to achieve its goals It feels more powerful than providing pre-defined set functions as MCP that will have less flexibility as a CLI
  - deleted 10mo ago
    [deleted]
- NiloCK 10mo ago
  I still don't really understand `skills` as ... anything? You said yourself that you've been doing this since llama 2 days - what do you mean by "become available"? It is useful in a user-education sense to communicate that it's good to actively document useful procedures like this, and it is likely a performance / utilization boost that the models are tuned or prompt-steered toward discovering this stuff in a conventional location. But honestly reading about skills mostly feels like reading: > # LLM provider has adopted a new paradigm: prompts > What's a prompt? > You tell the LLM what you'd like to do, and it tries to do it. OR, you could ask the LLM a question and it will answer to the best of its ability. Obviously I'm missing something.

jmalicki 10mo ago

Yes. I find these very useful for enforcing e.g. skills like debugging, committing code, make prs, responding to pr feedback from ai review agents, etc. without constantly polluting the context window. So when it's time to commit, make sure you run these checks, write a good commit message, etc. Debugging is especially useful since AI agents can often go off the rails and go into loops rewriting code - so it's in a skill I can push for "read the log messages. Inserting some more useful debug assertions to isolate the failure. Write some more unit tests that are more specific." Etc.

prescriptivist 10mo ago

Skills have a lot of uses, but one in particular I like is replacing one off MCP server usage. You can use (or write) an MCP server for you CI system and then add the instructions to your AGENTS.md to query the CI MCP for build results for the current branch. Then you need to find a way to distribute the MCP server so the rest of the team can use it or cook it into your dev environment setup. But all you really care about is one tool in the MCP server, the build result. Or... You can hack together a shell, python, whatever script that fetches build results from your CI server, dumps them to stdout in a semi structured format like markdown, then add a 10-15 line SKILL.md and you have the same functionality -- the skill just executes the one-off script and reads the output. You package the skill with the script, usually in a directory in the project you are working on, but you can also distribute them as plugins (bundles) that claud code can install from a "repository", which can just be a private git repo. It's a little UNIX-y in a way, little tools that pipe output to another tool and they are useful in a standalone context or in a chain of tools. Whereas MCP is a full blown RPC environment (that has it's uses, where appropriate).

wiether 10mo ago
How do you manage the credentials to requests your CI server in this case? They are hardcoded in the script associated to your SKILL?
- true2octave 10mo ago
  Credentials are tied to the service principal of the user It’s straightforward for cloud services

marwamc 10mo ago

My understanding is this: A skill is made up of SKILL.md which is what tells claude how and when to use this skill. I'm a bit of a control freak so I'll usually explicitly direct claude to "load the wireframe-skill" and then do X. Now SKILL.md can have references to more finegrained behaviors or capabilities of our skill. My skills generally tend to have a reference/{workflows,tools,standards,testing-guide,routing,api-integration}.md. These references are what then gets "progressively loaded" into the context. Say I asked claude to use the wireframe-skill to create profileView mockup. While creating the wireframe, claude will need to figure out what API endpoints are available/relevant for the profileView and the response types etc. It's at this point that claude reads the references/api-integration.md file from the wireframe skill. After a while I found I didn't like the progressive loading so I usually direct claude to load all references in the skill before proceeding - this usually takes up maybe 20k to 30k tokens, but the accuracy and precision (imagined or otherwise ha!) is worth it for my use cases.

kxrm 10mo ago
> I'm a bit of a control freak so I'll usually explicitly direct claude to "load the wireframe-skill" and then do X. You shouldn't do this, it's generally considered bad practice. You should be optimizing your skill description. Often times if I am working with Claude Code and it doesn't load I skill, I ask it why it missed the skill. It will guide me to improving the skill description so that it is picked up properly next time. This iteration on skill description has allowed skills to stay out of context until they are needed rather predictably for me so far.
- adastra22 10mo ago
  There are different ways to use the tool. If you chat with the model, you want it to naturally pick the right tool to use based on vibes and context so you don’t have to repeat yourself. If you are plugging a call it Claude code within a larger, structured workflow, you want the tool selection to be deterministic.
- rane 10mo ago
  It's not enough. Sometimes skills just randomly won't be invoked.

delaminator 10mo ago

Claude Code is not very good at “remembering” its skills. Maybe they get compacted out of the context. But you can call upon them manually. I often do something like “using your Image Manipulation skill, make the icons from image.png” Or “use your web design skill to create a design for the front end” Tbh i do like that. I also get Claude to write its own skills. “Using what we learned about from this task, write a skill document called /whatever/using your writing skills skill” I have a GitHub template including my skills and commands, if you want to see them. https://github.com/lawless-m/claude-skills https://github.com/lawless-m/claude-skills

Sammi 10mo ago
I'm kinda confused about why this even is something that we need an extra feature for when it's basically already built in to the agentic development feature. I just keep a folder of md files and I add whatever one is relevant when it's relevant. It's kinda straight forward to do... Just like you I don't edit much in these files on my own. Mostly just ask the model to update an md file whenever I think we've figured out something new, so the learning sticks. I have files for test writing, backend route writing, db migration writing, frontend component writing etc. Whenever a section gets too big to live in agents.md it gets it's own file.
- jorl17 10mo ago
  Because the concept of skills is not tied to code development :) Of course if that's what you're talking about, you are already very close to the "interface" that skills are presented in, and they are obvious (and perhaps not so useful) But think of your dad or grandma using a generic agent, and simply selecting that they want to have certain skills available to it. Don't even think of it as a chat interface. This is just some option that they set in their phone assistant app. Or, rather, it may be that they actually selected "Determine the best skills based on context", and the assistant has "skill packs" which it periodically determines it needs to enable based on key moments in the conversation or latest interactions. These are all workarounds for the problems of learning, memory...and, ultimately, limited context. But they for sure will be extremely useful.
- 10mo ago

taytus 10mo ago

Easy, let me try to explain: You want to achieve X, so you ask your AI companion, "How do I do X?" Your companion thinks and tries a couple of things, and they eventually work. So you say, "You know what, next time, instead of figuring it out, just do this"... that is a skill. A recipe for how to do things.