7 ms·
Show HN: My LLM CLI tool can run tools now, from Python code or plugins
- behnamoh 1y agowhat are the use cases for llm, the CLI tool? I keep finding tgpt or the bulletin AI features of iTerm2 sufficient for quick shell scripting. does llm have any special features that others don't? am I missing something?
- simonw 1y agoI find it extremely useful as a research tool. It can talk to probably over 100 models at this point, providing a single interface to all of them and logging full details of prompts and responses to its SQLite database. This makes it fantastic for recording experiments with different models over time. The ability to pipe files and other program outputs into an LLM is wildly useful. A few examples: llm -f code.py -s 'Add type hints' > code_typed.py git diff | llm -s 'write a commit message' llm -f https://raw.githubusercontent.com/BenjaminAster/CSS-Minecraft/refs/heads/main/main.css \ -s 'explain all the tricks used by this CSS' It can process images too! https://simonwillison.net/2024/Oct/29/llm-multi-modal/ https://simonwillison.net/2024/Oct/29/llm-multi-modal/ llm 'describe this photo' -a path/to/photo.jpg LLM plugins can be a lot of fun. One of my favorites is llm-cmd which adds the ability to do things like this: llm install llm-cmd llm cmd ffmpeg convert video.mov to mp4 It proposes a command to run, you hit enter to run it. I use it for ffmpeg and similar tools all the time now. https://simonwillison.net/2024/Mar/26/llm-cmd/ https://simonwillison.net/2024/Mar/26/llm-cmd/ I'm getting a whole lot of coding done with LLM now too. Here's how I wrote one of my recent plugins: llm -m openai/o3 \ -f https://raw.githubusercontent.com/simonw/llm-hacker-news/refs/heads/main/llm_hacker_news.py \ -f https://raw.githubusercontent.com/simonw/tools/refs/heads/main/github-issue-to-markdown.html \ -s 'Write a new fragments plugin in Python that registers issue:org/repo/123 which fetches that issue number from the specified github repo and uses the same markdown logic as the HTML page to turn that into a fragment' I wrote about that one here: https://simonwillison.net/2025/Apr/20/llm-fragments-github/ https://simonwillison.net/2025/Apr/20/llm-fragments-github/ LLM was also used recently in that "How I used o3 to find CVE-2025-37899, a remote zeroday vulnerability in the Linux kernel’s SMB implementation" story - to help automate running 100s of prompts: https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899-a-remote-zeroday-vulnerability-in-the-linux-kernels-smb-implementation/ https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-...
- th0ma5 1y ago"LLM was used to find" is not what they did > had I used o3 to find and fix the original vulnerability I would have, in theory [...] they ran a scenario that they thought could have lead to finding it, which is pretty much not what you said. We don't know how much their foreshadowing crept into their LLM context, and even the article says it was also sort of chance. Please be more precise and don't give into these false beliefs of productivity. Not yet at least.
- simonw 1y agoI said "LLM was also used recently in that..." which is entirely true. They used my LLM CLI tool as part of the work they described in that post.
- th0ma5 1y agoVery fair, I expect others to confuse what you mean productivity of your tool called LLM vs. the doubt that many have on the actually productivity of LLM the large language model concept.
- setheron 1y agoWow what a great overview; is there a big doc to see all these options? I'd love to try it -- I've been trying `gh` copilot pulgin but this looks more appealing.
- simonw 1y agoI really need to put together a better tutorial - there's a TON of documentation but it's scattered across a bunch of different places: - The official docs: https://llm.datasette.io/ https://llm.datasette.io/ - The workshop I gave at PyCon a few weeks ago: https://building-with-llms-pycon-2025.readthedocs.io/ https://building-with-llms-pycon-2025.readthedocs.io/ - The "New releases of LLM" series on my blog: https://simonwillison.net/series/llm-releases/ https://simonwillison.net/series/llm-releases/ - My "llm" tag, which has 195 posts now! https://simonwillison.net/tags/llm/ https://simonwillison.net/tags/llm/
- furyofantares 1y agoI don't use llm, but I have my own "think" tool (with MUCH less support than llm, it just calls openai + some special prompt I have set) and what I use it for is when I need to call an llm from a script. Most recently I wanted a script that could produce word lists from a dictionary of 180k words given a query, like "is this an animal?" The script breaks the dictionary up into chunks of size N (asking "which of these words is an animal? respond with just the list of words that match, or NONE if none, and nothing else"), makes M parallel "think" queries, and aggregates the results in an output text file. I had Claude Code do it, and even though I'm _already_ talking to an LLM, it's not a task that I trust an LLM to do without breaking the word list up into much smaller chunks and making loads of requests.
- cyanydeez 1y agoyoure pnly a few steps away from creating a LLM synaptic network
- furyofantares 1y agoI'm automating spending money at an exponential rate.
- behnamoh 1y agounrelated note: your blog is nice and I've been following you for a while, but as a quick suggestion: could you make the code blocks (inline or not) highlighted and more visible?
- simonw 1y agoI have syntax highlighting for blocks of Python code - e.g. this one https://simonwillison.net/2025/May/27/llm-tools/#tools-in-the-llm-python-api https://simonwillison.net/2025/May/27/llm-tools/#tools-in-th... - is that not visible enough? This post has an unusually large number of code blocks without syntax highlighting since they're copy-pasted outputs from the debug tool which isn't in any formal syntax.
- th0ma5 1y ago[flagged]
- dang 1y agoComments like this break both the HN guidelines (https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html) and the Show HN guidelines (https://news.ycombinator.com/showhn.html https://news.ycombinator.com/showhn.html). Can you please review those and please not post like this?
- th0ma5 1y agoI don't know the context that could be cordial when there is so much outright dishonesty about the state of LLM uptake. Can we have some guidance about how to call out bullshit? I know that the fake it to you make it or sell the shovels to the gold rush people are staples on here, but I don't what to do with a technology that is purely for that and seemingly for nothing else. Why do all of the breathless pro-LLM posts get pushed to the top of every LLM story, and you have to go to the second comment aways to see the avalanche of people calling bullshit? Thank you for your time moderating, it would be helpful to understand the guidance in face of the brigrading on here.
- pvg 1y agoCan we have some guidance about how to call out bullshit? That's right in the site docs linked above which you should check out.
- th0ma5 1y agoI guess I'll need clarification -- getting trapped in a non-productive quagmire is the experience of a lot of people. The author chooses when to use terms in different contexts to get out of arguments ... and also, we still don't have any good way to test these things, so I'm not sure where I'm running afoul of the rules when it seems like everyone else is.
- oliviergg 1y agoThank you for this release. I believe your library is a key component to unlocking the potential of LLMs without the limitations/restricitions of existing clients. Since you released version 0.26 alpha, I’ve been trying to create a plugin to interact with a some MCP server, but it’s a bit too challenging for me. So far, I’ve managed to connect and dynamically retrieve and use tools, but I’m not yet able to pass parameters.
- simonw 1y agoYeah I had a bit of an experiment with MCP this morning, to see if I could get a quick plugin demo out for it. It's a bit tricky! The official mcp Python library really wants you to run asyncio and connect to the server and introspect the available tools.
- mihau 1y agoHi Simon! I'm a heavy user of the llm tool, so as soon as I saw your post, I started tinkering with MCP. I’ve just published an alpha version that works with stdio-based MCP servers (tested with @modelcontextprotocol/server-filesystem) - https://github.com/Virtuslab/llm-tools-mcp https://github.com/Virtuslab/llm-tools-mcp. Very early stage, so please make sure to use with --ta option (Manually approve every tool execution). The code is still messy and there are a couple of TODOs in the README.md, but I plan to work on it full-time until the end of the week. Some questions: Where do you think mcp.json should be stored? Also, it might be a bit inconvenient to specify tools one by one with -T. Do you think adding a --all-tools flag or supporting glob patterns like -T name-prefix* in llm would be a good idea?
- simonw 1y agoOK this looks like a very promising start! You're using function-based tools at the moment, hence why you have to register each one individually. The alternative to doing that is to use what I call a "toolbox", described here: https://llm.datasette.io/en/stable/python-api.html#python-api-toolbox https://llm.datasette.io/en/stable/python-api.html#python-ap... Those get you two things you need: 1. A single class can have multiple tool methods in it, you just have to specify it once 2. Toolboxes can take configuration With a Toolbox, your plugin could work like this: llm -T 'MCP("path/to/mcp.json")' ... You might even be able to design it such that you don't need a mcp.json at all, and everything gets passed to that constructor. There's one catch: currently you would have to dynamically create the class with methods for each tool, which is possible in Python but a bit messy. I have an open issue to make that better here: https://github.com/simonw/llm/issues/1111 https://github.com/simonw/llm/issues/1111
- sorenjan 1y agoEvery time I update llm I have to reinstall all plugins, like gemini and ollama. My Gemini key is still saved, as are my aliases for my ollama models, so I don't get why the installed plugins are lost.
- simonw 1y agoSorry about that! Presumably you're updating via Homebrew? That blows away your virtual environment, hence why the plugins all go missing. I have an idea to fix that by writing a 'plugins.txt' file somewhere with all of your installed plugins and then re-installing any that go missing - issue for that is here: https://github.com/simonw/llm/issues/575 https://github.com/simonw/llm/issues/575
- sorenjan 1y agoNo, I'm using uv tool just like in that issue. I'll keep an eye on it, at least I know it's not just me.
- tionis 1y agoI'm also using uv tools and fixed it by doing something like this to upgrade: uv tool install llm --upgrade --upgrade --with llm-openrouter --with llm-cmd ...
- johnisgood 1y agoIs the double "--upgrade" a typo?
- tionis 1y agoYes, autocorrect on my phone worked against me
- tinodb 1y agoSame, but put that in a llm-upgrade.sh because the list was getting quite long :)
- dr_kretyn 1y agoMaybe a small plug of own similar library: terminal-agent (https://github.com/laszukdawid/terminal-agent https://github.com/laszukdawid/terminal-agent) which also supports tools and even MCP. There's a limited agentic capability but needs some polishing. Only once I made some progress on own app I learned about this `llm` CLI. Though that one more won't harm.
- deleted 1y ago[deleted]
- swyx 1y agonice one simon - i'm guessing this is mildly related to your observation that everyone is converging on the same set of tools? https://x.com/simonw/status/1927378768873550310 https://x.com/simonw/status/1927378768873550310
- simonw 1y agoActually a total coincidence! I have been trying to ship this for weeks.
- ttul 1y agoGPT-4.1 is a capable model, especially for structured outputs and tool calling. I’ve been using LLMs for my day to day grunt work for two years now and this is my goto as a great combination of cheap and capable.
- simonw 1y agoI'm honestly really impressed with GPT-4.1 mini. It is my default from messing around by their API because it is unbelievably inexpensive and genuinely capable at most of the things I throw at it. I'll switch to o4-mini when I'm writing code, but otherwise 4.1-mini usually does a great job. Fun example from earlier today: llm -f https://raw.githubusercontent.com/BenjaminAster/CSS-Minecraft/refs/heads/main/main.css \ -s 'explain all the tricks used by this CSS' That's piping the CSS from that incredible CSS Minecraft demo - https://news.ycombinator.com/item?id=44100148 https://news.ycombinator.com/item?id=44100148 - into GPT-4.1 mini and asking it for an explanation. The code is clearly written but entirely uncommented: https://github.com/BenjaminAster/CSS-Minecraft/blob/main/main.css https://github.com/BenjaminAster/CSS-Minecraft/blob/main/mai... GPT-4.1 mini's explanation is genuinely excellent: https://gist.github.com/simonw/cafd612b3982e3ad463788dd50287323#response https://gist.github.com/simonw/cafd612b3982e3ad463788dd50287... - it correctly identifies "This CSS uses modern CSS features at an expert level to create a 3D interactive voxel-style UI while minimizing or eliminating JavaScript" and explains a bunch of tricks I hadn't figured out. And it used 3,813 input tokens and 1,291 output tokens - https://www.llm-prices.com/#it=3813&ot=1291&ic=0.4&oc=1.6 https://www.llm-prices.com/#it=3813&ot=1291&ic=0.4&oc=1.6 - that's 0.3591 cents (around a third of a cent).
- puttycat 1y ago> while minimizing or eliminating JavaScript How come it doesn't know for sure?
- simonw 1y agoBecause I only showed it the CSS! It doesn't even get the HTML, it's guessed all of that exclusively from what's in the (uncommented) CSS code. Though it's worth noting that CSS Minecraft was first released three years ago, so there's a chance it has hints about it in the training data already. This is not a meticulous experiment. (I've had a search around though and the most detailed explanation I could find of how that code works is the one I posted on my blog yesterday - my hunch is that it figured it out from the code alone.)
- tantalor 1y agoThis greatly opens up the risk of footguns. The doc [1] warns about prompt injection, but I think a more likely scenario is self-inflicted harm. For instance, you give a tool access to your brokerage account to automate trading. Even without prompt injection, there's nothing preventing the bot from making stupid trades. [1] https://llm.datasette.io/en/stable/tools.html https://llm.datasette.io/en/stable/tools.html
- shepherdjerred 1y agoAny tool can be misused
- tantalor 1y agoYou're missing the point. Most tools are deployed by humans. If they do something bad, we can blame the human for using the tool badly. And we can predict when a bad choice by the human operator will lead to a bad outcome. Letting the LLM run the tool unsupervised is another thing entirely. We do not understand the choices the machines are making. They are unpredictable and you can't root-cause their decisions. LLM tool use is a new thing we haven't had before, which means tool misuse is a whole new class of FUBAR waiting to happen.
- johnisgood 1y agoBut why can we not hold humans responsible in the case of LLM? You do have to go out of your way to do all of these things with an LLM. And it is the human that does it. It is the humans that give it the permission to act on their behalf. We can definitely hold humans responsible. The question is: are we going to?
- tantalor 1y agoI think intent matters. Let's say you are making an AI-controlled radiation therapy machine. You prompt and train and eval the system very carefully, and you are quite sure it won't overdose any patients. Well, that's not really good enough, it can still screw up. But did you do anything wrong? Not really, you followed best practices and didn't make any mistakes. The LLM just sometimes kills people. You didn't intend that at all. I make this point because this is already how these systems work today. But instead of giving you a lethal dose of radiation, it uses slurs or promotes genocide or something else. The builders of those bots didn't intend that, and in all likelihood tried very hard to prevent it. It's not very fair to blame them.
- stavros 1y agoHave you guys had luck with tool calls? I made a simple assistant with access to my calendar, and most models fail to call the tool to add calendar events. GPT-4.1 also regularly tries to gaslight me into believing that it added the event when it didn't call the tool! Overall, I found tool use extremely hit-and-miss, to the point where I'm sure I'm doing something wrong (I'm using the OpenAI Agents SDK, FWIW).
- simonw 1y agoI get the impression that the key to getting great performance out of tool calls is having a really detailed system prompt, with a bunch of examples. Anthropic's system prompt just for their "web_search" tool is over 6,000 tokens long! https://simonwillison.net/2025/May/25/claude-4-system-prompt/#search-instructions https://simonwillison.net/2025/May/25/claude-4-system-prompt...
- stavros 1y agoThat's really interesting, thanks Simon! I was kind of expecting the LLM to be trained already, I'll use Claude's prompt and see. Thanks again.
- xrd 1y agoIs no one else bothered by that way of using tools? Tools feel like a way to get deterministic behavior from a very hallucinatory process. But unless you put a very lengthy and comprehensive non-deterministic English statement, you can't effectively use tools. As we all know, the more code, the more bugs. These long and often hidden prompts seem like the wrong way to go. And, this is why I'm very excited about this addition to the llm tool, because it feels like it moves the tool closer to the user and reduces the likelihood of the problem I'm describing.
- simonw 1y agoAs an experienced software engineer I'm bothered about pretty much everything about how we develop things on top of LLMs! I can't even figure out how to write automated tests for them. See also my multi-year obsession with prompt injection and LLM security, which still isn't close to being a solved problem: https://simonwillison.net/tags/prompt-injection/ https://simonwillison.net/tags/prompt-injection/ Yet somehow I can't tear myself away from them. The fact that we can use computers to mostly understand human language (and vision problems as well) is irresistible to me.
- bgwalter 1y ago[flagged]
- simonw 1y agoAre you an alt for th0ma5?
- bgwalter 1y agoAre you paid by OpenAI?
- simonw 1y agoNo. If I was they'd probably be a bit angry about how much time I spend writing about their competitors: https://simonwillison.net/tags/anthropic/ https://simonwillison.net/tags/anthropic/ https://simonwillison.net/tags/gemini/ https://simonwillison.net/tags/gemini/ https://simonwillison.net/tags/mistral/ https://simonwillison.net/tags/mistral/ https://simonwillison.net/tags/qwen/ https://simonwillison.net/tags/qwen/
- th0ma5 1y agoThey are not. I think it is a good criticism though. Many people seem to be touting productivity that is only in the context of productivity towards more LLM inference operations and not productive in the sense of solving real world computing problems. There is a lot of material that suggests positive results are a kind of wish casting and people are not aware of the agency they are bringing to the interaction. The fact that you can hack things that do cool stuff is more of a reflection that you do those things, and that these models are not capable of it. That's why I recommend you work with others and you'll see your concepts that you feel are generalizable are not, and any learnings or insights are not like learning math or how to read, but more like learning a specific video game's rules. This is also why it is enthralling to you because you actually have only the illusion of controlling it.
- minimaxir 1y ago
- kristopolous 1y agoIt's worth noting the streaming markdown renderer I wrote just for this tool: https://github.com/day50-dev/Streamdown https://github.com/day50-dev/Streamdown More background: https://github.com/simonw/llm/issues/12 https://github.com/simonw/llm/issues/12 (Also check out https://github.com/day50-dev/llmehelp https://github.com/day50-dev/llmehelp which features a tmux tool I built on top of Simon's llm. I use it every day. Really. It's become indispensable)
- rpeden 1y agoNeat! I've written streaming Markdown renderers in a couple of languages for quickly displaying streaming LLM output. Nice to see I'm not the only one! :)
- kristopolous 1y agoIt's a wildly nontrivial problem if you're trying to only be forward moving and want to minimize your buffer. That's why everybody else either rerenders (such as rich) or relies on the whole buffer (such as glow). I didn't write Streamdown for fun - there are genuinely no suitable tools that did what I needed. Also various models have various ideas of what markdown should be and coding against CommonMark doesn't get you there. Then there's other things. You have to check individual character width and the language family type to do proper word wrap. I've seen a number of interesting tmux and alacritty bugs in doing multi language support The only real break I do is I render h6 (######) as muted grey. Compare: for i in $(seq 1 6); do printf "%${i}sh${i}\n\n-----\n" | tr " " "#"; done | pv -bqL 30 | sd -w 30 to swapping out `sd` with `glow`. You'll see glow's lag - waiting for that EOF is annoying. Also try sd -b 0.4 or even -b 0.7,0.8,0.8 for a nice blue. It's a bit easier to configure than the usual catalog of themes that requires a compilation after modification like with pygments.
- icarito 1y agoThat's right this is a nontrivial problem that I struggled with too for gtk-llm-chat! I resolved it using the streaming markdown-it-py library.
- nlh 1y agoOk this is great and perfect timing -- I've been playing around with Warp (the terminal) and while I love the idea of their terminal-based "agent" (eg tool loop), I don't love the whole Cursor-esque model of "trust us we'll make a good prompt and LLM calls for you" (and charge you for it), so I was hoping for a simple CLI-based terminal agent just to solve for my lack of shell-fu. I am keenly aware this is a major footgun here, but it seems that a terminal tool + llm would be a perfect lightweight solution. Is there a way to have llm get permission for each tool call the way other "agents" do this? ("llm would like to call `rm -rf ./*` press Y to confirm...") Would be a decent way to prevent letting an llm run wild on my terminal and still provide some measure of protection.
- andresnds 1y agoIsn’t that how the default way codex CLI runs? I.e. without passing —full-auto
- johntash 1y ago> I don't love the whole Cursor-esque model of "trust us we'll make a good prompt and LLM calls for you" (and charge you for it Huh, I've been wondering why a terminal was advertising everywhere and sponsoring random projects. I didn't realize it's not a completely free thing.
- roxolotl 1y agoSimon thank you so much for this tool! I use it daily now since charmbraclet’s Mods[0] doesn’t support anthropics models. And now with tool calling it’ll be even more useful. I am curious though if there’s any appetite for improving performance? It’s noticeably slow to even just print the help on all of my machines(M1 32gb/M2 21g/ryzen 7700 64gb). 0: https://github.com/charmbracelet/mods https://github.com/charmbracelet/mods
- simonw 1y agoHow many plugins do you have installed? We've seen problems in the past where plugins with expensive imports (like torch) slow everything down a lot: https://github.com/simonw/llm/issues/949 https://github.com/simonw/llm/issues/949 I'm interested in tracking down the worst offenders and encouraging them to move to lazy imports instead.
- roxolotl 1y agoI’ve only got the Anthropic and Gemini plugins installed. I’d be happy to do a bit more digging. I’m away for a bit but would be happy to file an issue with more context when I get a chance.
- simonw 1y agoTry running this and see if anything interesting comes out of it: sudo uvx py-spy record -o /tmp/profile.svg -- llm --help
- hanatanaka1984 1y agoGreat work Simon! I use your tool daily. Pipes and easy model switching for local (ollama) and remote makes this very easy to work with.
- prettyblocks 1y agoI've been trying to maintain a (mostly vibe-coded) zsh/omz plugin for tab completions on your LLM cli and the rate at which you release new features makes it tough to keep up! Fortunately this gets me 90% of the way there: llm -f README.md -f llm.plugin.zsh -f completions/_llm -f https://simonwillison.net/2025/May/27/llm-tools/ https://simonwillison.net/2025/May/27/llm-tools/ "implement tab completions for the new tool plugins feature" My repo is here: https://github.com/eliyastein/llm-zsh-plugin https://github.com/eliyastein/llm-zsh-plugin And again, it's a bit of a mess, because I'm trying to get as many options and their flags as I can. I wouldn't mind if anyone has any feedback for me.
- sillysaurusx 1y agoKind of crazy this isn’t sci-fi, it’s just how coding is done now. Future generations are going to wonder how we ever got anything done, the same way we wonder how assembly programmers managed to.
- xk_id 1y agoThe transition from assembly to C was to a different layer of abstraction within the same context of deterministic computation. The transition from programming to LLM prompting is to a qualitatively different context, because the process is no longer deterministic, nor debuggable. So your analogy fails to apply in a meaningful way to this situation.
- pollinations 1y agoWhy isn't it debuggable?
- sillysaurusx 1y agoUltimately it’s about being able to create features within a certain period of time, not just to write code. And in that context the shift from assembly to C and the shift from deterministic programming to LLM prompting seem to have similar magnitudes of impact.
- blocksocial130 1y ago[dead]
- rat87 1y agoNice. I was just looking at how to copy the python version of the barebones ruby agent https://news.ycombinator.com/item?id=43984860 https://news.ycombinator.com/item?id=43984860. I found sw-llm and tried to find how to pass it tools but was having difficulty finding it in the docs
- icarito 1y agoFor all of you using `llm` - perhaps take a look at [Gtk-llm-chat](https://github.com/icarito/gtk-llm-chat https://github.com/icarito/gtk-llm-chat). I put a lot of effort into it - it integrates with `llm` command line tool and with your desktop, via a tray icon and nice chat window. I recently released 3.0.0 with packages for all three major desktop operating systems.
- kristopolous 1y agoInteresting. What do you use it for beyond the normal chatting
- icarito 1y agoI sometimes use llm from the command line, for instance with a fragment, or piping a resource from the web with curl, and then pick up the cid with `llm gtk-chat --cid MYCID`.
- kristopolous 1y agoI'm actually planning on abandoning Simon's infra soon. I want a multi-stream, routing based solution that is more aware of the modern API advancements. The Unix shell is good at being the glue between programs. We've increased the dimensionality with LLMs. Some kind of ports based system like named pipes with consumers and producers. Maybe something like gRPC or NATS (https://github.com/nats-io https://github.com/nats-io). MQTT might also work. Network transparent would be great.
- howmayiannoyyou 1y agoThis is pretty great.
- samuel 1y agoThis is great, and a feature I was hoping for, for a long time. At this point I would have expected something mcp or openapi based but probably is simpler and more flexible this way. Implementing it as plugin shouldn't be hard, I think.
- lynx97 1y agoThis is great! AIUI, llama.cpp does support tools, but I haven't figured out yet what to do to make llm use it. Is there anything I can put into extra-openai-models.yaml to make this work?
- simonw 1y agoThat's likely a change that needs to be made to either https://github.com/simonw/llm-gguf https://github.com/simonw/llm-gguf or https://github.com/simonw/llm-llama-server https://github.com/simonw/llm-llama-server ... OK, I got the second one working! brew install llama.cpp llama-server --jinja -hf unsloth/gemma-3-4b-it-GGUF:Q4_K_XL Then in another window: llm install llm-llama-server llm -m llama-server-tools -T llm_time 'what time is it?' --td Wrote it up here: https://simonwillison.net/2025/May/28/llama-server-tools/ https://simonwillison.net/2025/May/28/llama-server-tools/
- johnisgood 1y agoSimon, at this point you have a lot of LLM-related tools, and I am not sure which one is outdated, which one is the newest, fanciest stuff, which one is the one that one should use (and when), and so forth. Is there a blog post / article that addresses this?
- simonw 1y agoFor my stuff it's basically just the latest https://llm.datasette.io https://llm.datasette.io and its collection of plugins. If you're interested in what I recommend generally that changes a lot, but my most recent piece about that is here: https://simonwillison.net/2025/May/15/building-on-llms/ https://simonwillison.net/2025/May/15/building-on-llms/
- lynx97 1y agoGreat, this works here. I wonder, with extra-openai-models.yaml I was able to set the api_base and vision/audio: true. How do I do this with the llama-server-tools plugin? vision works, but llm refuses to attach audio because it thinks the model does not support audio /which it does). EDIT: I think I just found what I want. There is no need for the plugin, extra-openai-models.yaml just needs "supports_tools: true" and "can_stream: false".
- pawanjswal 1y agoI think, this LLM 0.26 just turned every terminal into a playground for AI-powered tools.
- a_bonobo 1y agoHow does this differ from langchain's tool calling?
- tiniuclx 1y agoThanks for the work you put into this, Simon. I've had a great experience with LLM, it's amazing for quickly iterating on AI application ideas.
- rahimnathwani 1y agoThanks for making this. I used it (0.26a0) last week to create a demo for a customer-facing chatbot using proprietary data. The key elements I had to write: - The system prompt - Tools to pull external data - Tools to do some calculations Your library made the core functionality very easy. Most of the effort for the demo was to get the plumbing working (a nice-looking web UI for the chatbot that would persist the conversation, update nicely if the user refreshed their browser due to a connection issue, and allow the user to start a new chat session). I didn't know about `after_call=print`. So I'm glad I read this blog post!
- chrissoundz 1y agoI think the project should really be given a name other than 'llm'. Not something that can be easily found or identified otherwise.
- WhereIsTheTruth 1y agopip, brew, pipx, uv Can we stop already? stop following webdevs practices
- never_inline 1y agoThey are all compatible with existing stuff, which makes it kinda better than js.
- never_inline 1y agoThanks for writing the LLM CLI. It's been a very useful tool to test out and prototype using various LLM features like multimodal, schema output and now tools as well! I specifically like that I can just write a python function with type annotations and plug it to the LLM.
- aitchnyu 1y agoDo local multimodal llms have low latency? If it can plug into monitor video stream and the accessibility representation of UI (like Shortcat), it could answer if a printer is connected to the computer and preview a full page print and wait for us to hit print. https://shortcat.app/ https://shortcat.app/
- deleted 1y ago[deleted]
- kristel100 1y ago[dead]
- anthk 1y agoAlmost like RiveScript from Perl running code from within a chatbot :) Things now come in full circle :D
- losvedir 1y agoVery cool! I've wondered how exactly, say, Claude Code knows about and uses tools. Obviously, an LLM can be "told" about tools and how to use them, and the harness can kind of manage that. But I assumed Claude Code has a very specific expectation around the tool call "API" that the harness uses, probably reinforced very heavily by some post-training / fine tuning. Do you think your 3rd party tool-calling framework using Claude is at any disadvantage to Anthropic's own framework because of this? Separately, on that other HN post about the GitHub MCP "attack", I made the point that LLMs can be tricked into using up to the full potential of the credential. GitHub has fine-grained auth credentials, and my own company does as well. I would love for someone to take a stab at a credential protocol that the harness can use to generate fine-grained credentials to hand to the LLM. I'm envisioning something where the application (e.g. your `llm` CLI tool) is given a more powerful credential, and the underlying LLM is taught how to "ask for permission" for certain actions/resources, which the user can grant. When that happens the framework gets the scoped credential from the service, which the LLM can then use in tool calls.
- simonw 1y agoThat credentials trick is possible right now using LLM's tool support. You'd have to write a pretty elaborate tool setup which exposes the "ask for more credentials" tool and then prompts the user when that's called. The tool should keep the credentials and never pass the actual tokens back to the LLM, but it can pass e.g. a symbol "creds1" and tell the LLM to request to make calls with "creds1" in future requests.
- consumer451 1y agoHello Simon, sorry for asking about this tangent here, but have you seen this paper? Is it as important as it appears to be? Should this metric be on all system cards? > We evaluate 12 popular LLMs that claim to support contexts of at least 128K tokens. While they perform well in short contexts (<1K), performance degrades significantly as context length increases. At 32K, for instance, 10 models drop below 50% of their strong short-length baselines. Even GPT-4o, one of the top-performing exceptions, experiences a reduction from an almost-perfect baseline of 99.3% to 69.7%. https://arxiv.org/abs/2502.05167 https://arxiv.org/abs/2502.05167
- simonw 1y agoI had not seen that one! That's really interesting. I'd love to see them run that against Gemini 2.5 Pro and Gemini 2.5 Flash, to my understanding they're way ahead of other models on the needle in a haystack tests these days.
- consumer451 1y agoYes, I wish their methodology was run against new models, in an on-going fashion.
- jasonriddle 1y ago@simonw, how much have you used Claude code? I'm wondering if it makes sense to use Claude code as the main driver, but have crafted specialized "llm" based tools for Claude to use. I would love to only have to use llm, but with the amount of functionality that Claude code provides, it seems unlikely that llm would ever support all of its features. Also, I have realized that llm is narrower in scoop, which is something I like about the tool.
- simonw 1y agoI use Claude Code a few times a week but honestly I'm a little intimidated by how quickly it chews through tokens! I think it's not impossible that LLM might one day be able to handle the same kind of processes with the right collection of tool plugins and a really sophisticated system prompt but it's not something I'm shooting for right now - Claude Code is a very complicated system!