7 ms·

Something I realized about this category of tool (I call them "terminal agents" but that already doesn't work now there's an official VS Code extension for this

by simonw 1y ago

Something I realized about this category of tool (I call them "terminal agents" but that already doesn't work now there's an official VS Code extension for this - maybe just "coding agents" instead) is that they're actually an interesting form of general agent. Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. They're incredibly dangerous to use if you don't know how to isolate them in a safe container but wow the stuff you can do with them is fascinating.

willio58 1y ago
One thing I really like using them for is refactoring/reorganizing. The tedious nature of renaming, renaming all implementations, moving files around, creating/deleting folders, updating imports exports, all melts away when you task an agent with it. Of course this assumes they are good enough to do them with quality, which is like 75% of the time for me so far.
- singularity2001 1y ago
  does it use the smart refractoring hooks of the IDEs or does it do blunt text replacement
  - 0x696C6961 1y ago
    The later
  - t0mas88 1y ago
    Currently it's very slow because it does text replace. It would be way faster if it could use the IDE functions via an MCP.
  - Yeroc 1y ago
    Blunt text replacement so far. There are third-party VSCode MCP and LSM MCP servers out there that DO expose those higher-level operations. I haven't tried them out myself -- but it's on my list because I expect they'd cut down on token use and improve latency substantially. I expect Anthropic to eventually build that into their IDE integration.
- dgunay 1y ago
  I've found that it can be hard or expensive for the agent to do "big but simple" refactors in some cases. For example, I recently tasked one with updating all our old APIs to accept a more strongly typed user ID instead of a generic UUID type. No business logic changes, just change the type of the parameter, and in some cases be wary of misleading argument names by lazy devs copy pasting code. This ended up burning through the entire context window of GPT-5-codex and cost the most $ of anything I've ever tasked an agent with.
  - felixyz 1y ago
    The way I do this is I task the agent with writing a script which in turn does the updates. I can inspect that script, and I can run it on a subset of files/folders, and I can git revert changes if something went wrong and ask the agent to fix the script or fine-tune it myself. And I don't burn through tokens :) Also, another important factor (as in everything) is to do things in many small steps, instead of giving one big complicated prompt.
- golergka 1y ago
  Especially when you work with a language where an unfinished refactoring with give you the type error.
athrowaway3z 1y ago
They're only as dangerous as the capabilities you give them. I just created a `codex` and `claude` user on my Linux box and practically always run in yolo mode. I've not had a problem so far. Also, I think shellagent sounds cooler.
- simonw 1y ago
  That's a great way to run this stuff. I expect the portion of Claude Code users who have a dedicated user setup like this is pretty tiny!
  - steveklabnik 1y ago
    One nice thing is that Anthropic provides a sample DevContainer: https://github.com/anthropics/claude-code/tree/main/.devcontainer https://github.com/anthropics/claude-code/tree/main/.devcont... Not the exact setup, but also pretty solid.
- globular-toast 1y ago
  I tried this but it's incredibly annoying as you'll get a mixture of file ownerships and permissions. Instead I run it in bubblewrap sandbox: https://blog.gpkb.org/posts/ai-agent-sandbox/ https://blog.gpkb.org/posts/ai-agent-sandbox/
  - jama211 1y ago
    This is a really clever solution
  - athrowaway3z 1y ago
    Set umask from 022 to 002 to give group members the same permissions as a user.
  - raphman 1y ago
    Thanks - seems to work quite well.
- tuyiown 1y ago
dingnuts 1y ago
[flagged]
- simonw 1y ago
  So tell us how to safely run this stuff then. I was under the impression that Docker container escapes are actually very rare. How high do you rate the chance of a prompt injection attack against Claude running in a docker container on macOS managing to break out of that container? (Actually hang on, you called me out for suggesting containers like Docker are safe but that's not what I said - I said "a safe container" - which is a perfectly responsible statement to make: if you know how to run them in a "safe container" you should do so. Firecracker or any container not running on your own hardware would count there.)
  - simonw 1y ago
    I'll also point out that I've been writing about security topics for 22 years. https://simonwillison.net/tags/security/ https://simonwillison.net/tags/security/
  - th0ma5 1y ago
    > So tell us how to safely run this stuff then. That's the secret, cap... you can't. And it's due to in band signalling, something I've mentioned on numerous occasions. People should entertain the idea that we're going to have to reeducated people about what is and isn't possible because the AI world has been playing make believe so much they can't see the fundamental problems to which there is no solution. https://en.m.wikipedia.org/wiki/In-band_signaling https://en.m.wikipedia.org/wiki/In-band_signaling
    - tptacek 1y ago
      Seems pretty glib. Be more specific about what "can't" be done? The preceding argument was about the inadequacy of namespaced shared-kernel containers for workload isolation. But there are lots of ways to isolate workloads.
- dang 1y ago
resters 1y ago
> They're incredibly dangerous to use if you don't know how to isolate them in a safe container but wow the stuff you can do with them is fascinating. True but all it will take is one report of something bad/dangerous actually happening and everyone will suddenly get extremely paranoid and start using correct security practices. Most of the "evidence" of AI misalignment seems more like bad prompt design or misunderstanding of how to use tools correctly.
- igor47 1y ago
  This seems unlikely. We've had decades of horrible security issues, and most people have not gotten paranoid. In fact, after countless data leaks, crypto miner schemes, ransomware, and massive global outages, now people are running LLM bots with the full permission of their user and no guardrails and bragging about it on social media.
bhl 1y ago
Cursor will pivot to a computer use company. The gap between coding agents in your terminal and computer agents that work on your entire operating system is just too narrow and will be crossed over quick.
- teaearlgraycold 1y ago
  Once this tech is eliminating jobs on a massive scale I'll believe the AI hype. Not to say that couldn't be right around the corner - I have no clue. But being able to perform even just data entry tasks with better-than-human accuracy would be a huge deal.
  - baq 1y ago
    That’s the risk - a lot of people suddenly flipping their beliefs at once, especially they’re the same people who are losing the jobs. It’s a civil unrest scenario.
pancakemouse 1y ago
Something I've seen discussed very little is that Claude Code can be opened in a directory tree of any type of document you like (reports, spreadsheets, designs, papers, research, ...) and you can play around in all sorts of ways. Anthropic themselves hint at this by saying their whole organisation uses it, but the `Code` moniker is probably limiting adoption. They could release a generalised agent with a friendlier UI tomorrow and get much wider workplace adoption.
- matlock 1y ago
  Over the weekend I had it extract and Analyse Little but Fierce, a simplified and kid friendly DnD 5e and extract markdown files that help me DMing for my kids. Then it Analyse No, thank you evil as I want to base the setting on it but with LBF rules. And then have the markdown turn into nice looking pdfs. Claude code is so much more than coding and it’s amazing.
- clbrmbr 1y ago
  Indeed. I’m having success using it as a tool for requirements querying. (When a sales person asks “does product A have feature X” I can just ask Claude because I’ve got all the requirements in markdown files.
- tkgally 1y ago
  That’s how I use it. I’m not a developer, and using Claude Code with Git turned out to be more complicated than I wanted. Now I just give it access to a folder on my Mac, put my prompt and any associated files in that folder, and have it work there. It works fine for my needs. I would like a friendlier interface than the terminal, though. It looks like the “Imagine with Claude” experiment they announced today is a step in that direction. I’m sure many other companies are working on similar products.
- withinboredom 1y ago
  I have it master my music. I drop all the stems in a folder, tell it what I want, and off it goes to write a python script specifically for the album. It’s way better than doing it in the DAW, which usually takes me hours (or days in some cases). It can get it to 90% in minutes, only requiring some fine-tuning at the end.
visarga 1y ago
> Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. They still don't have good integration with the web browser, if you are debugging frontend you need to carry screenshots manually, it cannot inspect the DOM, run snippets of code in the console, etc.
- nicewood 1y ago
  True. Although worth mentioning that there is tooling and (e.g. Playwright) MCPs around this. But definitely not integrated well enough!
- simonw 1y ago
  You can tell them to take screenshots using Playwright and they will. They can also use Playwright to inspect the console and manipulate the DOM. I've seen Codex CLI install Playwright Python when I asked it to do this and it found it wasn't yet available in the environment.
- gazpachotron 1y ago
  I'd recommend the chrome-devtools-mcp for that: https://github.com/ChromeDevTools/chrome-devtools-mcp/ https://github.com/ChromeDevTools/chrome-devtools-mcp/ It's pretty new, but so far it's been a lifesaver.
- deleted 1y ago
  [deleted]
polyrand 1y ago
Instead of containers, which may not always be available, I'm experimenting with having control over the shell to whitelist the commands that the LLM can run [0]. Similar to an allow list, but configured outside the terminal agent. Also trying to make it easy to use the same technique in macOS and Linux [0]: https://ricardoanderegg.com/posts/control-shell-permissions-llm-codex/ https://ricardoanderegg.com/posts/control-shell-permissions-...
- khafra 1y ago
  An interesting exercise would be to let a friend into this restricted shell, with a prize for breaking out and running rm -rf / --no-preserve-root. Then you know to switch to something higher-security once LLM capabilities reach the level of that friend.
- philipp-gayret 1y ago
  I really like this and we're doing a similar approach but instead using Claude Code hooks. What's really nice about this style of whitelisting is that you can provide context on what to do instead; Let's say if `terraform apply` is banned, you can tell it why and instruct it to only do `terraform plan`. Has been working amazing for me.
  - polyrand 1y ago
    Me too! I also have a bunch of hooks in claude code for this. But codex doesn't have a hooks feature as polished as claude code (same for their command permissions, it's worse than Claude Code as of today). That's why I explored this "workaround" with bash itself.
- user3939382 1y ago
  You have to put them in the same ACL, chroot, whatever permission context for authorization you’d apply to any other user human or otherwise. For some resources it’s cumbersome to setup but anything else is a hope and a prayer.
- hboon 1y ago
  I didn’t check, but sometimes Claude Code writes scripts and run them (their decision); does your approach guard against that?
ehnto 1y ago
It's broad utility was immediately clear as soon as I saw it formulating bash commands. I've used it to troubleshoot some issues on my linux install, but it's also why the folder sandbox gives me zero confidence that it can't still brick my machine. It will happily run system wide commands like package managers, install and uninstall services, it even deleted my whole .config folder for pulseaudio. Of course I let it do all these things, briefly inspecting each command, but hopefully everyone is aware that there is no real sandbox if you are running claude code in your terminal. It only blocks some of the tool usages it has, but as soon as it's using bash it can do whatever it wants.
Rutledge 1y ago
I call them 'CLI agents'!
monkeydust 1y ago
Been starting to wonder if this marks a step change in UX - moving away from pretty well designed screens where designers labor over positioning of artifacts like buttons, user input dialogs and color palettes to a CLI! I cant imagine CLI will work for everything but for a lot of things, when powered by LLM they are incredible and yea equally dangerous at the same time for many reasons.
budududuroiu 1y ago
I’m experimenting with Nix shells for this tool isolation and whitelisting
- nextaccountic 1y ago
  That's not enough for security. Morally it should be - there's no reason we shouldn't be able to run untrusted software easily - but it won't have a firewall for example Maybe something like bubblewrap could help
ozgung 1y ago
> Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. One criticism on current generation of AI is that they have no real world experience. Well, they have enormous amount of digital world experience. That, actually, has more economical value.
- ACCount37 1y ago
  They have a lot of secondhand knowledge and very little firsthand knowledge. RLVR works so well because it's a way to give LLMs some of the latter.
ACCount37 1y ago
Back in 2022, when ChatGPT was new, quite a few people were saying "LLMs are inherently safe because they can't do anything other than write text". Some must have even believed what they were saying. Clearly not. Just put an LLM into some basic scaffolding and you get an agent. And as capabilities of those AI agents grow, so would the degree of autonomy people tend to give them.
- IMTDb 1y ago
  > LLMs are inherently safe because they can't do anything other than write text That is still very much the case; the danger comes from what you do from the text that is generated. Put a developer in a meeting room and no computer access, no internet etc; and let him scream instructions through the window. If he screams "delete prod DB", what do you do ? If you end up having to restore a backup that's on you, but the dude inherently didn't do anything remotely dangerous. The problem is that the scaffolding people put around LLM is very weak, the equivalent of saying "just do to everything the dude is telling, no question asked, no double check in between, no logging, no backups". There's a reason our industry has development policies, 4 eyes principles, ISO/SOC standards. There already are ways to massively improve the safety of code agents; just put Claude code in a BSD jail and you already have a much safer environment than what 99% of people are doing, this is not that tedious to make. Other safer execution environments (command whitelisting, arguments judging, ...) will be developed soon enough.
  - ACCount37 1y ago
    That's like saying "humans are inherently safe because you can throw them in a jail forever and then there's nothing they can do". But are all humans in jails? No, the practical reason being that it limits their usefulness. Humans like it better when other humans are useful. The same holds for AI agents. The ship has sailed: no one is going to put every single AI agent in jail. The "inherent safety" of LLMs comes only from their limited capabilities. They aren't good enough yet to fail in truly exciting ways.
    - IMTDb 1y ago
saberience 1y ago
Incredibly dangerous to use? Seems like a wild exaggeration. I’ve been using Claude code since launch, must have used it for 1000 hours or more by now, and it’s never done anything I didn’t want it to do. Why would I run it in a sandbox? It writes code for me and occasionally runs a build and tests. I’m not sure why you’re so fixated on the “danger”, when you use these things all the time you end up realizing that the safety aspect is really nowhere near as bad as the “AI doomers” seem to make out.
- geeunits 1y ago
  Because it grabs the headlines and upvotes more. It is becoming quite the bore to read as it offers nothing new, or an accurate representation of the facts. Thanks for calling it out. Same experience regarding thousands of hours of usage since launch, tested from sandboxed docker to take over an entire macbook air and here's an ssh login to a dev server whilst you're at it. I spot check with audits every other day and only wish for more autonomy with the agents, over less.
- guhcampos 1y ago
  It is dangerous. Just yesterday my cursor agent made some changes to a live kubernetes cluster even over my specific instruction not to. I gave it kubectl to analyze and find the issues with a large Prometheud + AlertManager configuration, then switched windows to work on something else. When I was back the MF was patching live resources to try and diagnose the issue.
  - saberience 1y ago
    But this is just like giving a junior engineer access to a prod K8s cluster and having them work for hours on stuff related to said cluster... you wouldn't do it. Or at least, I wouldn't do it. In my own career, when I was a junior, I fucked up a prod database... which is why we generally don't give junior/associate people to much access to critical infra. Junior Engineers aren't "dangerous" but we just don't give them too much access/authority too soon. Claude Code is actually way smarter than a junior engineer in my experience, but I wouldn't give it direct access to a prod database or servers, it's not needed.
brookst 1y ago
Dangerous how? Claude code literally asks before running any command. I suppose they’re dangerous in the same way any terminal shell is dangerous, but it seems a bit of a moral panic. All tools can be dangerous if misused.
- simonw 1y ago
  Many people (myself included) run them in YOLO mode with approvals turned off, because it's massively more productive. And that's despite me understanding how unsafe that is more than most! Even with approvals humans will fall victim to dialog fatigue, where they'll click approve on everything without reading it too closely.
  - kid64 1y ago
    Anthropic's official recommendation is to only use YOLO mode in an airgapped container (see "Safe YOLO mode" here: https://www.anthropic.com/engineering/claude-code-best-practices https://www.anthropic.com/engineering/claude-code-best-pract...)
  - brookst 1y ago
    Well sure, it’s like riding a motorcycle without a helmet: while it is true that motorcycles are dangerous, it’s hardly fair to characterize their danger based on the no-helmet risks.
  - porksoda 1y ago
    That is just nuts! Not in my dreams will claude yolo commands into my system. What are we even talking about? I think life itself grants us the right to get high or pet wild animals or swim the atlantic or sudo rm-rf... Or yes-and-accept-edits at 3AM with a 50 hour uptime (yes guilty) but then we don't get to complain that it's dangerous. We surely were warned.
hmokiguess 1y ago
Obligatory mention: https://xkcd.com/2044/ https://xkcd.com/2044/
pmarreck 1y ago
I too am amazed. Real-world example from last week: After using gpt5-codex inside codex-cli to produce this fork of DOSBox (https://github.com/pmarreck/dosbox-staging-ANSI-server https://github.com/pmarreck/dosbox-staging-ANSI-server) that adds a little telnet server that allows me to screen-scrape VGA textmode data and issue virtual keystrokes (so, full roundtrip scripting, which I ended up needing for a side project to solve a Y2K+25 bug in a DOS app still in production use... yes, these still exist!) via 4000+ lines of C++ (I took exactly one class in C++), and it passes all tests and is non-blocking, I was able to turn around and (within the very same session!) have it help me price it to the client with full justification as well as a history of previous attempts to solve the problem (all of which took my billable time, of course), and since it had the full work history both in Git as well as in its conversation history, it was able to help me generate a killer invoice. So (if all goes well) I may be getting $20k out of this one, thanks to its help. Does the C++ code it made pass the muster of an experienced C++ dev? Probably not (would be happy to accept criticisms, lol, although I think I need to dress up the PR a bit more first), but it does satisfy the conditions of 1) builds, 2) passes all its own tests as well as DOSBox's, 3) is nonblocking (commands to it enter a queue and are processed one set of instructions at a time per tick), 4) works as well as I need it to for the main project. This still leaves it suitable for one-off tasks, of which there is a ton of need for. This is a superpower in the right hands.
simonw 1y ago
I just published a related piece to this idea, on "Designing agentic loops" as a key skill you need to solve problems using these new coding agent tools: https://simonwillison.net/2025/Sep/30/designing-agentic-loops/ https://simonwillison.net/2025/Sep/30/designing-agentic-loop...
redhale 1y ago
Totally agree. The CLI agents (or whatever) lower the barrier of entry to building a custom agent all the way down to basically just writing markdown. Excellent article in this vein: https://jxnl.co/writing/2025/09/04/context-engineering-rapid-agent-prototyping/ https://jxnl.co/writing/2025/09/04/context-engineering-rapid...

by simonw 1y ago

willio58 1y ago

One thing I really like using them for is refactoring/reorganizing. The tedious nature of renaming, renaming all implementations, moving files around, creating/deleting folders, updating imports exports, all melts away when you task an agent with it. Of course this assumes they are good enough to do them with quality, which is like 75% of the time for me so far.

singularity2001 1y ago
does it use the smart refractoring hooks of the IDEs or does it do blunt text replacement
- 0x696C6961 1y ago
  The later
- t0mas88 1y ago
  Currently it's very slow because it does text replace. It would be way faster if it could use the IDE functions via an MCP.
- Yeroc 1y ago
  Blunt text replacement so far. There are third-party VSCode MCP and LSM MCP servers out there that DO expose those higher-level operations. I haven't tried them out myself -- but it's on my list because I expect they'd cut down on token use and improve latency substantially. I expect Anthropic to eventually build that into their IDE integration.
dgunay 1y ago
I've found that it can be hard or expensive for the agent to do "big but simple" refactors in some cases. For example, I recently tasked one with updating all our old APIs to accept a more strongly typed user ID instead of a generic UUID type. No business logic changes, just change the type of the parameter, and in some cases be wary of misleading argument names by lazy devs copy pasting code. This ended up burning through the entire context window of GPT-5-codex and cost the most $ of anything I've ever tasked an agent with.
- felixyz 1y ago
  The way I do this is I task the agent with writing a script which in turn does the updates. I can inspect that script, and I can run it on a subset of files/folders, and I can git revert changes if something went wrong and ask the agent to fix the script or fine-tune it myself. And I don't burn through tokens :) Also, another important factor (as in everything) is to do things in many small steps, instead of giving one big complicated prompt.
golergka 1y ago
Especially when you work with a language where an unfinished refactoring with give you the type error.

athrowaway3z 1y ago

They're only as dangerous as the capabilities you give them. I just created a `codex` and `claude` user on my Linux box and practically always run in yolo mode. I've not had a problem so far. Also, I think shellagent sounds cooler.

simonw 1y ago
That's a great way to run this stuff. I expect the portion of Claude Code users who have a dedicated user setup like this is pretty tiny!
- steveklabnik 1y ago
  One nice thing is that Anthropic provides a sample DevContainer: https://github.com/anthropics/claude-code/tree/main/.devcontainer https://github.com/anthropics/claude-code/tree/main/.devcont... Not the exact setup, but also pretty solid.
globular-toast 1y ago
I tried this but it's incredibly annoying as you'll get a mixture of file ownerships and permissions. Instead I run it in bubblewrap sandbox: https://blog.gpkb.org/posts/ai-agent-sandbox/ https://blog.gpkb.org/posts/ai-agent-sandbox/
- jama211 1y ago
  This is a really clever solution
- athrowaway3z 1y ago
  Set umask from 022 to 002 to give group members the same permissions as a user.
- raphman 1y ago
  Thanks - seems to work quite well.
tuyiown 1y ago

dingnuts 1y ago

[flagged]

simonw 1y ago
So tell us how to safely run this stuff then. I was under the impression that Docker container escapes are actually very rare. How high do you rate the chance of a prompt injection attack against Claude running in a docker container on macOS managing to break out of that container? (Actually hang on, you called me out for suggesting containers like Docker are safe but that's not what I said - I said "a safe container" - which is a perfectly responsible statement to make: if you know how to run them in a "safe container" you should do so. Firecracker or any container not running on your own hardware would count there.)
- simonw 1y ago
  I'll also point out that I've been writing about security topics for 22 years. https://simonwillison.net/tags/security/ https://simonwillison.net/tags/security/
- th0ma5 1y ago
  > So tell us how to safely run this stuff then. That's the secret, cap... you can't. And it's due to in band signalling, something I've mentioned on numerous occasions. People should entertain the idea that we're going to have to reeducated people about what is and isn't possible because the AI world has been playing make believe so much they can't see the fundamental problems to which there is no solution. https://en.m.wikipedia.org/wiki/In-band_signaling https://en.m.wikipedia.org/wiki/In-band_signaling
  - tptacek 1y ago
    Seems pretty glib. Be more specific about what "can't" be done? The preceding argument was about the inadequacy of namespaced shared-kernel containers for workload isolation. But there are lots of ways to isolate workloads.
dang 1y ago

resters 1y ago

> They're incredibly dangerous to use if you don't know how to isolate them in a safe container but wow the stuff you can do with them is fascinating. True but all it will take is one report of something bad/dangerous actually happening and everyone will suddenly get extremely paranoid and start using correct security practices. Most of the "evidence" of AI misalignment seems more like bad prompt design or misunderstanding of how to use tools correctly.

igor47 1y ago
This seems unlikely. We've had decades of horrible security issues, and most people have not gotten paranoid. In fact, after countless data leaks, crypto miner schemes, ransomware, and massive global outages, now people are running LLM bots with the full permission of their user and no guardrails and bragging about it on social media.

bhl 1y ago

Cursor will pivot to a computer use company. The gap between coding agents in your terminal and computer agents that work on your entire operating system is just too narrow and will be crossed over quick.

teaearlgraycold 1y ago
Once this tech is eliminating jobs on a massive scale I'll believe the AI hype. Not to say that couldn't be right around the corner - I have no clue. But being able to perform even just data entry tasks with better-than-human accuracy would be a huge deal.
- baq 1y ago
  That’s the risk - a lot of people suddenly flipping their beliefs at once, especially they’re the same people who are losing the jobs. It’s a civil unrest scenario.

pancakemouse 1y ago

Something I've seen discussed very little is that Claude Code can be opened in a directory tree of any type of document you like (reports, spreadsheets, designs, papers, research, ...) and you can play around in all sorts of ways. Anthropic themselves hint at this by saying their whole organisation uses it, but the `Code` moniker is probably limiting adoption. They could release a generalised agent with a friendlier UI tomorrow and get much wider workplace adoption.

matlock 1y ago
Over the weekend I had it extract and Analyse Little but Fierce, a simplified and kid friendly DnD 5e and extract markdown files that help me DMing for my kids. Then it Analyse No, thank you evil as I want to base the setting on it but with LBF rules. And then have the markdown turn into nice looking pdfs. Claude code is so much more than coding and it’s amazing.
clbrmbr 1y ago
Indeed. I’m having success using it as a tool for requirements querying. (When a sales person asks “does product A have feature X” I can just ask Claude because I’ve got all the requirements in markdown files.
tkgally 1y ago
That’s how I use it. I’m not a developer, and using Claude Code with Git turned out to be more complicated than I wanted. Now I just give it access to a folder on my Mac, put my prompt and any associated files in that folder, and have it work there. It works fine for my needs. I would like a friendlier interface than the terminal, though. It looks like the “Imagine with Claude” experiment they announced today is a step in that direction. I’m sure many other companies are working on similar products.
withinboredom 1y ago
I have it master my music. I drop all the stems in a folder, tell it what I want, and off it goes to write a python script specifically for the album. It’s way better than doing it in the DAW, which usually takes me hours (or days in some cases). It can get it to 90% in minutes, only requiring some fine-tuning at the end.

visarga 1y ago

> Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. They still don't have good integration with the web browser, if you are debugging frontend you need to carry screenshots manually, it cannot inspect the DOM, run snippets of code in the console, etc.

nicewood 1y ago
True. Although worth mentioning that there is tooling and (e.g. Playwright) MCPs around this. But definitely not integrated well enough!
simonw 1y ago
You can tell them to take screenshots using Playwright and they will. They can also use Playwright to inspect the console and manipulate the DOM. I've seen Codex CLI install Playwright Python when I asked it to do this and it found it wasn't yet available in the environment.
gazpachotron 1y ago
I'd recommend the chrome-devtools-mcp for that: https://github.com/ChromeDevTools/chrome-devtools-mcp/ https://github.com/ChromeDevTools/chrome-devtools-mcp/ It's pretty new, but so far it's been a lifesaver.
deleted 1y ago
[deleted]

polyrand 1y ago

Instead of containers, which may not always be available, I'm experimenting with having control over the shell to whitelist the commands that the LLM can run [0]. Similar to an allow list, but configured outside the terminal agent. Also trying to make it easy to use the same technique in macOS and Linux [0]: https://ricardoanderegg.com/posts/control-shell-permissions-llm-codex/ https://ricardoanderegg.com/posts/control-shell-permissions-...

khafra 1y ago
An interesting exercise would be to let a friend into this restricted shell, with a prize for breaking out and running rm -rf / --no-preserve-root. Then you know to switch to something higher-security once LLM capabilities reach the level of that friend.
philipp-gayret 1y ago
I really like this and we're doing a similar approach but instead using Claude Code hooks. What's really nice about this style of whitelisting is that you can provide context on what to do instead; Let's say if `terraform apply` is banned, you can tell it why and instruct it to only do `terraform plan`. Has been working amazing for me.
- polyrand 1y ago
  Me too! I also have a bunch of hooks in claude code for this. But codex doesn't have a hooks feature as polished as claude code (same for their command permissions, it's worse than Claude Code as of today). That's why I explored this "workaround" with bash itself.
user3939382 1y ago
You have to put them in the same ACL, chroot, whatever permission context for authorization you’d apply to any other user human or otherwise. For some resources it’s cumbersome to setup but anything else is a hope and a prayer.
hboon 1y ago
I didn’t check, but sometimes Claude Code writes scripts and run them (their decision); does your approach guard against that?

ehnto 1y ago

It's broad utility was immediately clear as soon as I saw it formulating bash commands. I've used it to troubleshoot some issues on my linux install, but it's also why the folder sandbox gives me zero confidence that it can't still brick my machine. It will happily run system wide commands like package managers, install and uninstall services, it even deleted my whole .config folder for pulseaudio. Of course I let it do all these things, briefly inspecting each command, but hopefully everyone is aware that there is no real sandbox if you are running claude code in your terminal. It only blocks some of the tool usages it has, but as soon as it's using bash it can do whatever it wants.

Rutledge 1y ago

I call them 'CLI agents'!

monkeydust 1y ago

Been starting to wonder if this marks a step change in UX - moving away from pretty well designed screens where designers labor over positioning of artifacts like buttons, user input dialogs and color palettes to a CLI! I cant imagine CLI will work for everything but for a lot of things, when powered by LLM they are incredible and yea equally dangerous at the same time for many reasons.

budududuroiu 1y ago

I’m experimenting with Nix shells for this tool isolation and whitelisting

nextaccountic 1y ago
That's not enough for security. Morally it should be - there's no reason we shouldn't be able to run untrusted software easily - but it won't have a firewall for example Maybe something like bubblewrap could help

ozgung 1y ago

> Claude Code, Codex CLI etc can effectively do anything that a human could do by typing commands into a computer. One criticism on current generation of AI is that they have no real world experience. Well, they have enormous amount of digital world experience. That, actually, has more economical value.

ACCount37 1y ago
They have a lot of secondhand knowledge and very little firsthand knowledge. RLVR works so well because it's a way to give LLMs some of the latter.

ACCount37 1y ago

Back in 2022, when ChatGPT was new, quite a few people were saying "LLMs are inherently safe because they can't do anything other than write text". Some must have even believed what they were saying. Clearly not. Just put an LLM into some basic scaffolding and you get an agent. And as capabilities of those AI agents grow, so would the degree of autonomy people tend to give them.

IMTDb 1y ago
> LLMs are inherently safe because they can't do anything other than write text That is still very much the case; the danger comes from what you do from the text that is generated. Put a developer in a meeting room and no computer access, no internet etc; and let him scream instructions through the window. If he screams "delete prod DB", what do you do ? If you end up having to restore a backup that's on you, but the dude inherently didn't do anything remotely dangerous. The problem is that the scaffolding people put around LLM is very weak, the equivalent of saying "just do to everything the dude is telling, no question asked, no double check in between, no logging, no backups". There's a reason our industry has development policies, 4 eyes principles, ISO/SOC standards. There already are ways to massively improve the safety of code agents; just put Claude code in a BSD jail and you already have a much safer environment than what 99% of people are doing, this is not that tedious to make. Other safer execution environments (command whitelisting, arguments judging, ...) will be developed soon enough.
- ACCount37 1y ago
  That's like saying "humans are inherently safe because you can throw them in a jail forever and then there's nothing they can do". But are all humans in jails? No, the practical reason being that it limits their usefulness. Humans like it better when other humans are useful. The same holds for AI agents. The ship has sailed: no one is going to put every single AI agent in jail. The "inherent safety" of LLMs comes only from their limited capabilities. They aren't good enough yet to fail in truly exciting ways.
  - IMTDb 1y ago

saberience 1y ago

Incredibly dangerous to use? Seems like a wild exaggeration. I’ve been using Claude code since launch, must have used it for 1000 hours or more by now, and it’s never done anything I didn’t want it to do. Why would I run it in a sandbox? It writes code for me and occasionally runs a build and tests. I’m not sure why you’re so fixated on the “danger”, when you use these things all the time you end up realizing that the safety aspect is really nowhere near as bad as the “AI doomers” seem to make out.

geeunits 1y ago
Because it grabs the headlines and upvotes more. It is becoming quite the bore to read as it offers nothing new, or an accurate representation of the facts. Thanks for calling it out. Same experience regarding thousands of hours of usage since launch, tested from sandboxed docker to take over an entire macbook air and here's an ssh login to a dev server whilst you're at it. I spot check with audits every other day and only wish for more autonomy with the agents, over less.
guhcampos 1y ago
It is dangerous. Just yesterday my cursor agent made some changes to a live kubernetes cluster even over my specific instruction not to. I gave it kubectl to analyze and find the issues with a large Prometheud + AlertManager configuration, then switched windows to work on something else. When I was back the MF was patching live resources to try and diagnose the issue.
- saberience 1y ago
  But this is just like giving a junior engineer access to a prod K8s cluster and having them work for hours on stuff related to said cluster... you wouldn't do it. Or at least, I wouldn't do it. In my own career, when I was a junior, I fucked up a prod database... which is why we generally don't give junior/associate people to much access to critical infra. Junior Engineers aren't "dangerous" but we just don't give them too much access/authority too soon. Claude Code is actually way smarter than a junior engineer in my experience, but I wouldn't give it direct access to a prod database or servers, it's not needed.

brookst 1y ago

Dangerous how? Claude code literally asks before running any command. I suppose they’re dangerous in the same way any terminal shell is dangerous, but it seems a bit of a moral panic. All tools can be dangerous if misused.

simonw 1y ago
Many people (myself included) run them in YOLO mode with approvals turned off, because it's massively more productive. And that's despite me understanding how unsafe that is more than most! Even with approvals humans will fall victim to dialog fatigue, where they'll click approve on everything without reading it too closely.
- kid64 1y ago
  Anthropic's official recommendation is to only use YOLO mode in an airgapped container (see "Safe YOLO mode" here: https://www.anthropic.com/engineering/claude-code-best-practices https://www.anthropic.com/engineering/claude-code-best-pract...)
- brookst 1y ago
  Well sure, it’s like riding a motorcycle without a helmet: while it is true that motorcycles are dangerous, it’s hardly fair to characterize their danger based on the no-helmet risks.
- porksoda 1y ago
  That is just nuts! Not in my dreams will claude yolo commands into my system. What are we even talking about? I think life itself grants us the right to get high or pet wild animals or swim the atlantic or sudo rm-rf... Or yes-and-accept-edits at 3AM with a 50 hour uptime (yes guilty) but then we don't get to complain that it's dangerous. We surely were warned.

hmokiguess 1y ago

Obligatory mention: https://xkcd.com/2044/ https://xkcd.com/2044/

pmarreck 1y ago

I too am amazed. Real-world example from last week: After using gpt5-codex inside codex-cli to produce this fork of DOSBox (https://github.com/pmarreck/dosbox-staging-ANSI-server https://github.com/pmarreck/dosbox-staging-ANSI-server) that adds a little telnet server that allows me to screen-scrape VGA textmode data and issue virtual keystrokes (so, full roundtrip scripting, which I ended up needing for a side project to solve a Y2K+25 bug in a DOS app still in production use... yes, these still exist!) via 4000+ lines of C++ (I took exactly one class in C++), and it passes all tests and is non-blocking, I was able to turn around and (within the very same session!) have it help me price it to the client with full justification as well as a history of previous attempts to solve the problem (all of which took my billable time, of course), and since it had the full work history both in Git as well as in its conversation history, it was able to help me generate a killer invoice. So (if all goes well) I may be getting $20k out of this one, thanks to its help. Does the C++ code it made pass the muster of an experienced C++ dev? Probably not (would be happy to accept criticisms, lol, although I think I need to dress up the PR a bit more first), but it does satisfy the conditions of 1) builds, 2) passes all its own tests as well as DOSBox's, 3) is nonblocking (commands to it enter a queue and are processed one set of instructions at a time per tick), 4) works as well as I need it to for the main project. This still leaves it suitable for one-off tasks, of which there is a ton of need for. This is a superpower in the right hands.

simonw 1y ago

I just published a related piece to this idea, on "Designing agentic loops" as a key skill you need to solve problems using these new coding agent tools: https://simonwillison.net/2025/Sep/30/designing-agentic-loops/ https://simonwillison.net/2025/Sep/30/designing-agentic-loop...

redhale 1y ago

Totally agree. The CLI agents (or whatever) lower the barrier of entry to building a custom agent all the way down to basically just writing markdown. Excellent article in this vein: https://jxnl.co/writing/2025/09/04/context-engineering-rapid-agent-prototyping/ https://jxnl.co/writing/2025/09/04/context-engineering-rapid...