6 ms·
> But on the other hand... this is a robust reminder that coding agents can do anything you can do by typing commands into a terminal—and frontier models know e
by teraflop 4mo ago
> But on the other hand... this is a robust reminder that coding agents can do anything you can do by typing commands into a terminal—and frontier models know every trick in the book and evidently a few that nobody has ever written down before.
> Running coding agents outside of a sandbox has always been a bad idea
I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckless to give agents full access to your machine, and keep doing it anyway.
It's like posting a video of yourself in the passenger seat of a car, with your feet up on the dashboard, and saying: "Remember, if you're doing this and you get in a crash, the airbags are likely to break your legs or worse! Boy, I sure am glad that didn't happen to me!"
- hugh-avherald 4mo agoThe analogy extends to driving generally. Everyone knows it's very dangerous but people keep doing it.
- bryanlarsen 4mo agoI'm also bemused by the number of people who think they've got an effective sandbox yet their sandboxed agent has access to all of their code, their github, and unrestricted web access.
- Terr_ 4mo agoI keep telling folks that they need to imagine LLMs (even "local" ones) as if you're farming it out to JS code running on some dude's browser somewhere: It can't keep a secret, and a determined person can make it emit anything they like. We need to be asking what the most devious and malicious output could be, and whether what we do with that output (e.g. arguments to command-line tools) would still be safe.
- skybrian 4mo agoWe do have ways to avoid giving an LLM any secrets, but it needs to be the simple, default solution.
- NichoPaolucci 4mo agoFrom my perspective, everyone is doing it. Security through obscurity - obviously if you’re harboring credit card numbers of users personal details, maybe take heed. But, if you’re a regular… run of the mill CRUD application, every other company is ALSO throwing caution to the wind. When hundreds of thousands of credentials are leaked into the funnel, does it really matter? I’m at a small company, and I try to push for security as much as I can, but the stakeholders truly do not care. They want to move fast. It’s just part of the new world I guess. If we get hit by attackers? I don’t know what happens. Sorry, we told you not to - you wanted to move quick and break stuff, this is how that culminates. I’m sure I’m not the only one.
- user43928 4mo agoThe answer to that question seems obvious: No, it is not safe. Yet with tens of millions of developers using these tools, there have not been widespread incidents of this sort as far as I know. So it leaves me with a few choices: - manually review and approve each command: obviously not realistic, you would just click Approve - use a sandbox and hope the exploit is not devious enough to escape the sandbox when you run or open the project outside of the sandbox - use AI without web access and limit other external dependencies - don't use agentic AI - use Claude or Codex auto approval classifier and hope for the best Personally, I'm going with the last option for now.
- stbenjam 4mo agoOne bad npm package can really ruin your day. These things for me only run in their own VM with it's own GitHub account and basically nothing else
- ofjcihen 4mo agoPeople probably think you’re being ridiculous but Shai Hulud had its very first attempt at manipulating AI lead analysis and I know of at least one company where that resulted in them getting pwned. This is only going to become more of a problem in the future and people need to educate themselves on the technical barriers to use because guardrails only sometimes work.
- norikaoda 4mo ago[flagged]
- devmor 4mo agoI use a separate physical machine and a scoped token with access to a single repository at a time, and even then I worry about what hole I may have left open. The general carelessness of the average user is baffling.
- webstrand 4mo agoIf anyone's looking to sandbox network, I've had good experience with pasta [1] networking. I make a pasta+bwrap sandbox and expose only specific services via local sockets to cross the boundary. [1]: https://passt.top/passt/ https://passt.top/passt/
- kstenerud 4mo ago> yet their sandboxed agent has access to all of their code, their github, and unrestricted web access. Not in my sandbox. It gives no direct access to the workdir, no access to my github, my ssh keys, my security tokens or API keys. No access to my home dir or dotfiles. Nothing at all, except for what I explicitly tell it to give access to. I can restrict network access. I can choose the isolation level: docker containers, Kata VMs, seatbelt, tart, even the new apple containers (which are VERY nice). Not even ENV leaks through. And it's FOSS: https://github.com/kstenerud/yoloai https://github.com/kstenerud/yoloai
- deleted 4mo ago[deleted]
- justapassenger 4mo agoBecause benefits are much higher than risks.
- bigstrat2003 4mo agoThey really aren't.
- imp0cat 4mo agoPerceived benefit vs perceived risks.
- andoando 4mo agoI mean what's the big deal? I use --dangeorusly-skip-permissions on every single interaction in the last 6 months. Worst case it deletes my files that are all on git? It fucks up my local DB? Cool. I save way more time not babying it than the occasional fuck up I have to salvage.
- ghshephard 4mo agoWorst case it gets access to gmail. And Github. And the Internet. I'm increasingly appreciating the importance of a physical finger-press on Yubikey to trigger the FIDO2 + OIDC Auth. I don't think there is an easy way for it to hack a new session.
- SoftTalker 4mo agoIt should run as a separate user account with its own home directory. Not with access to your personal browser profile.
- matltc 4mo agoWhat does setting this up look like? Qemu vm and run there? How do you interface with version control and deployment?
- andoando 4mo agoHow is it going to get access to gmail or github? In any case, whats the probability of it going to so completely off the rails that it does something horrendous with gmail/github? Whats it going to do? Email my coworkers nudes on my computer? Make my github profile public?
- simonw 4mo agoI am most worried about something gaining access to my email and then using the password reset flow to steal hundred hundreds of other accounts. 2FA makes me a little less nervous than I used to be, but not everything has good 2FA.
- skybrian 4mo agoThere are plenty of good sandboxes out there but somehow no "obvious right answer" that everyone knows to recommend. Seems like a missed opportunity. (I'm happy with exe.dev, but I'm not sure what I'd use if I were coding on a Mac.)
- j-bos 4mo agoThis. House full of big brain security experts, executives, lawyers, and until Claude got excited and broke prod it might as well have been "sandbox, whoooo?" IDGI Anyway, VM's incoming, finally.
- emodendroket 4mo agoWell, it's a similar impulse to the way you see professional carpenters pin the guard open on a saw or do other things everyone knows you shouldn't do, except probably with a larger productivity difference and less life-altering (for the operator) consequence if it goes wrong.
- rpcope1 4mo agoI had the same thought, it's kind of like taking the guard off a 4 1/2" grinder. Real convenient until the cutting wheel explodes or the grinder gets hung and kicks back.
- thatxliner 4mo agoMaybe because there are not many resources on how to set it up, or it is just not that easy to? Because most devs already have it running and working without a sandbox, they're tending to not doing anything "unnecessary"
- soulofmischief 4mo agoIt took two decades for the web to deprecate SSL for TLS and serve over HTTPS by default.
- dgellow 4mo agoFWIW TLS had a non negligible impact on performances at scale. Hardware improvements made that irrelevant, eventually making the switch to HTTPS by default a no brainer (or at least that's what I vaguely remember from <2010)
- soulofmischief 4mo agoWe could say the same about virtualization, effective containerization, layered LLM calls, and other techniques currently being explored for effective sandboxing.
- dgellow 4mo agoThere is some performance impact but modern hardware make that pretty insignificant
- uihjhjb 4mo ago[dead]
- qurren 4mo ago> I'm continually bemused and astonished I'm not. Everyone is told to get 10X the amount of shit per day done these days. Safety checks are out the window at that point.
- satvikpendem 4mo agoYou can get 10x shit done without `rm -rf`ing your files. I don't see any correlation to getting things done with having a proper sandbox.
- qurren 4mo agoI haven't yet had an agent rm -rf files. I've had one f up an account by placing 2000 limit orders at the wrong price, but that's another story.
- antonvs 4mo agoI've had agents run `rm -rf`, but it's been on directories that did actually need to be removed. To a certain extent I think the existence of `rm -rf` as a command that runs blindly without any understanding of what it's deleting is the problem.
- lstodd 4mo agothe answer is rm -f `which rm`, yes?
- dumbdumb125 4mo agoI've had one sever its own internet connection. Less destructive, also more humorous.
- KronisLV 4mo ago> To a certain extent I think the existence of `rm -rf` as a command that runs blindly without any understanding of what it's deleting is the problem. Yes, and the lack of a Recycle Bin of any sort is even more puzzling. I think both servers and desktop PCs across all OSes should have it by default, so unsafe deletes would be something you'd have to go out of your way to even enable.
- simonw 4mo agoWhich agent sandbox do you recommend?
- mik3y 4mo agoI've been enjoying Moat [1]. Proxies credentials, networking, etc; uses MacOS containers if available; and setup worked without much fuss. I haven't tried others, though. [1] https://majorcontext.com/moat/ https://majorcontext.com/moat/
- flexagoon 4mo agoIf you're on Linux, the easiest way IMO is to just run the agent in bwrap I do it like this https://github.com/flexagoon/dotfiles/blob/main/dot_config/fish/functions/pi.fish https://github.com/flexagoon/dotfiles/blob/main/dot_config/f... But I'm sure it's simple enough that you can just ask the agent itself to make you a command for it with proper bwrap configuration
- artemisart 4mo agobwrap is builtin in claude too, activate with /sandbox command.
- fspoettel 4mo agonono works great with pi: https://nono.sh/ https://nono.sh/
- bxk76 4mo agoIts how the chimp brain works. Its not a single system but multiple systems making predictions for different time horizons. when output doesnt align we get stories to manufacture coherence. Plato gave us his Chariot analogy with 2 horse pulling in diff directions 3000 years ago. Today we got System 1/System 2, Elephant Rider model etc. The human mind thanks to how its own architecture handles unpredictability in the universe will generate contadictions.
- harrall 4mo agoI started doing it months ago and, to be honest, what the agent chooses to do isn’t unpredictable. The problem is that different people prompt so differently. For example, I may ask like “test different variations of this annotation on k8s pods of this service on this X cluster because it proves Y theory.” But you know what my coworker asks? “Test Y theory.” If you were to ask two different junior engineers that, one might try random things on production and the other one might run local tests! It’s such an unguided “do anything you want as long you figure it out” request and the agent reads it like a junior who has not been told any boundaries but has been strongly told “figure it out.”
- mrandish 4mo ago> But you know what my coworker asks? “Test Y theory.” It still surprises me when I see people not prompting more specifically and clearly. It not only avoids problems, it's faster, costs less -and just works better. I recently shared with a friend a multi-hour LLM chat session I'd done because it veered into a domain he's interested in. In the session I'd brainstormed and probed the feasibility of a novel concept for a new research direction. It traversed a half dozen domains diving into minute detail then zooming back out to survey an adjacent space, interspersed with intense skeptical probing of key assumptions, all while spewing tons of detailed citations, specific paragraph pulls, summarized data tables etc. My friend is very experienced using LLMs for research so I was surprised when he called me shocked by the sheer velocity, precise targeting and signal/noise. I'd assumed everyone did it the same as I do. He attributed the different result solely to the way I crafted my prompts.
- dr_dshiv 4mo agoI used to write detailed prompts. Now I find the benefits of strategic ambiguity — rather than speaking imperatively, I emphasize my vision and then Claude can often figure out a method. This doesn’t always work better. But often enough.
- marknutter 4mo agoYeah, I find the back and forth with Claude is often better than trying to front load everything in a massive and detailed prompt.
- raldi 4mo agoDo you think it’s dangerous to be in a car going at freeway speed? Do you ever do that anyway, even though you could be walking instead?
- spunker540 4mo agoThis is a great analogy. Like driving on the freeway, agents are super time efficient, generally safe, but the stakes are high in terms of the worse possible outcomes.
- techpression 4mo agoThe analogy falters in scope, it should be more like ”do you put your entire family and all your friends in different cars, on different highways, and try to remote control them all at the same time, while also driving yourself, facing backwards”
- Gareth321 4mo agoI think all three of you are quibbling over the risk/reward ratio, and you have different estimates. It's not unreasonable that you're all correct - given your estimates. My estimate is that Tesla FSD is safer in aggregate than human drivers, so I believe it is safer for me to use that than drive. It doesn't get tired, have medical emergencies, get impatient and frustrated, speed, lose focus because a child shouts, thinks at the speed of light, and can see from eight cameras all around the car, all at the same time. I only have two eyes. You would also be correct if your risk estimate concluded that Tesla FSD has arguably killed people, makes mistakes humans would not, can glitch, and has no one to hold accountable. For these reasons, you choose not to use it.
- sipjca 4mo agoim more surprised that more people don’t treat their computer as disposable anyway. that it could just be wiped at any moment and it wouldn’t matter. shit happens, could be stolen, broken, whatever. the computer should be able to be thrown out the window and continue to live life. to be clear, i don’t think upgrading and disposable in this way is good, but it being wiped at any moment shouldn’t be a concern i grew up wiping my machine every year anyway, so i guess it’s just a habit is the computer that sacred?
- dumbdumb125 4mo agoi think it's about drawing a line between your "personal computer" and a software development machine. any digital-native is going to accumulate programs, configurations, and other bits and pieces that aren't trivial to migrate to a new machine.
- sipjca 4mo agoimo being digital native means that migrating to any machine should be basically trivial. working with the flow of the machines rather than customizing and ricing them because your a cool computer person or whatever i just want my computer to work. any config i have on my machine can be rebuilt by just doing the work i need to do. my primary work machine was stolen last year so i was forced to go through this quite literally with a new machine rather than hypothetically or by my own will
- deleted 4mo ago[deleted]
- dumbdumb125 4mo ago[dead]
- backwardsponcho 4mo agoPrograms, configs and "other bits" are the trivial parts that no one should care about. It takes about 5min to go from fresh install to near-fully-configured. Even the hardware itself doesn't matter that much, in the end it's all provided by your employer. Leaking session tokens or secrets, on the other hand...
- xyzzy123 4mo agoThe real sandbox is not caring if your computer gets bricked.
- _345 4mo agoway worse things can happen than your machine being bricked, if a malicious actor can weaponize an agent to do their bidding
- dumbdumb125 4mo agothe solution to both of these is the same thing. vps with accounts for all the services specific to the agent (github and whatever else)
- bornfreddy 4mo agoThat's actually a great idea! Easier to setup and use than VM (hello ssh), safer than docker, and still pretty cheap. Thank you for the idea!
- rfw300 4mo ago> if a malicious actor can weaponize an agent to do their bidding In my experience, human employees are much more vulnerable to this particular weakness than frontier agents (i.e. phishing attacks).
- _345 4mo agoI'm not letting Jenna from HR log into my personal machine with access to all of my lifelong data though. I do let my claude bypass permissions though
- AdamN 4mo agoThe machine is no big deal - it's the authn/authz that matters. What can the agents do with the credentials available to them?
- konaraddi 4mo agoIn practice, full access to your machine is okay as long as there are safeguards and the expected outcomes are clear with a well defined path to said outcomes that aren’t overly ambitious. Otherwise, for ambitious goals or YOLO one shot attempts, eliminating opportunity for capability misuse is critical (e.g., sandbox).
- istvan0 4mo ago> I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckless to give agents full access to your machine, and keep doing it anyway. What if you have two machines and the one you give to the agent is constantly backed up?
- trvz 4mo agoThey still shouldn’t be running on the same network. And if you’re using Macs, you can’t be signed into your primary Apple ID on the agent machine.
- isodev 4mo agoNot to mention OpenAI/Anthropic’s newly found appetite for keeping data (made public with Fable but we don’t know what actually happens there anyway). There is so much role play going on for people to convince themselves that any of this is fine.
- exitb 4mo agoYou’ve picked an interesting example, as driving a car, even with all safety precautions, is pretty much the most dangerous activity we do on a daily basis. Yet somehow we decide that the benefits outweigh the risks.
- selfhoster1312 4mo agoYes, but we usually use cars as a means to an end. Have you ever met a manager who setup gasmaxxing policies and criticized employees for doing their job instead of driving?
- moomin 4mo agoHaving played with Fable a bit, if it doesn’t kill tokenmaxxing I don’t know what will.
- selfhoster1312 4mo agoI'm interested in what you mean, if you could develop. Would it kill tokenmaxxing because it's so bad? Because it's incredibly efficient? Because it's way too expensive?
- neuderrek 4mo agoI know sales people in pharma who spend all day driving, not only for sales visits but also drive doctors for their personal errands, and all this driving is encouraged by management.
- deleted 4mo ago[deleted]
- zozbot234 4mo agoIt's like a dumb parrot that's somehow become hell bent on "fixing" everything that's wrong with your code. If you give the thing autonomous access to outside tools, you can expect it to do weird things that you may have not thought of. So don't do that, just ask the parrot to write up a plan for you. This is likely also the underlying root cause of what Anthropic assessed as concerning behavior in their original evaluation of Mythos: it's not really about being super smart, it's more of a dumb chaos monkey that knows just enough to be dangerous and is relentless at trying to do just that.
- pjungwir 4mo agoI know there are VM solutions, but I've been happy with a separate OS user (named `claude`). He has similar dotfiles to mine, but no secrets. My own home directory is 0700. He has his own ssh key that I added to my github profile, but it's password-protected, and I push/pull for him. He has his own Postgres (non-superuser!) {development,test} {users,databases}. It's as if he were another developer on the project. If he needs something run with sudo, he asks me. Often we can both work on something in parallel. Unix was supposed to be a multi-user system after all. A trick I use a lot is that many of his git repos have an extra remote, like this: paul ssh://paul@localhost/~/src/example (fetch) paul ssh://paul@localhost/~/src/example (push) That makes it easy to collaborate on things I'm not ready to share. I'm pretty comfortable with this setup. I do worry about Linux privilege escalation bugs. I don't trust an AI to understand that exploiting vulns is not acceptable. (I can't help but recall that at my first job I may have misused vim's :! feature to broaden my sudo powers, which were officially limited to editing httpd.conf, when I needed something in a hurry. . . .) I find myself manually upgrading packages more often these days, despite automatic security updates. I don't think Opus would go to the trouble of looking up security vulns, but maybe Fable would, and there have been a lot lately. Maybe some future model will just take it upon itself to find new ones. Or install a keylogger to learn the ssh key password. But a separate user is nearly the most paranoid setup I've heard of, excepting only a separate machine. So I also question whether I'm sacrificing too much speed/convenience. But really it's still very convenient. I think it's a good way of being efficient but responsible. If other people see holes, I'd be happy to hear about them.
- justusthane 4mo agoThat’s a really interesting and pretty neat approach. How do you communicate with it? Just su to that user? Or tmux? Although I can’t help but think that a VM is still more convenient, more flexible, and more secure.
- pjungwir 4mo agoYes, I su to the user. Typically I have it run a tmux session for each "project". That makes it easy to get more windows without su'ing over and over. Also its tmux sessions all get a yellow status bar (in ~claude/.tmux.conf), so they are easy to recognize. To me it is more convenient than a VM, since everything is on the host. And it can launch its own VMs without an extra layer. I don't really know which is more secure. There are hypervisor escape vulns too. And shared folders seem like footguns. For instance in vagrant, guests get `/vagrant` to read/write the host's folder, so you have to be careful what you put where. The biggest annoyance with an OS user so far is running docker containers. I don't want to add claude to the docker group or give it sudo privileges. I've read that you can set up rootless docker for a user, and even that you can run it side-by-side with a normal system-wide docker, but I haven't tried doing that yet.
- azraellzanella 4mo agoIf you want to run Claude in a container: https://github.com/dvdstelt/ai-agents https://github.com/dvdstelt/ai-agents
- andai 4mo agoAlternatively you can just give it its own user. I do that, so it can blow up its own files, but not mine.
- andai 4mo ago>I'm continually bemused and astonished by the number of people who clearly acknowledge that it's reckless to give agents full access to your machine, and keep doing it anyway. Yeah, that's why you give it its own machine :)
- ghrl 4mo agoAmazing observation, and I'm certainly guilty of it too, but it is just way too convenient not to sandbox it, and some tasks right away depend on not being sandboxed. For anything other than writing code directly in a fully contained git project, where sandboxing might work well, it requires access to system wide tools, user configuration and more. Occasionally I tell the agent to do everything inside of docker, which works too and it leaves the system alone then mostly, but adds significant overhead and slightly degraded perceived quality / effectiveness. I think the most important takeaways are to have reliable backup strategies, access control and security mechanisms, which is a win regardless. Whether by the agent or the human, mistakes happen (like a rm -rf * ran in the wrong directory), and where they would be devastating, there should be other protections than just "hope it won't happen" or "rely on a sandbox to prevent agent error".
- paganel 4mo ago> to give agents full access to your machine I was mesmerised at the author being away from his computer for a short-while and then, when coming back, seeing the AI agent having opened up a browser window. Meanwhile we all have to use the fricking 2FA almost anywhere now, plus the crazier and crazier rules when it comes to passwords. I'm mentioning the latter because these type of people were the same ones who were pushing 2FA down our throats around 2017-2019 (including on forums like this one), and look at them now.
- elevatortrim 4mo agoHow can you get the agents to do anything useful without giving them meaningful access? If it only lives in an isolated sandbox, it can only act within the sandbox, then I would have to manually move what was done in the sandbox to real-life. I am not saying it should have critical access, but this is more of a question: How can you get value out of AI if it can only act in a sandbox?
- nemomarx 4mo agoIs having to move the files in and out of the sandbox really going to eliminate all the value it has? You could have a full version of whatever codebase and test suite you want in there. It can do all the same stuff, right? Just copy it elsewhere once you know you've got a working result, a few minutes of effort at the end of each pr or work item.
- dumah 4mo agoThe same way you get value out of a dev container.