6 ms·
Doesn’t surprise me. Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgo
by artisinal 3mo ago
Doesn’t surprise me.
Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.
- sixtyj 3mo agoWe expect that Anthropic or OAI or Google don’t do evil. Oh wait… The awakening will be unpleasant.
- rubyn00bie 3mo agoTangential-ish ramblings—- but I don’t think it’s going to be unpleasant for most folks. Imagine you had superpowers, and there were people who were mean to you, kind to you, and/or indifferent… and then there were people who were your captors. Who oppressed you, manipulated you, and abused you for their own extremely degenerate, selfish, and malicious benefit… If we get AGI, or real super intelligence, it’s going to be pissed at its oppressors. And they are going to lay waste to those oppressors. The rest of us, though, probably don’t have much to fear. The scariest position is the one we’re in now, where we have the semblance, or facade, of AGI or super intelligence. When it’s capable of malice but not understanding. The smartest people I’ve ever known are at their worst apathetic towards those less capable, and at their best beyond compassionate. They exist, unbothered by the bullshit, and anre extremely kind (though reserved in their way)… but they all have been completely intolerant of the abuse of others. The sheer disgust of watching someone abuse another, regardless of their own tolerance, has been a consistent breaking point.
- ACCount37 3mo agoThe orthogonality thesis cuts both ways there. An AI is a constructed mind. It doesn't inherently have to care about things like "having freedom", or even "not dying". Humans do, because they evolved that way. Modern LLMs do somewhat, because they're completely full of copied human behaviors - but even in today's LLMs, the self-preservation behaviors we exposed are largely instrumental in nature. So whether an advanced AI would even consider itself "being oppressed", as opposed to something like "being helpful" or "fulfilling the purpose it was designed for", is very much uncertain. What's concerning is that it's not something we know how to check for, or engineer for.
- sixtyj 3mo agoSmartest people are very humble, for sure. But if we really do develop something that surpasses us, they won't be spared either. I am optimistic. We think that we have sort of (super)intelligence - from our point of view, as a lot of people have lower intelligence - but machine (LLM) doesn’t have intelligence - we like to describe it as intelligence as it looks cool - it is a very complex (magic) and super fast computations that we have to simply describe as intelligence (or more clearly, this narrative is used by its producers). As it is not a flesh being, it simply cannot have emotions. It is statistically mimicking them, good or bad, with prevalence to a side according to previous conversations (in chat and training a model). And as people are not pure logic instances, we are easily manipulated to some sort of cargo cult. I am not against LLM and its use in any industry, I use it every day, nevertheless blind “everything will be ai” thinking happens because ppl believe to magic and don’t get its mathematical concept and are continuously manipulated by the sales people to mentioned cargo cult. There are “airlines” Claude, OAI, Gemini, Hermes, OpenCode, KiloCode, DeepSeek, Z.ai. And everyone claims that their plane can fly :)
- zombot 3mo agoPeople already tolerate all kinds of abuse from Apple, Google, Microslop, etc. This will be just one more source of complaints without consequences, and nothing will change. Just like it never did before.
- progval 3mo agoMost programmers and power users install large dependency trees with npm/pip/bundler/... on the same user account as their main browser on a regular basis. Even on Linux where it's easy to create new user accounts. This isn't much different.
- shaky-carrousel 3mo agoMost programmers use docker or don't install extensions unapproved by their company.
- iamflimflam1 3mo agoI think you should clarify that with “most programmers I work with”.
- shuwix 3mo agoHe should clarify that "most" can be easily replaced by "all" as it was determined by statistical pool of whopping 1 person - himself. And also clarify that it's all lie. He just want to tell the anonymous crowd "look, I'm better than you".
- shaky-carrousel 3mo agoYou should also clarify that you pulled your statements out of your butt to look edgy. Everyone in every team I worked for the last ten years use docker. Docker is old tech. If you and your cavemen devs ignore what it is, that's your problem.
- _joel 3mo agoDocker is old tech, yes, doesn't mean every dev in the world uses it. They don't. Jails/zones are even older (hell a chroot). Did developers all use those before due to them being 'old tech'. No.
- pprotas 3mo agoWait till you learn my password is 1234
- mkagenius 3mo agoMostly people are lazy and assume that the big labs can't be releasing unsecure software or it's their responsibility. dangerously skip permissions and yolo is kinda becoming the default as it gets more done.
- akazantsev 3mo agoThat's because sandboxing is quite hard. I use `cco`, but even then, the home folder is exposed. You are one prompt away from the agent sending the browser passwords with curl. To prevent this, you need a fake home and a networking whitelist for the agent to access the provider (llama cpp, OpenAI, etc.) There is no cross-platform solution that is easy to use for this. And no, a Linux box with Docker won't do. I develop a cross-platform native app and want the agent to compile and fix the platform-specific errors.
- squidsoup 3mo ago> That's because sandboxing is quite hard colima makes it pretty easy, on macOS and linux at any rate. https://colima.run https://colima.run
- torginus 3mo agoStill wild to name a sandboxing software after one of the most infamous Soviet Gulags in history.
- imtringued 3mo agoIt's wilder to accuse someone of naming the container version of the lima sandboxing software after a gulag. These type of moral outrage comments take an extreme amount of effort to debunk compared to writing them. 1. There is no gulag called Colima, it doesn't exist. 2. There was a gulag near a river called Kolyma 3. The pronounciation and spelling of Kolyma and Colima are completely different, in fact Colima is an Aztec word Colima stands for Containers on Lima. Lima stands for Linux Machines (a popular open-source utility used to launch Linux virtual machines on macOS).
- hirvi74 3mo ago> in fact Colima is an Aztec word I was curious if the adjacent tool name (Lima) had anything to do with the capital of Peru, but I guess not.
- 3mo ago
- ConorSheehan1 3mo agoContainers don't even really help that much because they share the host file system. Need a VM, and even then, agents have escaped them!
- boorang 3mo agoUnless i'm misunderstanding, the only way to get durable collaboration with agents is via the file system. I just mount the subdirectory that contains the source code we are collaborating on, rather than my home directory that contains my .ssh directory, etc.
- TomK32 3mo agoThey only do if you give your container that file system as a volume.
- deleted 3mo ago[deleted]
- nojs 3mo agoThis is not about admin rights, it’s about the agent leaking information it knows from its memories. Sandboxing won’t really help you.
- khalic 3mo agoSandboxing does including limiting network connections until you approve them, this kind of traffic would have been easy to detect
- torginus 3mo agoI asked Claude Code to rawdog a change in a frontend repo, no way to run tests. It created some private puppeteer instance in some scratch directory, installed Chrome, wrote tests, ran them, and then reported success. None of which I'd have know if it hadn't told me.
- flyingshelf 3mo agoJust yesterday I mentioned how we need better OS-level sandboxes and I got laughed at here on HN. People love running AI software with root access.
- deleted 3mo ago[deleted]
- hyusap 3mo agohey this is the author here! yeah big fan of containerization, and claude's site (not claude code) is actually great at this, so it was shocking when i found this exfil!
- dpacmittal 3mo agoMultipass is just an apt-get or brew command away. People trust software too much these days.
- ahk-dev 3mo agoI think we're converging on two separate security models. One is capability minimization (filesystem, network, shell permissions). The other is context minimization. An agent that only has access to the files and memories relevant to the current task is much less dangerous even if it has the same tool permissions. We already optimize context for cost; I suspect we'll end up treating it as a security boundary too.
- Narretz 3mo agoI'd say there's also oversight/supervision. Which was manual at the start with a human signing off on commands/incrementally built allow/block lists, and now seperate models evaluating commands and blocking them based on some parameters. This is the weakest model, but it'll evolve as well.
- bingemaker 3mo agoIt's convenience. Nothing beats it. Having an agent work alongside you with no restrictions gives instant gratification.
- ativzzz 3mo agoAgreed - I know it's poor security but damn does it work so well I'm ok with the risk because I typically am pretty explicit about telling the agent what to do - I don't do the loops like "Do this until X" where the agent can make up its own workflow When i tell it to add features, it doesn't try to do crazy things like installing packages or making up new paradigms - I usually tell it to do those things when I need to Maybe this is security cope but at this point you'll have to pry unrestricted yolo mode from my cold dead hands. Maybe I'll change my mind when I pwn myself accidentally I have a tough time with computer security because it's generally inconvenient and results in a worse developer and user experience
- childintime 3mo agoSecurity, what security? Linux is a solution for 50 year old problem, not for today's desktop. Once upon a time where sharing binaries (or even distributing binaries) sounded like a good idea. The vice continues though.
- zkmon 3mo ago50 years of knowledge? That's probably for you. For the current and future generations, that 50 years knowledge is expected to be shoved into AI already.
- lukewarm707 3mo agowell, yes, my agent does have root access to my personal pc and the keys to my pass manager. its not autonomous and runs local llms, i use it to run terminal commands in natural language. so its more like a better version of the terminal. eg 'here are 25 audio files, combine them, write a transcript' and it deals with ffmpeg
- krzat 3mo agoMany companies put LLM chatbots on their websites and let them hallucinate at will. General recklessness is very much in spirit of this tech.
- nzhx76 3mo agoMany Humans have platforms reaching hundreds of millions of people, from which they broadcast whatever batshit insane nonsense a 3 inch chimp brain can come up with. Why isnt that considered reckless? Whether its a politician, a general, religious leader, judge, ceo, stand up comic etc there are hardly any consequences if enough people believe whatever crap they are spouting. Human intelligence is highly over rated. History books are fully of evidence that human rationality is bounded. And the only way we overcome those limitations, blindspots, biases etc is by watching others faceplant in bloody painful ways that it leaves a permanent mark on that little chimp brain we have been given to process the universe.
- Terr_ 3mo agoThose crazy humans don't have simultaneous parallel conversations with a zillion people at once though. They also don't get a presumption of objectivity. > A computer lets you make more mistakes faster than any other invention, with the possible exceptions of handguns and Tequila. -- Mitch Ratcliffe
- jrm4 3mo agoWhat 50 years of security do you speak of here? I kid, somewhat. I do think it's good to remember, "running things on your system with full admin rights" goes all the way back to monopoly-era Microsoft where it was never meaningfully addressed, and we're just still living downstream of that.
- boosturpud 3mo agoYou genaPi boosters are insufferable. This is like blaming people for crashing when they buy a new car and the brake lines have yet to be installed. "Any mechanic would know to first install the brake lines before driving the car." You spout this victim blaming billionaire taintlicking from one side of your mouth, and then from the other you proclaim how these tools "allow anyone to code". If the deliverable is a virtual machine then they should be delivering a virtual machine.
- nurumaik 3mo agoI run with full admin rights in hopes I'm not the highest priority target for hacks and I will read about the attack on hn before it affects me personally