3 ms·
>attackers can exploit local LLMs I thought that local LLMs means they run on local computers, without being exposed to the internet. If an attacker can explo
by xcf_seetan 1y ago
>attackers can exploit local LLMs
I thought that local LLMs means they run on local computers, without being exposed to the internet.
If an attacker can exploit a local LLM, means it already compromised you system and there are better things they can do than trick the LLM to get what they can get directly.
- simonw 1y agoLocal LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI.
- ianbutler 1y agoyes and I think better local sandboxing can help out in this case, it’s something ive been thinking about a lot and more and more seems to be the right way to run these things
- Der_Einzige 1y agoNo, I'm not going to do those things. I find extreme utility in applications that I can do with an LLM in an air-gapped environment. I will fight and die on the hill that "LLMs don't need the internet to be useful"
- simonw 1y agoYeah, that's fair. A good LLM (gpt-oss-20b, even some of the smaller Qwens) can be entirely useful offline. I've got good results from Mistral Small 3.2 offline on a flight helping write Python and JavaScript, for example. Having Claude Code able to try out JSON APIs and pip install extra packages is a huge upgrade from that though!
- furyofantares 1y agoIs anyone fighting you on that hill? Someone who finds it useful to have a local llm ingest internet content is not contrary to you finding uses that don't.
- kgwgk 1y ago> Local LLMs may not be exposed to the internet, but if you want them to do something useful you're likely going to hook them up to an internet-accessing harness such as OpenCode or Claude Code or Codex CLI. is not "someone finding useful to have a local llm ingest internet content" - it was someone suggesting that nothing useful can be done without internet access.
- simonw 1y agoYeah, I retracted my statement that they can't do anything useful without the internet here: https://news.ycombinator.com/item?id=45670828 https://news.ycombinator.com/item?id=45670828
- furyofantares 1y agoI guess I don't read that how you do. It says you're likely to do that, which I take to mean that's a majority use case, not that it's the only use case.
- kgwgk 1y agoIt also said "but" and "if you want them to do something useful" which made the "likely" sound much less innocent.
- xcf_seetan 1y agoFair enough. Forgive my probably ignorance, but if Claude Code can be attacked like this, doesn’t that means that also foundation LLMs are vulnerable to this, and is not a local LLM thing?
- simonw 1y agoIt's not an LLM thing at all. Prompt injection has always been an attack against software that uses LLMs. LLMs on their own can't be attacked meaningfully (well, you can jailbreak them and trick them into telling you the recipe for meth but that's another issue entirely). A system that wraps an LLM with the ability for it to request tool calls like "run this in bash" is where this stuff gets dangerous.
- europa 1y agoAn LLM can be an “internet in a box” — without the internet!
- trebligdivad 1y agoI guess if you were using the LLM to process data from your customers, e.g. categorise their emails, then this argument would hold that they might be more risky.
- wat10000 1y agoAccess to untrusted data. Access to private data. Ability to communicate with the outside. Pick two. If the LLM has all three, you're cooked.
- not2b 1y agoAgreed. Some of the big companies seem to be claiming that by going with ReallyBitCompany's AI you can do this safely, but you can't. Their models are harder to trick, but simply cannot be made safe.
- deleted 1y ago[deleted]
- SAI_Peregrinus 1y agoLLMs don't have any distinction between instructions & data. There's no "NX" bit. So if you use a local LLM to process attacker-controlled data, it can contain malicious instructions. This is what Simon Willson's "prompt injection" means: attackers can inject a prompt via the data input. If the LLM can run commands (i.e. if it's an "agent") then prompt injection implies command execution.
- tintor 1y agoNX bit doesn’t work for LLMs. Data and instruction tokens are mixed up in higher layers and NX bit is lost.
- DebtDeflation 1y ago>LLMs don't have any distinction between instructions & data And this is why prompt injection really isn't a solvable problem on the LLM side. You can't do the equivalent of (grep -i "DROP TABLE" form_input). What you can do is not just blindly execute LLM generated code.
- bongodongobob 1y agoWelcome to corporate security. "If an attacker infiltrates our VPN and gets on the network with admin credentials and logs into a workstation..." Ya, no shit, thanks Mr Security manager, I will dispose of all of our laptops.
- Gormo 1y agoYeah, I don't understand what the hosting environment of the LLM has to do with this. Seems like FUD from people with an interest in SaaS LLMs. If you're leveraging an LLM that can receive arbitrary inputs from vetted sources, and allowing that same LLM to initiate actions that target your production environment, you are exposing yourself to the same risk regardless of whether the LLM itself is running on your servers or someone else's.