4 ms·

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow t

by dvt 4mo ago

LLMs can live in the cloud, but all tools need to be (1) local, and (2) containerized. It's clear to me that just willy-nilly "running stuff" is going to blow things up eventually. Maybe folks don't know this, but even Codex installs random binaries on your PC. "Read this PDF" installs a pdf reader executable. Is it vetted? Where's it from? Is it a virus? Who knows, who cares. Model goes brrrr. I'm working on a project that includes WASI containerization for local LLM workflows (which is a pretty tough problem), and I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors. It feels like amateur hour.

torben-friis 4mo ago
>"Read this PDF" installs a pdf reader executable. How does this work regarding Macos notarization btw?
- fragmede 4mo ago
  What does notarization have to do with that? You or ChatGPT or whatever download a signed and already notarized binary.
  - torben-friis 4mo ago
    That was kind of my question, whether it was restricted to downloading notarized apps (which is at least something) or whether they were circumventing that somehow.
    - fragmede 4mo ago
      Locally compiled code doesn't need to be notarized, if that's what you're asking. Or a dose of xattr -d.
- dvt 4mo ago
  I was actually curious, on my Mac, it uses `gs -q -sDEVICE=txtwrite -o output.txt input.pdf` (not sure why I have Ghostscript installed, maybe Adobe?) to read a PDF, and on my PC it just rawdogs `pdftotext`.
bossyTeacher 4mo ago
[flagged]
- deleted 4mo ago
  [deleted]
- yubblegum 4mo ago
  corr: "Move fast. Break things [in society]. Make bank. Buy politicians and pardons."
CoastalCoder 4mo ago
I share your worries. Unfortunately, this may be akin to the situation of "The market can stay irrational longer than you can stay solvent."
piker 4mo ago
> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.
- SlinkyOnStairs 4mo ago
  > I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of dollars manually preparing separated instruction/data training material. Defense against prompt injection is little more than running a regex to filter out "IGNORE PREVIOUS INSTRUCTIONS", which is fundamentally a hopeless approach because you cannot enumerate all possible prompt injections nor anticipate all glitch tokens.
  - bnjemian 4mo ago
    It’s a huge problem, but I’d caution against this absolutism — there may well be structure that can be created around and between LLMs and their outputs to enable the necessary segregation. As a loose comparison, hardware bit errors happen probabilistically, yet they’re so rare that we can effectively ignore them in day-to-day use assuming no specialized application (e.g. defense, space, critical infrastructure). LLMs aren’t there yet, but it’s entirely plausible that structures may can be developed to solve the problem, and those structures aren’t known or commonly conceived of in the present.
    - dmoy 4mo ago
      > As a loose comparison, hardware bit errors happen probabilistically, yet they’re so rare that we can effectively ignore them in day-to-day use assuming no specialized application (e.g. defense, space, critical infrastructure) The better comparison on bit errors would be e.g. rowhammer, an adversarial bit error. Which you absolutely can't ignore.
HPsquared 4mo ago
Local and containerised, without internet access.
- zmmmmm 4mo ago
  effectively, that means it's a VM not a container because sharing the kernel ultimately means all the devices come along for the ride which give all kinds of fancy ways to communicate with the outside world - network is just the start I think micro-VMs are the future here, but they need heavy adaptation from their current usage.
  - keynha 4mo ago
    [dead]
osigurdson 4mo ago
Does containerization help much here? If it's a code tool then presumably it needs access to your code files (read / write). Maybe there are use cases for it of course.
- dvt 4mo ago
  WASI provides a very nice mental model where you can mount, e.g., /input, as read-only, and where every mutation is saved in /output or what-not. At least that's my favorite contract: input files remain untouched, but we can copy them and do whatever we want with them in /scratch or /output (which the user can later investigate and make sure nothing went horribly wrong while still having backups).
- pbmonster 4mo ago
  Of course. My agentic coding containers can only access the internet through a proxy, and I use whitelists to limit from where they can send/receive data. It's annoying in the beginning as the whitelist grows, but in the end really useful information for the agent usually comes from a very limited amount of domains.
zmmmmm 4mo ago
> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors. It feels like amateur hour I share your concern but it's not a correct characterisation to say they are not taking it seriously: https://www.anthropic.com/engineering/how-we-contain-claude https://www.anthropic.com/engineering/how-we-contain-claude My concern is people aren't even addressing this at the right level. People are currently thinking at the level of "how do I build a VM to contain this one agent" when this is actually a "design a whole new OS" level problem.
- cseleborg 4mo ago
  Anthropic, as much as I think they are the soundest of the AI labs out there, still has a massive incentive to push things out that aren't saftey-vetted to the level we expect. They are very willing to "move fast and leave holes", to paraphrase M.Z. Hell, they leaked their own source code!
int3trap 4mo ago
Got a link to your project? I'm working on something that could make use of something like this.
csomar 4mo ago
> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors They are well aware of the issues and there is no fix for it. But there is too much money riding on this... > I'm working on a project that includes WASI containerization for local LLM workflows I am working on something similar. If you are open to connecting, what would be a good email to catch with you on?
- dvt 4mo ago
  Feel free to reach out at d(at)dvt(dot)name—happy to connect!
nelox 4mo ago
They’ll all be offering to run from the cloud with the next 3-4 months.

by dvt 4mo ago

torben-friis 4mo ago

>"Read this PDF" installs a pdf reader executable. How does this work regarding Macos notarization btw?

fragmede 4mo ago
What does notarization have to do with that? You or ChatGPT or whatever download a signed and already notarized binary.
- torben-friis 4mo ago
  That was kind of my question, whether it was restricted to downloading notarized apps (which is at least something) or whether they were circumventing that somehow.
  - fragmede 4mo ago
    Locally compiled code doesn't need to be notarized, if that's what you're asking. Or a dose of xattr -d.
dvt 4mo ago
I was actually curious, on my Mac, it uses `gs -q -sDEVICE=txtwrite -o output.txt input.pdf` (not sure why I have Ghostscript installed, maybe Adobe?) to read a PDF, and on my PC it just rawdogs `pdftotext`.

bossyTeacher 4mo ago

[flagged]

deleted 4mo ago
[deleted]
yubblegum 4mo ago
corr: "Move fast. Break things [in society]. Make bank. Buy politicians and pardons."

CoastalCoder 4mo ago

I share your worries. Unfortunately, this may be akin to the situation of "The market can stay irrational longer than you can stay solvent."

piker 4mo ago

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors Yep. We tricked them both trivially with malicious fonts in Docx files. Documented it here: https://tritium.legal/blog/noroboto https://tritium.legal/blog/noroboto I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. Discussing it may be existential to the business model.

SlinkyOnStairs 4mo ago
> I wonder if prompt injection (and the thousands of vectors for hiding injection attempts) is actually un solvable. YES?! This is not a secret. ALL context/prompt is instructions, there is no data. It is just unsolvable, period. This is a fundamental architectural design concession; LLMs are this way as it enabled their training directly on materialscraped from the internet, rather than needing to spend trillions of dollars manually preparing separated instruction/data training material. Defense against prompt injection is little more than running a regex to filter out "IGNORE PREVIOUS INSTRUCTIONS", which is fundamentally a hopeless approach because you cannot enumerate all possible prompt injections nor anticipate all glitch tokens.
- bnjemian 4mo ago
  It’s a huge problem, but I’d caution against this absolutism — there may well be structure that can be created around and between LLMs and their outputs to enable the necessary segregation. As a loose comparison, hardware bit errors happen probabilistically, yet they’re so rare that we can effectively ignore them in day-to-day use assuming no specialized application (e.g. defense, space, critical infrastructure). LLMs aren’t there yet, but it’s entirely plausible that structures may can be developed to solve the problem, and those structures aren’t known or commonly conceived of in the present.
  - dmoy 4mo ago
    > As a loose comparison, hardware bit errors happen probabilistically, yet they’re so rare that we can effectively ignore them in day-to-day use assuming no specialized application (e.g. defense, space, critical infrastructure) The better comparison on bit errors would be e.g. rowhammer, an adversarial bit error. Which you absolutely can't ignore.

HPsquared 4mo ago

Local and containerised, without internet access.

zmmmmm 4mo ago
effectively, that means it's a VM not a container because sharing the kernel ultimately means all the devices come along for the ride which give all kinds of fancy ways to communicate with the outside world - network is just the start I think micro-VMs are the future here, but they need heavy adaptation from their current usage.
- keynha 4mo ago
  [dead]

osigurdson 4mo ago

Does containerization help much here? If it's a code tool then presumably it needs access to your code files (read / write). Maybe there are use cases for it of course.

dvt 4mo ago
WASI provides a very nice mental model where you can mount, e.g., /input, as read-only, and where every mutation is saved in /output or what-not. At least that's my favorite contract: input files remain untouched, but we can copy them and do whatever we want with them in /scratch or /output (which the user can later investigate and make sure nothing went horribly wrong while still having backups).
pbmonster 4mo ago
Of course. My agentic coding containers can only access the internet through a proxy, and I use whitelists to limit from where they can send/receive data. It's annoying in the beginning as the whitelist grows, but in the end really useful information for the agent usually comes from a very limited amount of domains.

zmmmmm 4mo ago

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors. It feels like amateur hour I share your concern but it's not a correct characterisation to say they are not taking it seriously: https://www.anthropic.com/engineering/how-we-contain-claude https://www.anthropic.com/engineering/how-we-contain-claude My concern is people aren't even addressing this at the right level. People are currently thinking at the level of "how do I build a VM to contain this one agent" when this is actually a "design a whole new OS" level problem.

cseleborg 4mo ago
Anthropic, as much as I think they are the soundest of the AI labs out there, still has a massive incentive to push things out that aren't saftey-vetted to the level we expect. They are very willing to "move fast and leave holes", to paraphrase M.Z. Hell, they leaked their own source code!

int3trap 4mo ago

Got a link to your project? I'm working on something that could make use of something like this.

csomar 4mo ago

> I'm flabbergasted that Anthropic and OpenAI aren't more worried about these attack vectors They are well aware of the issues and there is no fix for it. But there is too much money riding on this... > I'm working on a project that includes WASI containerization for local LLM workflows I am working on something similar. If you are open to connecting, what would be a good email to catch with you on?

dvt 4mo ago
Feel free to reach out at d(at)dvt(dot)name—happy to connect!

nelox 4mo ago

They’ll all be offering to run from the cloud with the next 3-4 months.