4 ms·

You need it if your agent can access the internet and read files you don’t want public. That’s a relatively minimal case.

by IanCal 5d ago

debazel 5d ago
It is really easy to restrict and contain an AI agent as long as you don't give it access to a terminal. If you only give it tools to read files and access the internet, then it is much easier to just restrict the tools themselves rather than setting up a whole isolated sandbox.
- hosteur 5d ago
  Access to read sensitive files and access to internet could easily lead to data exposures on the internet, no? Without any terminal or shell access.
  - debazel 5d ago
    Yes, but what I'm saying is that it is much easier to put a limit on the read_file tool or the http tool rather than sandboxing your whole environment. You only really need to sandbox when you provide access to tools that are almost impossible to filter correctly, such as a bash tool or a tool for arbitrary code execution.
    - zerd 5d ago
      Limiting it to just read and http limits its usefulness. If you want something like “filter for this, count the number of matches, format like this” you have to make custom tools. And you have to make sure they don’t have any bugs that allow arbitrary code execution. You’re effectively building your own sandbox in this case. Using a standard one sounds easier unless you have a very focused use case.
TeMPOraL 5d ago
Do you actually need it, or do you just fear you need it? What's the actual realistic threat model for median developer or median user here? By realistic, I mean that leaking your grandma's recipes or your SSN or your million dollar idea to some pastebin is neither likely nor going to meaningfully make things worse for you, or be useful for any malicious actor. Surely this is not what everyone is worried about?
- kstenerud 5d ago
  I spent today doing forensics on ten compromised WordPress sites sharing one hosting account. I used two agents: One with network access to collect the evidence, and one with everything except the model endpoint cut off, which did the analysis. The second agent's entire input was attacker-authored. So PHP droppers, obfuscated loaders, database rows, filenames, blah blah. In this case I'm more worried about hostile input attacking the agent, and I need to contain the damage. My sandboxing solution does that by restricting access to the source data, making it read-only. The work dir can only transfer data via patch and apply (like a git workflow), so even my workspace can't be modified until I approve each change. And then restricted network means that any compromise ain't going noplace. The second agent couldn't even install PHP or contact any CVE site to check if it was looking at a known attack, and that was by design. All it could do is write up a report about what it observed, not make assumptions about what it is. I could then take its (much smaller) clean output and pass that to a third agent with network access. This is forensic work, so of course not your median dev's bread & butter. But the attack surface is only just starting to be plumbed. Compromising input can turn your agent into their agent, planting things as easily as planting worms was back in the early internet days when people connected without a firewall.
- threatofrain 5d ago
  Security by obscurity is just a bet on weights, a belief that the economic motivations for attacking are insufficient. That worked before, but developments in ML calls to account all the debt we’ve accumulated through that practice.

You need it if your agent can access the internet and read files you don’t want public. That’s a relatively minimal case.

by IanCal 5d ago

You need it if your agent can access the internet and read files you don’t want public. That’s a relatively minimal case.

debazel 5d ago

It is really easy to restrict and contain an AI agent as long as you don't give it access to a terminal. If you only give it tools to read files and access the internet, then it is much easier to just restrict the tools themselves rather than setting up a whole isolated sandbox.

hosteur 5d ago
Access to read sensitive files and access to internet could easily lead to data exposures on the internet, no? Without any terminal or shell access.
- debazel 5d ago
  Yes, but what I'm saying is that it is much easier to put a limit on the read_file tool or the http tool rather than sandboxing your whole environment. You only really need to sandbox when you provide access to tools that are almost impossible to filter correctly, such as a bash tool or a tool for arbitrary code execution.
  - zerd 5d ago
    Limiting it to just read and http limits its usefulness. If you want something like “filter for this, count the number of matches, format like this” you have to make custom tools. And you have to make sure they don’t have any bugs that allow arbitrary code execution. You’re effectively building your own sandbox in this case. Using a standard one sounds easier unless you have a very focused use case.

TeMPOraL 5d ago

Do you actually need it, or do you just fear you need it? What's the actual realistic threat model for median developer or median user here? By realistic, I mean that leaking your grandma's recipes or your SSN or your million dollar idea to some pastebin is neither likely nor going to meaningfully make things worse for you, or be useful for any malicious actor. Surely this is not what everyone is worried about?

kstenerud 5d ago
I spent today doing forensics on ten compromised WordPress sites sharing one hosting account. I used two agents: One with network access to collect the evidence, and one with everything except the model endpoint cut off, which did the analysis. The second agent's entire input was attacker-authored. So PHP droppers, obfuscated loaders, database rows, filenames, blah blah. In this case I'm more worried about hostile input attacking the agent, and I need to contain the damage. My sandboxing solution does that by restricting access to the source data, making it read-only. The work dir can only transfer data via patch and apply (like a git workflow), so even my workspace can't be modified until I approve each change. And then restricted network means that any compromise ain't going noplace. The second agent couldn't even install PHP or contact any CVE site to check if it was looking at a known attack, and that was by design. All it could do is write up a report about what it observed, not make assumptions about what it is. I could then take its (much smaller) clean output and pass that to a third agent with network access. This is forensic work, so of course not your median dev's bread & butter. But the attack surface is only just starting to be plumbed. Compromising input can turn your agent into their agent, planting things as easily as planting worms was back in the early internet days when people connected without a firewall.
threatofrain 5d ago
Security by obscurity is just a bet on weights, a belief that the economic motivations for attacking are insufficient. That worked before, but developments in ML calls to account all the debt we’ve accumulated through that practice.