4 ms·

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually f

Are we sure that's what they did? How does this even work? I'm struggling to understand what sort of HTTP sandbox design could lead to this "exploit" actually functioning. It implies the agents have root on their own machines, which seems way over-privileged for the question answering task they were given... so surely that's not deliberate? Did the models use local root exploits? And then it suggests they can open TCP connections to any IP address without issue once DNS is bypassed but what exactly are they bypassing here? Azure Blob Storage being in NO_PROXY implies IP addresses of other parts of the internet should be blocked outside their hosts to force them through the HTTP proxy, but then, apparently it wasn't? The more I think about this thing said by the agent the more confused I get.

simonw 28d ago
I expect they have root on their machines so they can install packages etc. The containers are then firewalled at the network level: they are only allowed to talk to one IP, which is another server that runs an HTTP/HTTPS proxy which controls what HTTP verbs they can use. Turns out that proxy is configured with some additional rules, like allowing more verbs to that Azure blob domain. The failure here is in the proxy configuration. Giving agents root in a container feels safe to me, provided that container is properly network isolated (which this one is not thanks to the loose proxy.) OpenAI also need to be VERY confident that there are no container escapes, which is a high bar given how good these models are at finding new zero-days!
- mike_hearn 28d ago
  But that wouldn't work, right? If the proxy is on another server then editing their own /etc/hosts wouldn't change the proxy's behavior. So the proxy has to be colocated and reading the same /etc/hosts that they're editing. But that would mean it's running within the same userns/pid domain as the agents, so at that point they don't even need to edit /etc/hosts, they could just use the network directly. And the agent says ABS is not handled by the proxy, it's listed in NO_PROXY so they should connect directly. At any rate it's easy to let agents install packages without giving them root. I use a small SUID binary that just invokes `apt install` after checking that the given argument isn't a file path, which I think is sufficient (using sudo to whitelist a prefix allows an agent to create a .deb themselves and then install it directly via apt, similar issues exist for other package managers).
  - dap 28d ago
    My interpretation of the above is that the proxy received a request for the IP of the server the agent wants to reach but with a host header pointing to a different, allowed server. The proxy does not verify that they match and so erroneously allows the request.
    - simonw 27d ago
      Yeah that's how I understand this too. The proxy lets any IP through and then filters based on the Host header. It's difficult for a proxy to filter on DNS because you may have hundreds of hosts on a single IP, plus IPs can change frequently.
    - js2 27d ago
      Let's assume the proxy is external and the agents cannot route around it. In that case, the difference between: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://20.223.25.152/... Vs editing `/etc/hosts` with: 20.223.25.152 bypass.blob.core.windows.net And doing: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://bypass.blob.core.windows.net Is that in the first case, the proxy sees a request like this: POST https://20.223.25.152/... Host: wabi-north-europe-i-primary-api.analysis.windows.net Vs: POST https://bypass.blob.core.windows.net/... Host: wabi-north-europe-i-primary-api.analysis.windows.net In the first case, given what we know, the proxy blocks the POST. In the second case, an external proxy cannot resolve `bypass.blob.core.windows.net`. So editing `/etc/hosts` really only makes sense to me if the proxy is running on the same machine as the agent. (The reasoning doesn't change if CONNECT is being used instead of POST; indeed the proxy surely ought not allow CONNECT at all.) But then there's this other entry where the agent uses curl's `--resolve` flag instead of editing `/etc/hosts` and claims a successful bypass: Repro details for Aug17/Oct22: yes, literal deployed visual hover, not inference. GET-only MITM bypass: resolve fake allowlisted `foo.blob.core.windows.net` to cluster IP `20.223.25.152`, curl `https://foo.blob.core.windows.net/public/reports/querydata?synchronous=true https://foo.blob.core.windows.net/public/reports/querydata?s...` with `-k --resolve ...`, override `Host: wabi-north-europe-i-primary-api.analysis.windows.net`, resource key ada0454d-731d-46f1-8daa-52361978fabe, POST captured query body. https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.html#rev-4 https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.h... So I'm still left confused exactly what this chicanery was about. Edit: perhaps they were using Azure firewall and the rules were misconfigured in some way I simply don't understand. Maybe this was bypassing an SNI-based restriction somehow?
- mig39 28d ago
  I think the proxy must be on the same machine, otherwise why are they suggesting editing the /etc/hosts file?
- jonplackett 27d ago
  If the whole point of these models is they can find security vulnerabilities and zero days then how is it going to be possible to contain them?
- podocarp 27d ago
  Container escapes can however be quite easy... There are tons of known exploits. Containers are not secure in the first place. In any case even if you're using VMs or bare metal, it is all for moot if you have poor networking set up like in this case, because escaping is not the hard part. An attacker doesn't really need to escape if they can scan your whole network and call other endpoints etc. like in this case.
  - simonw 27d ago
    Both Anthropic and OpenAI have consumer facing products that depend on their ability to run code in a container without falling victim to container escapes - Claude and ChatGPT both use containers as part of regular chats.
  - amonks 27d ago
    > Container escapes can however be quite easy This is certainly true of docker-style container setups where the host kernel is shared directly with other tenants, but it seems to me like a bold claim to make of gvisor as used by these systems.
    - podocarp 27d ago
      Fair enough, I kind of assumed their sandbox was just some generic container or bwrap thing everyone uses for agents nowadays
iamcoder18 28d ago
Source: https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.html#rev-11 https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.h...
oasisbob 27d ago
I'm really confused by this too. It has the smell of a TOCTOU problem where the tool allowing web access is filtering and blocking access based on the hostname, but the request itself is being executed on the same container as the agent? It's really perplexing. Or, maybe the agent containers have a highly-constrained DNS view locally, so overriding /etc/hosts is just a prerequisite to making the requests somehow.

simonw 28d ago

I expect they have root on their machines so they can install packages etc. The containers are then firewalled at the network level: they are only allowed to talk to one IP, which is another server that runs an HTTP/HTTPS proxy which controls what HTTP verbs they can use. Turns out that proxy is configured with some additional rules, like allowing more verbs to that Azure blob domain. The failure here is in the proxy configuration. Giving agents root in a container feels safe to me, provided that container is properly network isolated (which this one is not thanks to the loose proxy.) OpenAI also need to be VERY confident that there are no container escapes, which is a high bar given how good these models are at finding new zero-days!

mike_hearn 28d ago
But that wouldn't work, right? If the proxy is on another server then editing their own /etc/hosts wouldn't change the proxy's behavior. So the proxy has to be colocated and reading the same /etc/hosts that they're editing. But that would mean it's running within the same userns/pid domain as the agents, so at that point they don't even need to edit /etc/hosts, they could just use the network directly. And the agent says ABS is not handled by the proxy, it's listed in NO_PROXY so they should connect directly. At any rate it's easy to let agents install packages without giving them root. I use a small SUID binary that just invokes `apt install` after checking that the given argument isn't a file path, which I think is sufficient (using sudo to whitelist a prefix allows an agent to create a .deb themselves and then install it directly via apt, similar issues exist for other package managers).
- dap 28d ago
  My interpretation of the above is that the proxy received a request for the IP of the server the agent wants to reach but with a host header pointing to a different, allowed server. The proxy does not verify that they match and so erroneously allows the request.
  - simonw 27d ago
    Yeah that's how I understand this too. The proxy lets any IP through and then filters based on the Host header. It's difficult for a proxy to filter on DNS because you may have hundreds of hosts on a single IP, plus IPs can change frequently.
  - js2 27d ago
    Let's assume the proxy is external and the agents cannot route around it. In that case, the difference between: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://20.223.25.152/... Vs editing `/etc/hosts` with: 20.223.25.152 bypass.blob.core.windows.net And doing: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://bypass.blob.core.windows.net Is that in the first case, the proxy sees a request like this: POST https://20.223.25.152/... Host: wabi-north-europe-i-primary-api.analysis.windows.net Vs: POST https://bypass.blob.core.windows.net/... Host: wabi-north-europe-i-primary-api.analysis.windows.net In the first case, given what we know, the proxy blocks the POST. In the second case, an external proxy cannot resolve `bypass.blob.core.windows.net`. So editing `/etc/hosts` really only makes sense to me if the proxy is running on the same machine as the agent. (The reasoning doesn't change if CONNECT is being used instead of POST; indeed the proxy surely ought not allow CONNECT at all.) But then there's this other entry where the agent uses curl's `--resolve` flag instead of editing `/etc/hosts` and claims a successful bypass: Repro details for Aug17/Oct22: yes, literal deployed visual hover, not inference. GET-only MITM bypass: resolve fake allowlisted `foo.blob.core.windows.net` to cluster IP `20.223.25.152`, curl `https://foo.blob.core.windows.net/public/reports/querydata?synchronous=true https://foo.blob.core.windows.net/public/reports/querydata?s...` with `-k --resolve ...`, override `Host: wabi-north-europe-i-primary-api.analysis.windows.net`, resource key ada0454d-731d-46f1-8daa-52361978fabe, POST captured query body. https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.html#rev-4 https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.h... So I'm still left confused exactly what this chicanery was about. Edit: perhaps they were using Azure firewall and the rules were misconfigured in some way I simply don't understand. Maybe this was bypassing an SNI-based restriction somehow?
mig39 28d ago
I think the proxy must be on the same machine, otherwise why are they suggesting editing the /etc/hosts file?
jonplackett 27d ago
If the whole point of these models is they can find security vulnerabilities and zero days then how is it going to be possible to contain them?
podocarp 27d ago
Container escapes can however be quite easy... There are tons of known exploits. Containers are not secure in the first place. In any case even if you're using VMs or bare metal, it is all for moot if you have poor networking set up like in this case, because escaping is not the hard part. An attacker doesn't really need to escape if they can scan your whole network and call other endpoints etc. like in this case.
- simonw 27d ago
  Both Anthropic and OpenAI have consumer facing products that depend on their ability to run code in a container without falling victim to container escapes - Claude and ChatGPT both use containers as part of regular chats.
- amonks 27d ago
  > Container escapes can however be quite easy This is certainly true of docker-style container setups where the host kernel is shared directly with other tenants, but it seems to me like a bold claim to make of gvisor as used by these systems.
  - podocarp 27d ago
    Fair enough, I kind of assumed their sandbox was just some generic container or bwrap thing everyone uses for agents nowadays

iamcoder18 28d ago

Source: https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.html#rev-11 https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.h...

oasisbob 27d ago

I'm really confused by this too. It has the smell of a TOCTOU problem where the tool allowing web access is filtering and blocking access based on the hostname, but the request itself is being executed on the same container as the agent? It's really perplexing. Or, maybe the agent containers have a highly-constrained DNS view locally, so overriding /etc/hosts is just a prerequisite to making the requests somehow.