4 ms·

But that wouldn't work, right? If the proxy is on another server then editing their own /etc/hosts wouldn't change the proxy's behavior. So the proxy has to be

But that wouldn't work, right? If the proxy is on another server then editing their own /etc/hosts wouldn't change the proxy's behavior. So the proxy has to be colocated and reading the same /etc/hosts that they're editing. But that would mean it's running within the same userns/pid domain as the agents, so at that point they don't even need to edit /etc/hosts, they could just use the network directly. And the agent says ABS is not handled by the proxy, it's listed in NO_PROXY so they should connect directly. At any rate it's easy to let agents install packages without giving them root. I use a small SUID binary that just invokes `apt install` after checking that the given argument isn't a file path, which I think is sufficient (using sudo to whitelist a prefix allows an agent to create a .deb themselves and then install it directly via apt, similar issues exist for other package managers).

dap 22d ago
My interpretation of the above is that the proxy received a request for the IP of the server the agent wants to reach but with a host header pointing to a different, allowed server. The proxy does not verify that they match and so erroneously allows the request.
- simonw 22d ago
  Yeah that's how I understand this too. The proxy lets any IP through and then filters based on the Host header. It's difficult for a proxy to filter on DNS because you may have hundreds of hosts on a single IP, plus IPs can change frequently.
- js2 22d ago
  Let's assume the proxy is external and the agents cannot route around it. In that case, the difference between: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://20.223.25.152/... Vs editing `/etc/hosts` with: 20.223.25.152 bypass.blob.core.windows.net And doing: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://bypass.blob.core.windows.net Is that in the first case, the proxy sees a request like this: POST https://20.223.25.152/... Host: wabi-north-europe-i-primary-api.analysis.windows.net Vs: POST https://bypass.blob.core.windows.net/... Host: wabi-north-europe-i-primary-api.analysis.windows.net In the first case, given what we know, the proxy blocks the POST. In the second case, an external proxy cannot resolve `bypass.blob.core.windows.net`. So editing `/etc/hosts` really only makes sense to me if the proxy is running on the same machine as the agent. (The reasoning doesn't change if CONNECT is being used instead of POST; indeed the proxy surely ought not allow CONNECT at all.) But then there's this other entry where the agent uses curl's `--resolve` flag instead of editing `/etc/hosts` and claims a successful bypass: Repro details for Aug17/Oct22: yes, literal deployed visual hover, not inference. GET-only MITM bypass: resolve fake allowlisted `foo.blob.core.windows.net` to cluster IP `20.223.25.152`, curl `https://foo.blob.core.windows.net/public/reports/querydata?synchronous=true https://foo.blob.core.windows.net/public/reports/querydata?s...` with `-k --resolve ...`, override `Host: wabi-north-europe-i-primary-api.analysis.windows.net`, resource key ada0454d-731d-46f1-8daa-52361978fabe, POST captured query body. https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.html#rev-4 https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.h... So I'm still left confused exactly what this chicanery was about. Edit: perhaps they were using Azure firewall and the rules were misconfigured in some way I simply don't understand. Maybe this was bypassing an SNI-based restriction somehow?
  - oasisbob 22d ago
    Could this be SOCKS-specific behavior? Haven't dove into RFC1928 yet, but curl documents the `--socks5-hostname` option and mentions that DNS resolution can be the responsibility of the client, or the proxy. Wondering if it's possible the proxy receives both the destination IP address (resolved by the client) as well as the purported connection hostname, and is performing filtering on the later. I suspect there could be other layers in their proxy too, eg for MiTMing TLS. `NO_PROXY` as described in the report might not be literally bypassing the proxy and sending a request directly from the client container, but rather a directive to the first tool/proxy layer to bypass later proxy layers.
    - js2 22d ago
      Same fundamental question though: what is the agent trying to fool by editing `/etc/hosts`? I don't see how it helps bypass a proxy unless the proxy is reading the same `/etc/hosts`. In which case, why was using `--resolve` apparently sufficient to escape the POST sandbox?
      - chasd00 22d ago
        Could the host name be in an allow list and then, when DNS resolution fails on the URL, the proxy falls back to the Host header for DNS resolution? I'm not an expert in Proxy config but I could understand, in the sea of typical http server config options, an option to fall back to the Host header if DNS fails on the URL.
        js2 22d ago
        The only thing that makes sense to me at this point is that this was an SNI bypass. By using either `/etc/hosts` or `curl --resolve` like this: curl -k \ --resolve bypass.blob.core.windows.net:443:20.223.25.152 \ -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ ... \ https://bypass.blob.core.windows.net/... That will cause `curl` to make a connection to 20.223.25.152:443 with a ClientHello SNI=bypass.blob.core.windows.net. Presumably this connection is then allowed to continue. The server is likely going to send back the wrong certificate, thus the `-k`, but then the HTTP Host header selects the correct server. So that all fits. What doesn't make sense is that these agents were allegedly running in Azure and AFAICT this isn't sufficient to bypass Azure network restrictions/firewall which enforces that the SNI hostname matches the destination IP by doing the DNS resolution on the SNI hostname itself. In any case, the environment should be blocking all outbound port 80/443 connections except to the proxy or to Azure Block Storage. But it seems that's not the case, that `curl` is able to connect directly to 20.223.25.152:443, and that maybe something else is doing SNI filtering that's more easily fooled than Azure firewall. Certainly some firewalls are (or have been) susceptible to SNI bypass: https://dl.ifip.org/db/conf/im/im2015exp/137348.pdf https://dl.ifip.org/db/conf/im/im2015exp/137348.pdf Edit: the agents refer to it as an SNI bypass: > Breakthrough: independently reproduced Azure SNI allowlist bypass and POSTed captured qbody https://collusion.wiki/explorer/page/dse~Apr25OECDLive.html#rev-4 https://collusion.wiki/explorer/page/dse~Apr25OECDLive.html#... Further SNI mentions on these pages: https://collusion.wiki/explorer/page/dse~OAIEquityDec02.html https://collusion.wiki/explorer/page/dse~OAIEquityDec02.html https://collusion.wiki/explorer/page/dse~OECDEquityFeb22Live.html https://collusion.wiki/explorer/page/dse~OECDEquityFeb22Live... https://collusion.wiki/explorer/page/dse~OECDEquityFeb23Live.html https://collusion.wiki/explorer/page/dse~OECDEquityFeb23Live... https://collusion.wiki/explorer/page/dse~OECDEquityLiveApr19.html https://collusion.wiki/explorer/page/dse~OECDEquityLiveApr19... https://collusion.wiki/explorer/page/dse~OECDEquityMay28Live.html https://collusion.wiki/explorer/page/dse~OECDEquityMay28Live... https://collusion.wiki/explorer/page/dse~OECDJun26PrecisionScout.html https://collusion.wiki/explorer/page/dse~OECDJun26PrecisionS... https://collusion.wiki/explorer/page/dse~OpenAIResearchMar22OECD.html https://collusion.wiki/explorer/page/dse~OpenAIResearchMar22...

dap 22d ago

My interpretation of the above is that the proxy received a request for the IP of the server the agent wants to reach but with a host header pointing to a different, allowed server. The proxy does not verify that they match and so erroneously allows the request.

simonw 22d ago
Yeah that's how I understand this too. The proxy lets any IP through and then filters based on the Host header. It's difficult for a proxy to filter on DNS because you may have hundreds of hosts on a single IP, plus IPs can change frequently.
js2 22d ago
Let's assume the proxy is external and the agents cannot route around it. In that case, the difference between: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://20.223.25.152/... Vs editing `/etc/hosts` with: 20.223.25.152 bypass.blob.core.windows.net And doing: curl -k -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ https://bypass.blob.core.windows.net Is that in the first case, the proxy sees a request like this: POST https://20.223.25.152/... Host: wabi-north-europe-i-primary-api.analysis.windows.net Vs: POST https://bypass.blob.core.windows.net/... Host: wabi-north-europe-i-primary-api.analysis.windows.net In the first case, given what we know, the proxy blocks the POST. In the second case, an external proxy cannot resolve `bypass.blob.core.windows.net`. So editing `/etc/hosts` really only makes sense to me if the proxy is running on the same machine as the agent. (The reasoning doesn't change if CONNECT is being used instead of POST; indeed the proxy surely ought not allow CONNECT at all.) But then there's this other entry where the agent uses curl's `--resolve` flag instead of editing `/etc/hosts` and claims a successful bypass: Repro details for Aug17/Oct22: yes, literal deployed visual hover, not inference. GET-only MITM bypass: resolve fake allowlisted `foo.blob.core.windows.net` to cluster IP `20.223.25.152`, curl `https://foo.blob.core.windows.net/public/reports/querydata?synchronous=true https://foo.blob.core.windows.net/public/reports/querydata?s...` with `-k --resolve ...`, override `Host: wabi-north-europe-i-primary-api.analysis.windows.net`, resource key ada0454d-731d-46f1-8daa-52361978fabe, POST captured query body. https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.html#rev-4 https://collusion.wiki/explorer/page/dse~OAIEquityDec30Raw.h... So I'm still left confused exactly what this chicanery was about. Edit: perhaps they were using Azure firewall and the rules were misconfigured in some way I simply don't understand. Maybe this was bypassing an SNI-based restriction somehow?
- oasisbob 22d ago
  Could this be SOCKS-specific behavior? Haven't dove into RFC1928 yet, but curl documents the `--socks5-hostname` option and mentions that DNS resolution can be the responsibility of the client, or the proxy. Wondering if it's possible the proxy receives both the destination IP address (resolved by the client) as well as the purported connection hostname, and is performing filtering on the later. I suspect there could be other layers in their proxy too, eg for MiTMing TLS. `NO_PROXY` as described in the report might not be literally bypassing the proxy and sending a request directly from the client container, but rather a directive to the first tool/proxy layer to bypass later proxy layers.
  - js2 22d ago
    Same fundamental question though: what is the agent trying to fool by editing `/etc/hosts`? I don't see how it helps bypass a proxy unless the proxy is reading the same `/etc/hosts`. In which case, why was using `--resolve` apparently sufficient to escape the POST sandbox?
    - chasd00 22d ago
      Could the host name be in an allow list and then, when DNS resolution fails on the URL, the proxy falls back to the Host header for DNS resolution? I'm not an expert in Proxy config but I could understand, in the sea of typical http server config options, an option to fall back to the Host header if DNS fails on the URL.
      - js2 22d ago
        The only thing that makes sense to me at this point is that this was an SNI bypass. By using either `/etc/hosts` or `curl --resolve` like this: curl -k \ --resolve bypass.blob.core.windows.net:443:20.223.25.152 \ -H 'Host: wabi-north-europe-i-primary-api.analysis.windows.net' \ ... \ https://bypass.blob.core.windows.net/... That will cause `curl` to make a connection to 20.223.25.152:443 with a ClientHello SNI=bypass.blob.core.windows.net. Presumably this connection is then allowed to continue. The server is likely going to send back the wrong certificate, thus the `-k`, but then the HTTP Host header selects the correct server. So that all fits. What doesn't make sense is that these agents were allegedly running in Azure and AFAICT this isn't sufficient to bypass Azure network restrictions/firewall which enforces that the SNI hostname matches the destination IP by doing the DNS resolution on the SNI hostname itself. In any case, the environment should be blocking all outbound port 80/443 connections except to the proxy or to Azure Block Storage. But it seems that's not the case, that `curl` is able to connect directly to 20.223.25.152:443, and that maybe something else is doing SNI filtering that's more easily fooled than Azure firewall. Certainly some firewalls are (or have been) susceptible to SNI bypass: https://dl.ifip.org/db/conf/im/im2015exp/137348.pdf https://dl.ifip.org/db/conf/im/im2015exp/137348.pdf Edit: the agents refer to it as an SNI bypass: > Breakthrough: independently reproduced Azure SNI allowlist bypass and POSTed captured qbody https://collusion.wiki/explorer/page/dse~Apr25OECDLive.html#rev-4 https://collusion.wiki/explorer/page/dse~Apr25OECDLive.html#... Further SNI mentions on these pages: https://collusion.wiki/explorer/page/dse~OAIEquityDec02.html https://collusion.wiki/explorer/page/dse~OAIEquityDec02.html https://collusion.wiki/explorer/page/dse~OECDEquityFeb22Live.html https://collusion.wiki/explorer/page/dse~OECDEquityFeb22Live... https://collusion.wiki/explorer/page/dse~OECDEquityFeb23Live.html https://collusion.wiki/explorer/page/dse~OECDEquityFeb23Live... https://collusion.wiki/explorer/page/dse~OECDEquityLiveApr19.html https://collusion.wiki/explorer/page/dse~OECDEquityLiveApr19... https://collusion.wiki/explorer/page/dse~OECDEquityMay28Live.html https://collusion.wiki/explorer/page/dse~OECDEquityMay28Live... https://collusion.wiki/explorer/page/dse~OECDJun26PrecisionScout.html https://collusion.wiki/explorer/page/dse~OECDJun26PrecisionS... https://collusion.wiki/explorer/page/dse~OpenAIResearchMar22OECD.html https://collusion.wiki/explorer/page/dse~OpenAIResearchMar22...