6 ms·

A Primer on Proxies

142 points by jgrahamc 5y ago

ignoramous 5y ago
Tommy Pauly's EPIQCon 2021 keynote QUIC at Apple is pretty interesting as well: https://www.youtube-nocookie.com/embed/nP1yzxHcgeM https://www.youtube-nocookie.com/embed/nP1yzxHcgeM > What if we wanted to proxy QUIC? What if we wanted to proxy entire IP datagrams, similar to VPN technologies like IPsec or WireGuard? This is where MASQUE comes in. Ref: https://ietf-wg-masque.github.io/draft-ietf-masque-connect-ip/draft-ietf-masque-connect-ip.html https://ietf-wg-masque.github.io/draft-ietf-masque-connect-i...
- eptcyka 5y ago
  Thanks for the link to the presentation, I was just debugging some weirdness with PF and unblockable QUIC traffic on macOS Monterey.
  - ignoramous 5y ago
    Np. But you must fills us in: Why was iOS QUIC unblockable at the firewall? And how did the presentation help?
    - eptcyka 5y ago
      I will do once I've figured out what is actually going on. Maybe it's as banal as a rule to always allow traffic to a specific IP that's added by some macOS component to PF.
jgrahamc 5y ago
The follow up is also a good read: https://blog.cloudflare.com/unlocking-quic-proxying-potential/ https://blog.cloudflare.com/unlocking-quic-proxying-potentia...
ThePhysicist 5y ago
In my experience HTTP/2 doesn't boost proxy performance significantly, at least for HTTPs workloads. Normally the client sends a single CONNECT [hostname] request to the proxy and after that the proxy just forwards TCP packets on that connection, which is nothing that HTTP/2 can improve. A client can of course use the same HTTP/2 connection for multiple CONNECT requests, but opening several data streams to the proxy in parallel isn't much more costly as those streams tend to be long-lived and will often carry multiplexed HTTP/2 connections themselves, so further multiplexing on the proxy level often has little effect (in my experience at least). Regarding QUIC/MASQUE it's still difficult to find any clients that support it. I think Chrome supports using QUIC to connect to a regular HTTP CONNECT proxy, but I don't know of any browsers that have even experimental support for MASQUE as of now. Will be pretty great tough I think once it's supported more widely.
- Liuser 5y ago
  > Normally the client sends a single CONNECT [hostname] request to the proxy and after that the proxy just forwards TCP packets on that connection, which is nothing that HTTP/2 can improve. Trying to test my understanding - Does the CONNECT HTTP protocol need to match the underlying payload protocol. Eg After CONNECT over HTTP/1.1 tunnel is established it’s still possible for the client to still use HTTP/2 with the upstream server for its underlying payloads correct? My intuition is that it doesn’t need to match because the proxy has no way to know what http protocol is being used when the workload is encrypted.
  - simmervigor 5y ago
    The vanilla CONNECT method is an instruction to open a TCP connection to the target server. What is sent over that is entirely up to client and target the server it doesn't need to match. Its often TLS carrying HTTP but it could be anything. The proxy could inspect the traffic it is carrying and try to enforce some access control or policy. However, the use of TLS or other encrypted protocols limits the ability to do see what is happening. Then you get into a different logical layer of whether there is MITM happening, but that is tangential to conventional use case for HTTP proxy.
latchkey 5y ago
It would be nice if there was a way to more easily debug caching issues with CF workers being used as proxies. There is very little visibility into how headers affect things and it is poking in the dark to make it all work.
matt-attack 5y ago
Is it ok if I don’t like the term “reverse proxy”? I find it entirely confusing and non-intuitive. I put it up there with idiotic terms like “OTT” which AFAIK just means “connected to the internet”.
- jgrahamc 5y ago
  Has always bothered me also, but that's the industry term so we are kind of stuck with it.
  - simmervigor 5y ago
    Proxy components are officially called Intermediaries in thr HTTP semantic specification; see https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#intermediaries https://httpwg.org/http-core/draft-ietf-httpbis-semantics-la.... Intermediaries can have different purposes. The official alternative to reverse proxy is "gateway", which is unfortunately overloaded with other kinds of gateways in networking. Naming things is hard. Reverse proxy isn't great but all things considered is unique enough to allow folks to discriminate the sort of HTTP proxying that is happening
    - westurner 5y ago
      An HTTP reverse proxy forwards HTTP requests and adds e.g. X-Forwarded-For and X-Forwarded-Host headers. https://www.nginx.com/resources/wiki/start/topics/examples/forwarded/ https://www.nginx.com/resources/wiki/start/topics/examples/f... : X-Forwarded-For: 12.34.56.78, 23.45.67.89 X-Real-IP: 12.34.56.78 X-Forwarded-Host: example.com X-Forwarded-Proto: https TIL from the nginx docs that there's a standardized way to forward HTTP without the X- prefix on the unregistered headers: Forwarded: for=12.34.56.78;host=example.com;proto=https, for=23.45.67.89 What is the difference between a reverse proxy and a load balancer? k8s calls this "Ingress" and there are multiple "Ingress API" implementers; which essentially must reload the upstream server list on SIGHUP. https://kubernetes.io/docs/concepts/services-networking/ingress/ https://kubernetes.io/docs/concepts/services-networking/ingr... List of k8s Ingress Controllers: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/ https://kubernetes.io/docs/concepts/services-networking/ingr...
deleted 5y ago
[deleted]
whoknew1122 5y ago
I always enjoy the articles Cloudflare publishes. I still really don't like the company, but good job to the individuals who wrote the primer.
- datalopers 5y ago
  > I still really don't like the company Is that because they provide TLS termination for an increasingly large chunk of all internet traffic?
  - whoknew1122 5y ago
    I really don't want to get sidetracked into a political or philosophical conversation. But to me, a policy of content neutrality is tacitly supporting extremist content. Cloudflare has to be dragged kicking and screaming before it stops profiting and protecting extremist sites and sites which host illegal content. The concentration of internet infrastructure into a few companies is something to be concerned about. But my bigger beef with Cloudflare is that they profit from and protect despicable websites and only cave when public pressure is high enough.
    - mqnfred 5y ago
      That to me is a reason to like the company rather than not... Let's see where you stand once your opinions are on the chopping block
      - whoknew1122 5y ago
        The company sure made a courageous stance protecting a site which advocates explicitly for accelerationist race wars, and another image board that was notorious for housing child abuse imagery and conspiracies which led to real-world deaths. The answer to where a company should draw the line isn't 'OMG, slippery slope!' It's 'somewhere.' And preferably the CEO of the company shouldn't agonize about getting rid of stuff like the Daily Stormer and 8chan.
kazinator 5y ago
I haven't paid much attention to this stuff, but recently I discovered a trick for slowly migrating the web services on an old server to a new one: via reverse proxy definitions on the new server that pass through selective paths to the old one. To the outside, it looks like one server. The new server has the SSL identity with up-to-date crypto and all and terminates the HTTPS connections. As I'm doing this, suddenly it hits me: oh so this is Cloudfare in a mini nutshell?

142 points by jgrahamc 5y ago

Tommy Pauly's EPIQCon 2021 keynote QUIC at Apple is pretty interesting as well: https://www.youtube-nocookie.com/embed/nP1yzxHcgeM https://www.youtube-nocookie.com/embed/nP1yzxHcgeM > What if we wanted to proxy QUIC? What if we wanted to proxy entire IP datagrams, similar to VPN technologies like IPsec or WireGuard? This is where MASQUE comes in. Ref: https://ietf-wg-masque.github.io/draft-ietf-masque-connect-ip/draft-ietf-masque-connect-ip.html https://ietf-wg-masque.github.io/draft-ietf-masque-connect-i...

eptcyka 5y ago
Thanks for the link to the presentation, I was just debugging some weirdness with PF and unblockable QUIC traffic on macOS Monterey.
- ignoramous 5y ago
  Np. But you must fills us in: Why was iOS QUIC unblockable at the firewall? And how did the presentation help?
  - eptcyka 5y ago
    I will do once I've figured out what is actually going on. Maybe it's as banal as a rule to always allow traffic to a specific IP that's added by some macOS component to PF.

jgrahamc 5y ago

The follow up is also a good read: https://blog.cloudflare.com/unlocking-quic-proxying-potential/ https://blog.cloudflare.com/unlocking-quic-proxying-potentia...

ThePhysicist 5y ago

In my experience HTTP/2 doesn't boost proxy performance significantly, at least for HTTPs workloads. Normally the client sends a single CONNECT [hostname] request to the proxy and after that the proxy just forwards TCP packets on that connection, which is nothing that HTTP/2 can improve. A client can of course use the same HTTP/2 connection for multiple CONNECT requests, but opening several data streams to the proxy in parallel isn't much more costly as those streams tend to be long-lived and will often carry multiplexed HTTP/2 connections themselves, so further multiplexing on the proxy level often has little effect (in my experience at least). Regarding QUIC/MASQUE it's still difficult to find any clients that support it. I think Chrome supports using QUIC to connect to a regular HTTP CONNECT proxy, but I don't know of any browsers that have even experimental support for MASQUE as of now. Will be pretty great tough I think once it's supported more widely.

Liuser 5y ago
> Normally the client sends a single CONNECT [hostname] request to the proxy and after that the proxy just forwards TCP packets on that connection, which is nothing that HTTP/2 can improve. Trying to test my understanding - Does the CONNECT HTTP protocol need to match the underlying payload protocol. Eg After CONNECT over HTTP/1.1 tunnel is established it’s still possible for the client to still use HTTP/2 with the upstream server for its underlying payloads correct? My intuition is that it doesn’t need to match because the proxy has no way to know what http protocol is being used when the workload is encrypted.
- simmervigor 5y ago
  The vanilla CONNECT method is an instruction to open a TCP connection to the target server. What is sent over that is entirely up to client and target the server it doesn't need to match. Its often TLS carrying HTTP but it could be anything. The proxy could inspect the traffic it is carrying and try to enforce some access control or policy. However, the use of TLS or other encrypted protocols limits the ability to do see what is happening. Then you get into a different logical layer of whether there is MITM happening, but that is tangential to conventional use case for HTTP proxy.

latchkey 5y ago

It would be nice if there was a way to more easily debug caching issues with CF workers being used as proxies. There is very little visibility into how headers affect things and it is poking in the dark to make it all work.

matt-attack 5y ago

Is it ok if I don’t like the term “reverse proxy”? I find it entirely confusing and non-intuitive. I put it up there with idiotic terms like “OTT” which AFAIK just means “connected to the internet”.

jgrahamc 5y ago
Has always bothered me also, but that's the industry term so we are kind of stuck with it.
- simmervigor 5y ago
  Proxy components are officially called Intermediaries in thr HTTP semantic specification; see https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#intermediaries https://httpwg.org/http-core/draft-ietf-httpbis-semantics-la.... Intermediaries can have different purposes. The official alternative to reverse proxy is "gateway", which is unfortunately overloaded with other kinds of gateways in networking. Naming things is hard. Reverse proxy isn't great but all things considered is unique enough to allow folks to discriminate the sort of HTTP proxying that is happening
  - westurner 5y ago
    An HTTP reverse proxy forwards HTTP requests and adds e.g. X-Forwarded-For and X-Forwarded-Host headers. https://www.nginx.com/resources/wiki/start/topics/examples/forwarded/ https://www.nginx.com/resources/wiki/start/topics/examples/f... : X-Forwarded-For: 12.34.56.78, 23.45.67.89 X-Real-IP: 12.34.56.78 X-Forwarded-Host: example.com X-Forwarded-Proto: https TIL from the nginx docs that there's a standardized way to forward HTTP without the X- prefix on the unregistered headers: Forwarded: for=12.34.56.78;host=example.com;proto=https, for=23.45.67.89 What is the difference between a reverse proxy and a load balancer? k8s calls this "Ingress" and there are multiple "Ingress API" implementers; which essentially must reload the upstream server list on SIGHUP. https://kubernetes.io/docs/concepts/services-networking/ingress/ https://kubernetes.io/docs/concepts/services-networking/ingr... List of k8s Ingress Controllers: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/ https://kubernetes.io/docs/concepts/services-networking/ingr...

deleted 5y ago

[deleted]

whoknew1122 5y ago

I always enjoy the articles Cloudflare publishes. I still really don't like the company, but good job to the individuals who wrote the primer.

datalopers 5y ago
> I still really don't like the company Is that because they provide TLS termination for an increasingly large chunk of all internet traffic?
- whoknew1122 5y ago
  I really don't want to get sidetracked into a political or philosophical conversation. But to me, a policy of content neutrality is tacitly supporting extremist content. Cloudflare has to be dragged kicking and screaming before it stops profiting and protecting extremist sites and sites which host illegal content. The concentration of internet infrastructure into a few companies is something to be concerned about. But my bigger beef with Cloudflare is that they profit from and protect despicable websites and only cave when public pressure is high enough.
  - mqnfred 5y ago
    That to me is a reason to like the company rather than not... Let's see where you stand once your opinions are on the chopping block
    - whoknew1122 5y ago
      The company sure made a courageous stance protecting a site which advocates explicitly for accelerationist race wars, and another image board that was notorious for housing child abuse imagery and conspiracies which led to real-world deaths. The answer to where a company should draw the line isn't 'OMG, slippery slope!' It's 'somewhere.' And preferably the CEO of the company shouldn't agonize about getting rid of stuff like the Daily Stormer and 8chan.

kazinator 5y ago

I haven't paid much attention to this stuff, but recently I discovered a trick for slowly migrating the web services on an old server to a new one: via reverse proxy definitions on the new server that pass through selective paths to the old one. To the outside, it looks like one server. The new server has the SSL identity with up-to-date crypto and all and terminates the HTTPS connections. As I'm doing this, suddenly it hits me: oh so this is Cloudfare in a mini nutshell?