4 ms·

Why does the proxy need to terminate TLS? If it were just a TCP proxy, then at least the monitoring wouldn't have been fooled into thinking the certificate was

by agwa 3y ago

Why does the proxy need to terminate TLS? If it were just a TCP proxy, then at least the monitoring wouldn't have been fooled into thinking the certificate wasn't about to expire. Heck, a TCP proxy might even allow automatic renewal to work if the domain validation is being done using a TLS-ALPN challenge.

fanf2 3y ago
I think a TCP proxy would also work with http challenges.
- agwa 3y ago
  It would, but so would an HTTP proxy. It makes make think the hosting provider doesn't use HTTP challenges.
  - fanf2 3y ago
    An http proxy would need to be configured cleverly enough to serve its own acme challenges directly, and proxy any requests for the backend’s acme challenges. Which is I think the trick that was missed by the tailscale setup.
    - agwa 3y ago
      Good point!
bastawhiz 3y ago
Not that it's an amazing reason, but H3 doesn't run over TCP, and running a UDP proxy doesn't sound like a great time.
- ignoramous 3y ago
  > UDP proxy doesn't sound like a great time QUIC, in particular, is harder to proxy (if you're load balancing, say: https://quicwg.org/ops-drafts/draft-ietf-quic-manageability.html#name-specific-network-management https://quicwg.org/ops-drafts/draft-ietf-quic-manageability....). If it is point-to-point and you control both those points (forward A to B with ports open as approp), proxying any protocol should be straightforward, no?
  - cayde 3y ago
    I donet believe they control the Vercel endpoint "B"
RulerOf 3y ago
They may have AWS CloudFront CDN in front of it for IPv6. If you're doing that, you're terminating TLS at CloudFront. I don't believe that's optional.
amluto 3y ago
A non-TLS-terminating proxy is a great thing to host on a service like Hetzner. If you set up CAA correctly, then you are trusting the provider for latency and availability only, and you might as well avoid hilariously expensive services like CloudFront or an EC2-based proxy. Hmm, it looks like Tailscale is using NetActuate for pkgs.tailscale.com. I bet NetActuate could help serve up a non-terminating proxy with plenty of PoPs at a reasonable price. Their website doesn’t give pricing, but it sounds like the kind of company that doesn’t mark up egress 50x.
- SahAssar 3y ago
  > A non-TLS-terminating proxy is a great thing to host on a service like Hetzner. If you set up CAA correctly, then you are trusting the provider for latency and availability only, and you might as well avoid hilariously expensive services like CloudFront or an EC2-based proxy. Are you really getting any latency or availability improvements in that case? What does a non-TLS-terminating proxy give you?
  - amluto 3y ago
    Functionality. The reason for having a proxy at all seems to be that Vercel doesn’t support IPv4.
bradfitz 3y ago
It doesn't. That was one of our mistakes and action items to fix. The original proxy was stood up quickly when it was first discovered IPv6 was broken and the people standing up the proxy didn't know at the time how ACME worked. We'll be changing it to just a TCP proxy.
- ikiris 3y ago
  > and the people standing up the proxy didn't know at the time how ACME worked yikes
  - bradfitz 3y ago
    To be fair, it didn't help the provider's docs were inconsistent about whether dns-01 or http-01 was to be used.
    - PokestarFan 3y ago
      I've done enough fighting with Certbot to learn that the path of least resistance is to use dns-01 with the cloudflare plugin, and just make a very limited access key and store it on the servers that I am using to renew the cert.
p1mrx 3y ago
A TCP proxy discards the user's IP address, unless you use something like the PROXY protocol[1], which then needs to be supported by the target HTTPS server. You would also need a way to prevent unauthorized users from injecting their own PROXY header. This isn't a problem if you don't need the user's IP address at all, but it's often useful for logging and abuse detection. [1] https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt

by agwa 3y ago

fanf2 3y ago

I think a TCP proxy would also work with http challenges.

agwa 3y ago
It would, but so would an HTTP proxy. It makes make think the hosting provider doesn't use HTTP challenges.
- fanf2 3y ago
  An http proxy would need to be configured cleverly enough to serve its own acme challenges directly, and proxy any requests for the backend’s acme challenges. Which is I think the trick that was missed by the tailscale setup.
  - agwa 3y ago
    Good point!

bastawhiz 3y ago

Not that it's an amazing reason, but H3 doesn't run over TCP, and running a UDP proxy doesn't sound like a great time.

ignoramous 3y ago
> UDP proxy doesn't sound like a great time QUIC, in particular, is harder to proxy (if you're load balancing, say: https://quicwg.org/ops-drafts/draft-ietf-quic-manageability.html#name-specific-network-management https://quicwg.org/ops-drafts/draft-ietf-quic-manageability....). If it is point-to-point and you control both those points (forward A to B with ports open as approp), proxying any protocol should be straightforward, no?
- cayde 3y ago
  I donet believe they control the Vercel endpoint "B"

RulerOf 3y ago

They may have AWS CloudFront CDN in front of it for IPv6. If you're doing that, you're terminating TLS at CloudFront. I don't believe that's optional.

amluto 3y ago

A non-TLS-terminating proxy is a great thing to host on a service like Hetzner. If you set up CAA correctly, then you are trusting the provider for latency and availability only, and you might as well avoid hilariously expensive services like CloudFront or an EC2-based proxy. Hmm, it looks like Tailscale is using NetActuate for pkgs.tailscale.com. I bet NetActuate could help serve up a non-terminating proxy with plenty of PoPs at a reasonable price. Their website doesn’t give pricing, but it sounds like the kind of company that doesn’t mark up egress 50x.

SahAssar 3y ago
> A non-TLS-terminating proxy is a great thing to host on a service like Hetzner. If you set up CAA correctly, then you are trusting the provider for latency and availability only, and you might as well avoid hilariously expensive services like CloudFront or an EC2-based proxy. Are you really getting any latency or availability improvements in that case? What does a non-TLS-terminating proxy give you?
- amluto 3y ago
  Functionality. The reason for having a proxy at all seems to be that Vercel doesn’t support IPv4.

bradfitz 3y ago

It doesn't. That was one of our mistakes and action items to fix. The original proxy was stood up quickly when it was first discovered IPv6 was broken and the people standing up the proxy didn't know at the time how ACME worked. We'll be changing it to just a TCP proxy.

ikiris 3y ago
> and the people standing up the proxy didn't know at the time how ACME worked yikes
- bradfitz 3y ago
  To be fair, it didn't help the provider's docs were inconsistent about whether dns-01 or http-01 was to be used.
  - PokestarFan 3y ago
    I've done enough fighting with Certbot to learn that the path of least resistance is to use dns-01 with the cloudflare plugin, and just make a very limited access key and store it on the servers that I am using to renew the cert.

p1mrx 3y ago

A TCP proxy discards the user's IP address, unless you use something like the PROXY protocol[1], which then needs to be supported by the target HTTPS server. You would also need a way to prevent unauthorized users from injecting their own PROXY header. This isn't a problem if you don't need the user's IP address at all, but it's often useful for logging and abuse detection. [1] https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt