3 ms·

I always wanted user agents and DNS to do a little bit more coordination here. The user agent should ask the DNS server "how do I get to http for example.com",

I always wanted user agents and DNS to do a little bit more coordination here. The user agent should ask the DNS server "how do I get to http for example.com", and then DNS should reply with an IP address and a port number (so if you can't bind 443, you don't have to). Additionally, these records should have extra information, perhaps related to location (so that the entire world has a consistent view of A records associated with the host, but can still find the CDN edge node closest to them), and health checking (so if you have 1.2.3.4, 1.2.3.5, and 1.2.3.6 as options, and 1.2.3.6 is being rebooted, the browser could test them all and use the one that is supposedly working). Oh, and maybe we could go totally crazy and add some information about authentication, so the user agent could get a token from its identity provider of choice and send that with the initial request (if the user's preferences allow it), and save engineers from ever having to implement sign up / sign in / account recovery / etc. As it stands now, we have conventions for port numbers, and load balancers / VIPs for mapping the address in the DNS response to an actual computer running the desired services. We could kill a lot of complexity by letting user agents do the work of the load balancer here. (The port number is a personal pet peeve of mine. I want to run HTTP/3 for my website, but my cloud provider's load balancer can't represent a configuration where port 443/TCP and port 443/UDP are both managed by the same load balancer. Interestingly, browsers are configured to do HTTP/3 with an Alt-Svc on an HTTP/2 or HTTP/1.1 response, which includes the port number. Chrome refuses to use anything other than port 443, while Firefox is happy with something like 30443. As a result, I have to reconfigure my network, or my cloud provider has to fix their bugs... and there is just no need. The browser could do this for me for free!)

fragmede 4y ago
You need a layer 4 load balancer instead of a layer 7 balancer in that case.
- jrockway 4y ago
  It's worse than that. It seems to be a Kubernetes defect around merge keys; using the port number and not the protocol + port number. https://github.com/kubernetes/kubernetes/issues/39188 https://github.com/kubernetes/kubernetes/issues/39188 But it is possible to create Kubernetes services that work correctly serving UDP and TCP on the same port; look at any old kube-dns/coredns service, for example. Indeed, I can easily create such a service for my HTTP proxy (sending TCP 443, TCP 80, and UDP 443 to my L7 proxy), but the cloud provider's software explicitly rejects it because it can't possibly work. (Meanwhile, going into their UI and modifying their object behind the k8s reconciler's back works fine, but not well enough that I trust it to announce HTTP/3 support.) I think they got spooked by a bug, but didn't realize it's a corner case and not something that happens 100% of the time. I hope it will be fixed, because I'm pretty sure they added UDP support to load balancers specifically for HTTP/3.
tguvot 4y ago
you are kind of describing DNS SRV records. but browsers don't use it
kelnos 4y ago
DNS SRV records were designed for that. SIP makes heavy use of them; for example: _sip._tls.example.com SRV 5 100 5060 sip.example.com. That means that TLS-protected SIP is on port 5060 on sip.example.com. In theory you can do this for http/https as well, but in practice, unfortunately, browsers don't do SRV lookups.
tsimionescu 4y ago
I don't think this is a good idea at all. Most importantly, you can't rely on the client for doing load balancing! Load balancing is about making sure that one user can't affect the service quality for other users. Exposing all backend servers directly to users and asking them to choose which to use is a recipe for denial of service attacks and floods. Not to mention that LBs can help with auto-scaling based on health checks, but you can't rely on client health checks for that, and DNS would be entirely out of the loop. DNS as used most commonly by far on the internet is a non-authenticated non-encrypted protocol. Everyone with access to the network can see what DNS requests you are making, so putting even more information in them is not great. Further, DNS is already a single point of failure, but it at least has the advantage of relatively long cache times - so clients who already resolved the IP they need can keep working. Relying on the DNS Server instead of a separate LB seems like it would greatly increase the risk that you'd constantly be updating DNS records (for example, when your service auto-scales, you have to add/remove DNS records now) meaning you'd want very very short caches etc. Finally, fixed port numbers are a critical part of having a low attack footprint. Even if you convinced browsers to talk to your service on port 30443 automatically, you'd soon discover that many corporate customers can't reach your service, because their network/firewall blocks outbound traffic to unknown ports. Having dynamic port numbers for protocols would be an anti-feature in most cases. DNS is already doing the most that it should do. Bloating it with even more responsibility should be avoided, not encouraged.

fragmede 4y ago

You need a layer 4 load balancer instead of a layer 7 balancer in that case.

jrockway 4y ago
It's worse than that. It seems to be a Kubernetes defect around merge keys; using the port number and not the protocol + port number. https://github.com/kubernetes/kubernetes/issues/39188 https://github.com/kubernetes/kubernetes/issues/39188 But it is possible to create Kubernetes services that work correctly serving UDP and TCP on the same port; look at any old kube-dns/coredns service, for example. Indeed, I can easily create such a service for my HTTP proxy (sending TCP 443, TCP 80, and UDP 443 to my L7 proxy), but the cloud provider's software explicitly rejects it because it can't possibly work. (Meanwhile, going into their UI and modifying their object behind the k8s reconciler's back works fine, but not well enough that I trust it to announce HTTP/3 support.) I think they got spooked by a bug, but didn't realize it's a corner case and not something that happens 100% of the time. I hope it will be fixed, because I'm pretty sure they added UDP support to load balancers specifically for HTTP/3.

tguvot 4y ago

you are kind of describing DNS SRV records. but browsers don't use it

kelnos 4y ago

DNS SRV records were designed for that. SIP makes heavy use of them; for example: _sip._tls.example.com SRV 5 100 5060 sip.example.com. That means that TLS-protected SIP is on port 5060 on sip.example.com. In theory you can do this for http/https as well, but in practice, unfortunately, browsers don't do SRV lookups.

tsimionescu 4y ago

I don't think this is a good idea at all. Most importantly, you can't rely on the client for doing load balancing! Load balancing is about making sure that one user can't affect the service quality for other users. Exposing all backend servers directly to users and asking them to choose which to use is a recipe for denial of service attacks and floods. Not to mention that LBs can help with auto-scaling based on health checks, but you can't rely on client health checks for that, and DNS would be entirely out of the loop. DNS as used most commonly by far on the internet is a non-authenticated non-encrypted protocol. Everyone with access to the network can see what DNS requests you are making, so putting even more information in them is not great. Further, DNS is already a single point of failure, but it at least has the advantage of relatively long cache times - so clients who already resolved the IP they need can keep working. Relying on the DNS Server instead of a separate LB seems like it would greatly increase the risk that you'd constantly be updating DNS records (for example, when your service auto-scales, you have to add/remove DNS records now) meaning you'd want very very short caches etc. Finally, fixed port numbers are a critical part of having a low attack footprint. Even if you convinced browsers to talk to your service on port 30443 automatically, you'd soon discover that many corporate customers can't reach your service, because their network/firewall blocks outbound traffic to unknown ports. Having dynamic port numbers for protocols would be an anti-feature in most cases. DNS is already doing the most that it should do. Bloating it with even more responsibility should be avoided, not encouraged.