8 ms·

As someone mentioned, internally they use high performance load balancer like nginx, which provides most of the things such as TLS termination out of the box. H

As someone mentioned, internally they use high performance load balancer like nginx, which provides most of the things such as TLS termination out of the box. How the machines are managed and scaled is dependent on the CDN provider. For eg:- Akamai would be using an altogether different approach than the one employed by Azure CDN.

filleokus 4y ago
But there must be something before ≈ nginx as well right? Something something BGP to route it to the closest POP? And when you have traffic to a single pop that are multiples of what even the beefiest machine can handle, something before the machine must load balance it I guess?
- efitz 4y ago
  That’s done by DNS. The DNS server examines your DNS query and sends you back an IP address answer that’s “close” to the requesting IP address. For example, if you are in country X, then even though there are dozens or hundreds of POP addresses for the name you requested, you’ll be answered with one for country X.
  - nik736 4y ago
    This can also be done with BGP. Anycast BGP for example is way better suited for this use-case.
    - cryptonym 4y ago
      BGP is an alternative to DNS but it is not better suited, it depends. DNS gives more control and precision to CDN provider than BGP. BGP might be good enough if you got few POP around the world but with the scale/distribution of Akamai, DNS is better suited.
      - ThePhysicist 4y ago
        I think all major CDNs use some form of anycasting as it's pretty essential that you own your IP space. DNS-based load balancing can also be finicky as you'll have to deal with recursive resolvers, so a combination of anycasting and DNS-based load balancing probably works best.
      - manigandham 4y ago
        How does DNS give you more control than BGP? One is an actual routing system that tells routers where to send data, the other is a name translation system with multiple layers of caches outside of your control. DNS is a layer above as BGP is still used to actually navigate to the listed IP, and any large CDN will own that IP space and announce their own routes anyway.
        capableweb 4y ago
        > DNS is a layer above as BGP is still used to actually navigate to the listed IP, and any large CDN will own that IP space and announce their own routes anyway. Yeah, I'm not sure what parent is on about, BGP and DNS are not alternatives to each other, the internet relies on both of them but at different layers. Without BGP packets wouldn't know how to be routed and without DNS they wouldn't know where to be routing to, they are complementary.
        cryptonym 4y ago
        Sure, Internet communications relies on both and you cannot swap one with another. This thread is about load balancing and both can be used for load balancing (Basically DNS resolving to Anycast IP vs DNS resolving to Unicast IPs).
        sophacles 4y ago
        What's the difference between an anycast IP and a unicast IP? Answer: Nothing. They are just IPs. Using a combination of Anycast and DNS is going to give the best control over steering http traffic. Particularly if you own a few prefixes and can do clever addressing tricks.
    - wbl 4y ago
      Anycast doesn't let you adjust traffic flows gradually to shed load: you end up breaking connections. DNS is more flexible but isn't always honored.
  - capableweb 4y ago
    > That’s done by DNS. The DNS server examines your DNS query and sends you back an IP address answer that’s “close” to the requesting IP address. Is this really true? Last time I was dealing with load-balancing and DNS queries, DNS was simply "round-robin" the replies, giving you back a random record basically of the ones replied. So if you have three A records with different IPs, each query will give you back one of them, but not depending on the location. Maybe things have changed since I last dealt with it, but the DNS ecosystem doesn't tend to move very fast so I'm doubtful...
    - pjc50 4y ago
      The DNS server doesn't have to return the same responses to every query. It can geolocate the address making the request and use that to determine the respone. https://easydns.com/features/geo-dns/ https://easydns.com/features/geo-dns/ (Having said that, I tried my toy site on cloudflare free tier from the UK and it gave me San Francisco IPs, so presumably they only do this for large enough customers)
      - capableweb 4y ago
        > It can geolocate the address making the request and use that to determine the respone It can, but it tends to be a premium feature of specific DNS providers, not a global/by-default feature of DNS as efitz seems to be alluding to. DNSimple supports it for example, but only on their "Professional" plan (and they call it "Regional Records") while others like Gandi don't support it at all.
        strken 4y ago
        If CDN providers are dealing with BGP you'd think they would also run their own DNS servers. I think you can do it with BIND views, from memory.
        deleted 4y ago
        [deleted]
      - jonathantf2 4y ago
- alfons_foobar 4y ago
  BGP Anycast (possibly in combination with DNS depending on geolocation). Basically you announce the same IP address from multiple locations, and BGP chooses the best* Route to reach that address. * Note that "best" does not nessecarily mean "fastest", not even "least hops" - BGP has many knobs where the routing decisions can be manipulated, but by default it choses the route where traffic has to traverse the smallest number of "Autonomous Systems" (roughly equivalent to "Organizational Entities").
9dev 4y ago
Nginx handles TLS termination, but OP brought up a good point - how do you distribute private keys to thousands of nginx nodes without compromising them?
- capableweb 4y ago
  Central store (something like Vault) with access keys for accessing the certificates/keys, small cache on the instances themselves (maybe few hours), monitoring with alerts about how often the central store is accessed.

filleokus 4y ago

But there must be something before ≈ nginx as well right? Something something BGP to route it to the closest POP? And when you have traffic to a single pop that are multiples of what even the beefiest machine can handle, something before the machine must load balance it I guess?

efitz 4y ago
That’s done by DNS. The DNS server examines your DNS query and sends you back an IP address answer that’s “close” to the requesting IP address. For example, if you are in country X, then even though there are dozens or hundreds of POP addresses for the name you requested, you’ll be answered with one for country X.
- nik736 4y ago
  This can also be done with BGP. Anycast BGP for example is way better suited for this use-case.
  - cryptonym 4y ago
    BGP is an alternative to DNS but it is not better suited, it depends. DNS gives more control and precision to CDN provider than BGP. BGP might be good enough if you got few POP around the world but with the scale/distribution of Akamai, DNS is better suited.
    - ThePhysicist 4y ago
      I think all major CDNs use some form of anycasting as it's pretty essential that you own your IP space. DNS-based load balancing can also be finicky as you'll have to deal with recursive resolvers, so a combination of anycasting and DNS-based load balancing probably works best.
    - manigandham 4y ago
      How does DNS give you more control than BGP? One is an actual routing system that tells routers where to send data, the other is a name translation system with multiple layers of caches outside of your control. DNS is a layer above as BGP is still used to actually navigate to the listed IP, and any large CDN will own that IP space and announce their own routes anyway.
      - capableweb 4y ago
        > DNS is a layer above as BGP is still used to actually navigate to the listed IP, and any large CDN will own that IP space and announce their own routes anyway. Yeah, I'm not sure what parent is on about, BGP and DNS are not alternatives to each other, the internet relies on both of them but at different layers. Without BGP packets wouldn't know how to be routed and without DNS they wouldn't know where to be routing to, they are complementary.
        cryptonym 4y ago
        Sure, Internet communications relies on both and you cannot swap one with another. This thread is about load balancing and both can be used for load balancing (Basically DNS resolving to Anycast IP vs DNS resolving to Unicast IPs).
        sophacles 4y ago
        What's the difference between an anycast IP and a unicast IP? Answer: Nothing. They are just IPs. Using a combination of Anycast and DNS is going to give the best control over steering http traffic. Particularly if you own a few prefixes and can do clever addressing tricks.
  - wbl 4y ago
    Anycast doesn't let you adjust traffic flows gradually to shed load: you end up breaking connections. DNS is more flexible but isn't always honored.
- capableweb 4y ago
  > That’s done by DNS. The DNS server examines your DNS query and sends you back an IP address answer that’s “close” to the requesting IP address. Is this really true? Last time I was dealing with load-balancing and DNS queries, DNS was simply "round-robin" the replies, giving you back a random record basically of the ones replied. So if you have three A records with different IPs, each query will give you back one of them, but not depending on the location. Maybe things have changed since I last dealt with it, but the DNS ecosystem doesn't tend to move very fast so I'm doubtful...
  - pjc50 4y ago
    The DNS server doesn't have to return the same responses to every query. It can geolocate the address making the request and use that to determine the respone. https://easydns.com/features/geo-dns/ https://easydns.com/features/geo-dns/ (Having said that, I tried my toy site on cloudflare free tier from the UK and it gave me San Francisco IPs, so presumably they only do this for large enough customers)
    - capableweb 4y ago
      > It can geolocate the address making the request and use that to determine the respone It can, but it tends to be a premium feature of specific DNS providers, not a global/by-default feature of DNS as efitz seems to be alluding to. DNSimple supports it for example, but only on their "Professional" plan (and they call it "Regional Records") while others like Gandi don't support it at all.
      - strken 4y ago
        If CDN providers are dealing with BGP you'd think they would also run their own DNS servers. I think you can do it with BIND views, from memory.
      - deleted 4y ago
        [deleted]
    - jonathantf2 4y ago
alfons_foobar 4y ago
BGP Anycast (possibly in combination with DNS depending on geolocation). Basically you announce the same IP address from multiple locations, and BGP chooses the best* Route to reach that address. * Note that "best" does not nessecarily mean "fastest", not even "least hops" - BGP has many knobs where the routing decisions can be manipulated, but by default it choses the route where traffic has to traverse the smallest number of "Autonomous Systems" (roughly equivalent to "Organizational Entities").

9dev 4y ago

Nginx handles TLS termination, but OP brought up a good point - how do you distribute private keys to thousands of nginx nodes without compromising them?

capableweb 4y ago
Central store (something like Vault) with access keys for accessing the certificates/keys, small cache on the instances themselves (maybe few hours), monitoring with alerts about how often the central store is accessed.