5 ms·

I wonder how relevant "long" TTLs (5min, 30min, etc.) still are in an age of massive multi-gigabit fiber links? DNS was invented long ago, a very simple protoco

I wonder how relevant "long" TTLs (5min, 30min, etc.) still are in an age of massive multi-gigabit fiber links? DNS was invented long ago, a very simple protocol with a handful of bytes per packet - a trickle amid the torrent. Waiting an hour or more for a 4-byte change (an IPv4 address) to be committed to a distributed database seems incredibly antiquated.

bombcar 6y ago
It's arguably even MORE important in the age of fast links - because now a great percentage of the clock time is spent on handshakes and other "overhead" - so saving a round trip or three on a DNS lookup is comparatively larger than it used to be.
dijit 6y ago
Argh, I hate this kind of question, it overtly implies that because we have more of something (computing resources, memory, networking bandwidth) we should fill the void with something "better", where "better" means different things to different people. DNS doesn't really work this way anyway, you have a tree-like structure and if you're a good little resolver you don't just talk to your upstream DNS server, you walk the path from the root to the stem. I.E; you don't ask your ISPs resolver for server-a.www.google.com you ask the DNS root (".") for who owns com, then you ask "com" who owns google, and you keep going down the NS records you're on the stem, then you ask for an A or a CNAME record.. Then you cache that result _because it's expensive to do that lookup_- and often (ESPECIALLY) on the internet a webpage will ask you to download a dozen more things from the same domain, so a cache makes absolute sense. But, regardless of how expensive DNS is; you miss a _huge_ point about the tech that came before: it's foundational and fundamental to how things work, it's so "light" that we bake it into our products, and increasing the "cost" causes an exponential cost to end users because it's so widely adopted and essential. This industry seems to want to eat itself, taking things that work decently well and noodling on them until something much more costly comes along, why on earth would we take DNS before tackling the much more necessary project of fixing email?
- Daho0n 6y ago
  Which resolvers actually follow that path?
  - tinus_hn 6y ago
    All resolvers either follow that path or ask someone else to follow that path. It’s the only way to get the answer.
- QuadrupleA 6y ago
  I don't know how suggesting tighter TTLs is indicative of "the industry eating itself" or that we can't fix email at the same time. Relax. "com" and "." can obviously be cached a long time, they'll rarely change servers. No need to walk the whole tree every time - god forbid between multiple requests made by a single webpage like you're implying. Subdomains are typically only 1 or 2 names on top of the TLD. I'm all for being efficient with computer resources, and like others said there's a latency component here. But seriously, DNS packets are tiny, typically 50 bytes or so. Assuming, say, 80 bytes after UDP, IP, Ethernet framing etc. (just a ballpark) that's 640/10,000,000,000 bits in 1 second of time on a 10gbps fiber, or about 1 of 1.5 million DNS packets that fiber (of many fibers) could send every second. At that scale, I don't see why TTLs of say 1 minute wouldn't be fine. Multiply everything above by 60 seconds and that's still an effective cache of a 50 byte response in computer time. Authoritative NS records can change less often, so intermediate servers can just check direct with the authoritative servers when records go stale. Say 100 bytes roundtrip once a minute. I'm not taking about throwing away or redesigning DNS - just that tighter TTLs than typically recommended are probably fine and would make life a little easier with propagation delays.
  - carlhjerpe 6y ago
    Cloudflares automatic TTL is 5 minutes. Web browsers will look up your records and try to connect to one of the returned ips, if that one doesn't work it'll use another one. The fact that the client does this for us means we can keep the TTL quite high (5 minutes) without 5 minutes downtime if we lose connectivity to one IP. This is also used to load balance, a single load balancer can't handle a trillion connections, so we round robin DNS for load balancing before the load balancers. There's also anycasting which is something cloudflare uses a lot. Which means you're announcing the same IP block from many locations, and with how BGP works (which is the defacto protocol for exchanging route information on the internet) the "closer" you are the more likely the route is to be chosen (AS path distance). Meaning sites behind Cloudflare is protected by thousands of servers listening to the two same addresses you see if you're behind a Cloudflare protected site and behind these thousands of servers are even more thousands of servers. This means if i were to launch a 10Gbps DoS from my server, it would hit the Cloudflare datacenter closest to me, and there's nothing i can do about it, meaning the rest of the world is unaffected. No, I'm not afiliated with CF. It's just an approachable example, i believe (but don't know) that Fastly has the same featureset (that I've mentioned above)
mjevans 6y ago
Caching this lookup helps to reduce the __latency__ at the client end. A local, often in RAM, DNS cache should honor the TTLs (for validity expiration) if the entries aren't purged out by other activity.
chrisandchris 6y ago
Assume you‘ll use Amazon Route53 as your DNS provider. You‘ll pay per lookup after some free-tier so doubling your TTL from 5min to 10min halfes your cost [1] . Sure, it‘s ~$0.40 / million requests but if you have a TTL of a few minutes and a often-accessed site you‘ll have some requests per day. [1] https://aws.amazon.com/route53/pricing/?nc1=h_ls https://aws.amazon.com/route53/pricing/?nc1=h_ls

bombcar 6y ago

It's arguably even MORE important in the age of fast links - because now a great percentage of the clock time is spent on handshakes and other "overhead" - so saving a round trip or three on a DNS lookup is comparatively larger than it used to be.

dijit 6y ago

Argh, I hate this kind of question, it overtly implies that because we have more of something (computing resources, memory, networking bandwidth) we should fill the void with something "better", where "better" means different things to different people. DNS doesn't really work this way anyway, you have a tree-like structure and if you're a good little resolver you don't just talk to your upstream DNS server, you walk the path from the root to the stem. I.E; you don't ask your ISPs resolver for server-a.www.google.com you ask the DNS root (".") for who owns com, then you ask "com" who owns google, and you keep going down the NS records you're on the stem, then you ask for an A or a CNAME record.. Then you cache that result _because it's expensive to do that lookup_- and often (ESPECIALLY) on the internet a webpage will ask you to download a dozen more things from the same domain, so a cache makes absolute sense. But, regardless of how expensive DNS is; you miss a _huge_ point about the tech that came before: it's foundational and fundamental to how things work, it's so "light" that we bake it into our products, and increasing the "cost" causes an exponential cost to end users because it's so widely adopted and essential. This industry seems to want to eat itself, taking things that work decently well and noodling on them until something much more costly comes along, why on earth would we take DNS before tackling the much more necessary project of fixing email?

Daho0n 6y ago
Which resolvers actually follow that path?
- tinus_hn 6y ago
  All resolvers either follow that path or ask someone else to follow that path. It’s the only way to get the answer.
QuadrupleA 6y ago
I don't know how suggesting tighter TTLs is indicative of "the industry eating itself" or that we can't fix email at the same time. Relax. "com" and "." can obviously be cached a long time, they'll rarely change servers. No need to walk the whole tree every time - god forbid between multiple requests made by a single webpage like you're implying. Subdomains are typically only 1 or 2 names on top of the TLD. I'm all for being efficient with computer resources, and like others said there's a latency component here. But seriously, DNS packets are tiny, typically 50 bytes or so. Assuming, say, 80 bytes after UDP, IP, Ethernet framing etc. (just a ballpark) that's 640/10,000,000,000 bits in 1 second of time on a 10gbps fiber, or about 1 of 1.5 million DNS packets that fiber (of many fibers) could send every second. At that scale, I don't see why TTLs of say 1 minute wouldn't be fine. Multiply everything above by 60 seconds and that's still an effective cache of a 50 byte response in computer time. Authoritative NS records can change less often, so intermediate servers can just check direct with the authoritative servers when records go stale. Say 100 bytes roundtrip once a minute. I'm not taking about throwing away or redesigning DNS - just that tighter TTLs than typically recommended are probably fine and would make life a little easier with propagation delays.
- carlhjerpe 6y ago
  Cloudflares automatic TTL is 5 minutes. Web browsers will look up your records and try to connect to one of the returned ips, if that one doesn't work it'll use another one. The fact that the client does this for us means we can keep the TTL quite high (5 minutes) without 5 minutes downtime if we lose connectivity to one IP. This is also used to load balance, a single load balancer can't handle a trillion connections, so we round robin DNS for load balancing before the load balancers. There's also anycasting which is something cloudflare uses a lot. Which means you're announcing the same IP block from many locations, and with how BGP works (which is the defacto protocol for exchanging route information on the internet) the "closer" you are the more likely the route is to be chosen (AS path distance). Meaning sites behind Cloudflare is protected by thousands of servers listening to the two same addresses you see if you're behind a Cloudflare protected site and behind these thousands of servers are even more thousands of servers. This means if i were to launch a 10Gbps DoS from my server, it would hit the Cloudflare datacenter closest to me, and there's nothing i can do about it, meaning the rest of the world is unaffected. No, I'm not afiliated with CF. It's just an approachable example, i believe (but don't know) that Fastly has the same featureset (that I've mentioned above)

mjevans 6y ago

Caching this lookup helps to reduce the __latency__ at the client end. A local, often in RAM, DNS cache should honor the TTLs (for validity expiration) if the entries aren't purged out by other activity.

chrisandchris 6y ago

Assume you‘ll use Amazon Route53 as your DNS provider. You‘ll pay per lookup after some free-tier so doubling your TTL from 5min to 10min halfes your cost [1] . Sure, it‘s ~$0.40 / million requests but if you have a TTL of a few minutes and a often-accessed site you‘ll have some requests per day. [1] https://aws.amazon.com/route53/pricing/?nc1=h_ls https://aws.amazon.com/route53/pricing/?nc1=h_ls