4 ms·
Except that his way all the traffic instantly switches, and your way you have to wait for DNS propagation, which about 15% of the users on the internet will not
by jedberg 13y ago
Except that his way all the traffic instantly switches, and your way you have to wait for DNS propagation, which about 15% of the users on the internet will not pick up for over a week.
DNS is an awful way to do failover.
- colmmacc 13y agoTo your point ; it's ok to do both. Route 53 supports DNS TTLs as low as 0 seconds. ELB and S3 endpoints both have 60 second TTLs. My experience with flipping names like www.amazon.com doesn't reflect the 15% figure. I've seen about 97% of web traffic honouring the TTL and flipping quickly. Within 5 minutes almost all of the rest too. We also take CloudFront sites in and out of service for maintenance, and in 5 years I've never anything like a 15% straggler effect. That said, we do see a very small number of stragglers. While resolvers over-riding TTLs hasn't shown up as a significant problem, buggy clients can be; we come across clients now and then who either don't re-resolve ever (Various JVMs and their infinite caches are a common cause), or only re-resolve on failures (which is fine for failover, but not great for traffic management). If you have a distribution time plot for the 15% figure it'd be interesting to see; https://www.dns-oarc.net/ https://www.dns-oarc.net/ would be a good venue, https://lists.dns-oarc.net/mailman/listinfo/dns-operations https://lists.dns-oarc.net/mailman/listinfo/dns-operations is the open list. Ignoring TTLs for a week is very concerning; it would very likely break many DNSSEC configurations. Is it possible you were dealing with robots?
- btilly 13y agoI ran across the "DNS broken for a week", and it was broken for ISPs that run their own DNS for home customers. They didn't want to keep asking for upstream DNS, so just ignored the ttl. But I ran across this around the year 2000. And the biggest offender was named AOL. I could well believe that it is very, very different today.
- ensignavenger 13y agoI don't know that I have ever seen a straggling DNS update take a week, but I have seen many take 3-4 days. Particularly Mediacom customers. I think this situation has improved some, but I still would not consider DNS failover very reliable.
- jedberg 13y agoWhen we flip Netflix domains we see about a 15% straggler effect (although to be fair only about 3% take a week, but many take around 24 hours).
- revertts 13y agoHow much of that 15% is driven by ISPs versus misbehaving clients? (eg. set to boxes)
- hboon 13y agoIn your experience, are there still major ISPs that don't respect DNS TTL?
- jedberg 13y agoYes, about 15%. :) To be fair most flip within 24 hours, but about 3% or so take longer than a week. Typically rural ISPs or Alaskan ISPs (they don't want to pay to forward all the requests so they override the TTLs). This is when we flip Netflix IPs.