4 ms·

> some resolvers ignore TTLs Is that really true? I have heard this so often but never seen it in reality, so I tend to take this as an urban legend.

by micw 5y ago

lilyball 5y ago
AIUI some resolvers will set a lower bound on TTLs.
- micw 5y ago
  But what are "some" resolvers? I never saw one of those commonly in use.
Anthony-G 5y ago
I also haven’t come across this while working as a sysadmin for the past 10 years but I’ve read that broken resolvers (ignoring the TTL) provided by ISPs were more common back in the 90s.
jedberg 5y ago
Absolutely. My friend worked for an ISP in Alaska. They set their TTL to 48 hours for all records regardless of what the DNS said, because they didn't have the bandwidth to pull updates more often than that. Also, when I worked at reddit I had to change the IP for reddit.com in 2007. I set up a cache on the old IP to redirect any old traffic. Also, a week before the switch I had pulled the TTL down to 5 seconds. After I made the change, only 40% of the traffic shifted after a minute. That means at least 60% of the people on the internet were ignoring my 5 second TTL if not more. After a day 10% of the traffic was still going to the old IP. It took more than 48 hours for 99% of the traffic to switch. And after a month I still had about 20 hosts hitting the old IP (probably scripts that were hard coded with the old IP). At Netflix we had similar issues with lots of stragglers when we changed IPs. So yeah, a lot of resolvers ignore TTL.
- micw 5y ago
  I wonder what kind of ISP it was with to few bandwidth for proper DNS ^^ And I'm really curious which other ISPs are running such setups that you had to deal with and for what reason they do so.
  - jedberg 5y ago
    A small ISP in the early 2000's. :) I don't really know the cause of the other problems, but I'm guessing it was mostly browser and OS caching and not ISP caching.
- watermelon0 5y ago
  Don't really know how accurate this is, but I remember reading about some resolvers ignoring too low TTLs.
  - jedberg 5y ago
    It's quite possible that the 5 second TTL was a foot-gun. It was best practice at the time but not necessarily the right thing to do!
micw 5y ago
I was searching for some (no-anecdotal) evidence and found this study: https://labs.ripe.net/author/giovane_moura/dns-ttl-violations-in-the-wild-measured-with-ripe-atlas/ https://labs.ripe.net/author/giovane_moura/dns-ttl-violation... Seems that increased TTL in resolvers is still a thing in 2021 but it's a bit over-estimated. IMO it should not be considered when changing DNS. May their customers complain about the broken DNS every time a change is not "propagated" (sic!) there.
fragmede 5y ago
This is easy to see if you've got decent visibility into your system. Have your DNS TTL already set to 300 seconds (the minimum). Switch your A records over to a new load balancer. It'll take far longer than 5 minutes for traffic to die down, and if your site is popular enough, traffic to your old load balancer will never fully die down.

> some resolvers ignore TTLs Is that really true? I have heard this so often but never seen it in reality, so I tend to take this as an urban legend.

by micw 5y ago

> some resolvers ignore TTLs Is that really true? I have heard this so often but never seen it in reality, so I tend to take this as an urban legend.

lilyball 5y ago

AIUI some resolvers will set a lower bound on TTLs.

micw 5y ago
But what are "some" resolvers? I never saw one of those commonly in use.

Anthony-G 5y ago

I also haven’t come across this while working as a sysadmin for the past 10 years but I’ve read that broken resolvers (ignoring the TTL) provided by ISPs were more common back in the 90s.

jedberg 5y ago

Absolutely. My friend worked for an ISP in Alaska. They set their TTL to 48 hours for all records regardless of what the DNS said, because they didn't have the bandwidth to pull updates more often than that. Also, when I worked at reddit I had to change the IP for reddit.com in 2007. I set up a cache on the old IP to redirect any old traffic. Also, a week before the switch I had pulled the TTL down to 5 seconds. After I made the change, only 40% of the traffic shifted after a minute. That means at least 60% of the people on the internet were ignoring my 5 second TTL if not more. After a day 10% of the traffic was still going to the old IP. It took more than 48 hours for 99% of the traffic to switch. And after a month I still had about 20 hosts hitting the old IP (probably scripts that were hard coded with the old IP). At Netflix we had similar issues with lots of stragglers when we changed IPs. So yeah, a lot of resolvers ignore TTL.

micw 5y ago
I wonder what kind of ISP it was with to few bandwidth for proper DNS ^^ And I'm really curious which other ISPs are running such setups that you had to deal with and for what reason they do so.
- jedberg 5y ago
  A small ISP in the early 2000's. :) I don't really know the cause of the other problems, but I'm guessing it was mostly browser and OS caching and not ISP caching.
watermelon0 5y ago
Don't really know how accurate this is, but I remember reading about some resolvers ignoring too low TTLs.
- jedberg 5y ago
  It's quite possible that the 5 second TTL was a foot-gun. It was best practice at the time but not necessarily the right thing to do!

micw 5y ago

I was searching for some (no-anecdotal) evidence and found this study: https://labs.ripe.net/author/giovane_moura/dns-ttl-violations-in-the-wild-measured-with-ripe-atlas/ https://labs.ripe.net/author/giovane_moura/dns-ttl-violation... Seems that increased TTL in resolvers is still a thing in 2021 but it's a bit over-estimated. IMO it should not be considered when changing DNS. May their customers complain about the broken DNS every time a change is not "propagated" (sic!) there.

fragmede 5y ago

This is easy to see if you've got decent visibility into your system. Have your DNS TTL already set to 300 seconds (the minimum). Switch your A records over to a new load balancer. It'll take far longer than 5 minutes for traffic to die down, and if your site is popular enough, traffic to your old load balancer will never fully die down.