7 ms·

The problem isn't (wasn't) facebook.com's A records, it's that the authoritative nameservers for facebook.com are (were) unreachable. In theory someone could c

by jbotz 5y ago

The problem isn't (wasn't) facebook.com's A records, it's that the authoritative nameservers for facebook.com are (were) unreachable. In theory someone could change the NS records for facebook on the .com nameservers to point somewhere else and serve up a fake facebook.com domain, but... 1) those NS records have a 6 hour TTL, so it would take a while be effective, and 2) who has the authority to do that?

junon 5y ago
I would assume DNS providers were on call with FB staff to manually trigger flushes across the system to prop the records ASAP. Of course this doesn't affect client TTLs but resolution failures generally aren't cached anyway.
- yourad_io 5y ago
  Speaking of flushing, today I learned that Google public DNS has an open "flush" function: https://dns.google.com/cache https://dns.google.com/cache
  - celsoazevedo 5y ago
    Cloudflare: https://1.1.1.1/purge-cache/ https://1.1.1.1/purge-cache/ OpenDNS: https://cachecheck.opendns.com/ https://cachecheck.opendns.com/
- hamilyon2 5y ago
  Dns providers are basically every ISP in existence, it would be impractical to communicate with by phone calls
DaiPlusPlus 5y ago
> those NS records have a 6 hour TTL Why on earth would any system need a TTL beyond 1 hour? The longer the TTL the longer your outage period will be if a cache is poisoned.
- Thorrez 5y ago
  Could something with a normal 1 hour TTL be poisoned by a cache entry with a 30 day TTL?
- fomine3 5y ago
  Shorter TTL increases poisoning risk. Anyway attacker can set arbitrary TTL if they succeed poisoning so longer TTL won't increase risk.
- jbotz 5y ago
  Heh. I remember the day when a TTL of a week was pretty standard. Anyway, the since the NS records for a domain come from the next level up, i.e. in this case the .com nameserver which are also the root nameservers, if they had a short TTL then those root nameservers would get hit pretty hard. Also, it isn't your choice what the TTL on your NS records is, it's admins of your top-level domains.
- mcspiff 5y ago
  On the flip side, it’s the longer you can survive an outage by relying on enough clients having the cached records. So can go both ways a bit.
- throwaway09223 5y ago
  "Why on earth would any system need a TTL beyond 1 hour?" Load. Lower TTLs mean shorter term caches and more traffic. This is a big deal, especially for root servers -- which is why the TTL is not customizable.
- toast0 5y ago
  .com (used for Facebook authoritative DNS) and .net (used for WhatsApp) servers serve glue records with a 2 day TTL, and domain holders don't get to choose TTL. (the root servers serve glue records for the TLDs with a 2 day TTL as well). So, two days is the TTL you get for your NS records. Of course, it takes way more than two days for people to stop querying your old records. I gave up after 30-45 days.
dotancohen 5y ago
> The problem isn't (wasn't) facebook.com's A records, it's that the > authoritative nameservers for facebook.com are (were) unreachable. The problem isn't (wasn't) DNS, it's that a large number of websites rely on a single point of failure in order to function properly.

by jbotz 5y ago

junon 5y ago

I would assume DNS providers were on call with FB staff to manually trigger flushes across the system to prop the records ASAP. Of course this doesn't affect client TTLs but resolution failures generally aren't cached anyway.

yourad_io 5y ago
Speaking of flushing, today I learned that Google public DNS has an open "flush" function: https://dns.google.com/cache https://dns.google.com/cache
- celsoazevedo 5y ago
  Cloudflare: https://1.1.1.1/purge-cache/ https://1.1.1.1/purge-cache/ OpenDNS: https://cachecheck.opendns.com/ https://cachecheck.opendns.com/
hamilyon2 5y ago
Dns providers are basically every ISP in existence, it would be impractical to communicate with by phone calls

DaiPlusPlus 5y ago

> those NS records have a 6 hour TTL Why on earth would any system need a TTL beyond 1 hour? The longer the TTL the longer your outage period will be if a cache is poisoned.

Thorrez 5y ago
Could something with a normal 1 hour TTL be poisoned by a cache entry with a 30 day TTL?
fomine3 5y ago
Shorter TTL increases poisoning risk. Anyway attacker can set arbitrary TTL if they succeed poisoning so longer TTL won't increase risk.
jbotz 5y ago
Heh. I remember the day when a TTL of a week was pretty standard. Anyway, the since the NS records for a domain come from the next level up, i.e. in this case the .com nameserver which are also the root nameservers, if they had a short TTL then those root nameservers would get hit pretty hard. Also, it isn't your choice what the TTL on your NS records is, it's admins of your top-level domains.
mcspiff 5y ago
On the flip side, it’s the longer you can survive an outage by relying on enough clients having the cached records. So can go both ways a bit.
throwaway09223 5y ago
"Why on earth would any system need a TTL beyond 1 hour?" Load. Lower TTLs mean shorter term caches and more traffic. This is a big deal, especially for root servers -- which is why the TTL is not customizable.
toast0 5y ago
.com (used for Facebook authoritative DNS) and .net (used for WhatsApp) servers serve glue records with a 2 day TTL, and domain holders don't get to choose TTL. (the root servers serve glue records for the TLDs with a 2 day TTL as well). So, two days is the TTL you get for your NS records. Of course, it takes way more than two days for people to stop querying your old records. I gave up after 30-45 days.

dotancohen 5y ago

> The problem isn't (wasn't) facebook.com's A records, it's that the > authoritative nameservers for facebook.com are (were) unreachable. The problem isn't (wasn't) DNS, it's that a large number of websites rely on a single point of failure in order to function properly.