5 ms·
It's less vague than you realize. It points out that the problem was within Facebook's network between its datacenters. This not only suggests it's related to e
by yuliyp 5y ago
It's less vague than you realize. It points out that the problem was within Facebook's network between its datacenters. This not only suggests it's related to express backbone, but also suggests that the DNS BGP withdrawal which Cloudflare observed was not the primary issue.
It's not a full root cause analysis, to be sure, and leaves many open questions, but I definitely wouldn't describe it as painfully vague.
- EricE 5y agoA point of distinction - there is no "DNS BGP withdrawl". DNS is related to BGP only that without the right BGP routes in the routers, no packets can get to the facebook networks and thus the facbook DNS servers. That their DNS servers were taken out was a side affect of the root issue - they withdrew all the routes to their networks from the rest of the Internet. Not picking on you - but there has been a lot of confusion around DNS that is mostly a red herring and people should just drop it from the conversation. Everything on facebook networks disappeared, not just DNS. The main issue is they effectively took a pair of scissors to every one of their internet connections - d'oh!
- yuliyp 5y agoFB did not totally disconnect from the internet as far as what I observed. They broke their backbone (from the blog post, as well as seeing that their edge was returning 503s during the incident), and FB withdrew specific BGP announcements which covered their DNS servers are what I saw. I think this is consistent with what Cloudflare actually said, although not what has been often repeated since then.
- matsur 5y agoThis is not what we observed. Less specific routes were still present in routing tables, but the more specifics that routed DNS traffic disappeared.