As always I'm glad to see Cloudflare post such detailed outage reports. They are one of the few providers I know of that is willing to go into such depth and th

As always I'm glad to see Cloudflare post such detailed outage reports. They are one of the few providers I know of that is willing to go into such depth and that is one of the things I appreciate about them. That said, the outage that occurred was one that was indeed fully preventable. We don't exactly have as many locations as they do, but for internal resources at least, not pushing configuration changes to all devices (network included) is pretty standard practice. Basically I imagine for them a good routine to follow might be to script changes so that they are 'rolled out', something along the lines of push manual changes to a scripted 'random' router set (one in country A,B,C), wait 15 minutes and then push to the remaining router sets. That wouldn't work for all situations, such as if the entire network is seeing a DDoS or what have you, but I imagine they could adapt a routine that would prevent this particular scenario. With all of that said, as a Cloudflare customer and also having a call with them tomorrow scheduled already over the WAF stuff, I find it a bit... frustrating that this is occurring now and such a kind of mistake. Edit: As an aside, I wonder if the Puppet module for Junos will be extended to support route statements. That would make this kind of deployment much easier.