3 ms·
You mean Cloudflare can't at all blame Flexential for: 1) not informing CF of unplanned maintenance on electrical 2) not informing CF that they had failed ove
by auspiv 3y ago
You mean Cloudflare can't at all blame Flexential for:
1) not informing CF of unplanned maintenance on electrical
2) not informing CF that they had failed over to generator due to a ground fault on their PGE transformer
3) not informing CF that ALL their generators failed and were on backup batteries
4) not informing CF that power had completely failed within the DC
5) notifying CF that anything at all had happened for 48 minutes (which was after all the UPS batteries had died)
Yes, this is a one sided analysis. I haven't seen the Flexential one (is there one?).
https://blog.cloudflare.com/post-mortem-on-cloudflare-control-plane-and-analytics-outage/ https://blog.cloudflare.com/post-mortem-on-cloudflare-contro...
- greyface- 3y agoIt's buck-passing; the responsibility to keep Cloudflare services available to users lies with Cloudflare, not Flexential. A well-designed distributed system is able to withstand an unplanned DC failure without the user noticing. They can certainly note that Flexential's failure was a causal factor in their outage, but if they do, they're also admitting to inadequate engineering practices.
- kosolam 3y agoYep. Basically what I meant as well. Thanks for clarifying it even more.
- crote 3y agoFlexential can absolutely be blamed on all of that, and if I were their customer I'd be massively pissed off too. However, CF massively failed at the whole "entire DC goes down" mitigation thing. This time it was Flexential screwing up power, but it could've also been a fire, a flood, a tornado, an earthquake... Once you get to CFs scale, you simply cannot hide behind your vendors anymore. CF has gotten big enough that we can fully expect them to just deal with it.