3 ms·

As always, it's really impressive to see how much technical detail they release publicly in their RCAs. It sets a good example for the industry. Also — quite i

• Waterluvian 3y ago
  I feel there’s a sweet spot where if you do it too quickly, it’s a bad sign. And if it takes years, risk just keeps going up and up until it becomes basically impossible to do smoothly.
• zamalek 3y ago
  > quite impressive to make major infrastructure and architecture changes in a few months And have it work first time round.
• belter 3y ago
  There is not much impressive here. Their architecture seems to be relying on a couple of datacenters, and if somebody have turned on or not the right switches in the right places. This means you will continue to hear from regular outages and maybe nice postmortem blogs. I don't see a fundamental approach to reliability based on concepts around blast radius, expecting anything to fail anytime, and fundamental principles of bulkhead patterns. Here is a free hint: By talking so much about where their data centers are located, on my view, they already failed item one on my check list. Principle number one of Physical Security is, you don't say where your Data Centers are, except of course to very restricted number of "need to know group". As predictions have no value, unless they are made prior to events, I predict the next outage will be some common core component, with some on/off type of config, with some common core configuration, that "could not be foreseen". Then to the next blog...On to the next outage....
• mleonhard 3y ago
  The blog post contain no details at all about how they achieved high availability. I'm disappointed.