I should mention that promoting a secondary to a primary in an asynchronously replicated system can (and almost certainly will, under load) result in the loss o

I should mention that promoting a secondary to a primary in an asynchronously replicated system can (and almost certainly will, under load) result in the loss of committed transactions. Redis has two resynchronization modes. One just recovers a small delta from the primary, if it hasn't diverged that far. If the primary has accepted too many writes, the secondary has to initiate a full resync, which is much more expensive. Since Twilio's postmortem says their nodes initiate full resyncs, I suspect that had Twilio promoted a secondary, they would have lost a significant number of writes. Probably best, under these kinds of scenarios, to design the system such that lost writes don't cause overbilling. ;-)