4 ms·
People seem to be focusing on the fact that the two operations should have been linked, but I think they actually should have been further separated. The old i
by ryporter 11y ago
People seem to be focusing on the fact that the two operations should have been linked, but I think they actually should have been further separated. The old index presumably could have been left around for days, at the cost of some performance, and only deleted once they were sure that the new index was successfully being used by all production code.
While splitting these two changes may be silly in the case of a simple index change, I think that it's a good general policy to only deploy the minimal set of changes to a production database at once. On the production system I manage (admittedly, much simpler, and many orders of magnitude smaller), I always deploy changes in two stages, with the second stage generally deployed about a week later, once I'm sure that only new code will be running against the database.
This comment is not meant to second guess the Stripe developers (who produced a great postmortem), but to suggest another possible remediation.
- mst 11y agoI had a similar thought but if it's that critical an index it's entirely possible they couldn't afford the write overhead to keep both copies around.