> 87 minutes to revert a change? If their processes require a suite of tests be run on staging systems before the revert hits production, that may take conside

> 87 minutes to revert a change? If their processes require a suite of tests be run on staging systems before the revert hits production, that may take considerable time depending on how much is potentially affected. Also the 87 minutes might be for everything, perhaps it was a rolling update: update some nodes, test, update the next batch, repeat. This might mean that a good chunk of the infrastructure was updated for sooner and traffic could be diverted through those parts (presumably those nodes are over-specified, or just easily scaled, to be able to take a glut of extra load - as they would need to in more normal circumstances in order to contend with waxing & waning traffic patterns). Perhaps it is even the case that the errant update didn't get out to all areas before the problem became apparent, so they were not rolling back over the whole network. Again this may offer the opportunity to divert load to non-affected areas while the others are rolled back. So while the full revert took 87 minutes, things may have been up and running properly (except perhaps a little slower than usual) for most users/processes much quicker than that.