3 ms·
The faster you roll back, the less complications there are to deal with. Just think about all the damage that's accumulating if you've got millions of users hit
by svachalek 10y ago
The faster you roll back, the less complications there are to deal with. Just think about all the damage that's accumulating if you've got millions of users hitting the site while you're figuring out a way forward. So, as they say, you roll stuff out very slowly and roll it back quickly. Turning off feature flags as the article suggests is IMO just a different implementation of rolling back. Now you run the old code again and have all the same risks as an actual rollback (just more self-contained).
Edit: note that you also have to take into account how long it takes to notice the problem. If you're rolling out and notice something bad when you hit 1% or 10% of users, it's one thing, but if you hit 100% rollout and notice bad things happening a week later it's a different situation. For one thing, hopefully this indicates the problem is not spectacularly fatal, and for another you've already got users used to the new behaviors and who knows how much accumulated damage. So then it's worthwhile to step back a bit and figure out how to recover.