3 ms·
So you're saying with the 3 step rollout if rollout 3/3 goes bad you should rollback straight to 1/3? I guess that does make a bit of sense in the case of a dat
by Thorrez 7y ago
So you're saying with the 3 step rollout if rollout 3/3 goes bad you should rollback straight to 1/3? I guess that does make a bit of sense in the case of a database where the newly created values can stick around.
But also there could have been a bug in 1/3 that wasn't exposed until step 3/3. Meaning even rolling back to 1/3 won't solve the problem.
The 2 step rollout is safe if we assume there are no bugs in the code, but problematic if there are bugs. But the 3 step rollout is also problematic if there are bugs. I guess the 3 step rollout has the advantage that it partitions off a section that might be buggier than other sections so that it can be individually disabled. But that'll only help sometimes, and I'm not sure if the additional complexity is worth it.
Flag flip vs code push doesn't seem to make a difference to me. All 3 rollouts could be flag flips enabling code that was written much earlier but hidden behind flags.
Another problem is doing a double rollback like that seems a bit risky to me, because other features that were being deployed simultaneously might not have been designed to handle a double rollback, only a single rollback, so they could break. If we want to allow double rollbacks, we must require that all development not just handle single rollbacks smoothly but also double rollbacks.
- dillonmckay 7y agoYou shouldn’t be deploying other features simultaneously.