3 ms·
I suspect we're going to agree to disagree here, but I highly advise you to re-consider the idea of framing a roll back as an unforced failure to your team. The
by gfodor 7y ago
I suspect we're going to agree to disagree here, but I highly advise you to re-consider the idea of framing a roll back as an unforced failure to your team. The last dynamic you want in a retrospective is one where not only did an unexpected failure happen (a bug pushed to production), but then the team collectively 'let you down' by pulling the rollback lever, instead of thinking and working harder on fixing the issue live. In such a scenario you're forcing people to feel they need to "cry uncle" when they can't solve the problem quickly, and putting themselves into the middle of a conflict of interest between making a well-tested, reviewed change that is sure to fix the problem, and rolling the dice on a quick fix in the hope it'll reduce the total outage. That's not the recipe for a positive, blameless culture.
When we roll back on my team, it's uncommon but when it happens it's considered a success if it was made through a systematic decision-making process. Making a sane decision in the interest of our users to restore service quickly is always a win. I can assure you, it does not compromise your ability to do continuous delivery or small changes by having and occasionally using a rollback mechanism. If you are fearful of the idea that having such a mechanism and plan in place somehow will lead to people questioning your principles in a way you cannot defend, then that is a separate problem, since the two things you mention that are incompatible are in fact compatible and highly defensible.
It is not a legacy from "waterfall" or any of the other things you mention, because your claim can be refuted through a single counter example, and I've worked on 3 separate projects where such counter examples exist: we had a rollback method, it was used once in a while, and we shipped changes to production multiple times a day using continuous delivery. At no point on these projects did the ability or use of roll back lead to some kind of hard-to-explain loss in delivery velocity. On the contrary, I suspect if that mechanism did not exist, several failures that were easy to get back to green would have turned into a toxic hellhole, and my team mates would have been much more fearful around shipping, which is the high order bit when it comes to velocity and embracing continuous delivery of small changes.
- rinchik 7y ago"toxic hellhole", "blame culture" - I don't think we need those dark, marginal extremes to make a point I also suspect that we're going to agree to disagree here. There are so many nuances, it's impossible to properly communicate most of those without writing a chapter of a book. Appreciate your points though. Great food for thought right there.