3 ms·
A release checklist (for us) is inherently very risk-averse, as its raison d'être is to prevent a very expensive & dire catastrophe, and every item on the list
by ip26 8y ago
A release checklist (for us) is inherently very risk-averse, as its raison d'être is to prevent a very expensive & dire catastrophe, and every item on the list is (metaphorically) "written in blood" [0]. As a result it's politically intractable to remove anything.
[0]: https://medium.com/war-is-boring/u-s-navy-supercarriers-dominate-because-of-lessons-paid-for-with-blood-cb26f5509ac https://medium.com/war-is-boring/u-s-navy-supercarriers-domi...
- TomMckenny 8y agoAgreed. But items can move to other checklists. In this case the list item should become a unit test for the signing code since a unit test is essentially a check list for automated systems.
- grigjd3 8y agoWhere do you work, so that I know to avoid it? Documentation needs updating as things change.
- d1str0 8y agoSo the issue isn’t with checklists, it’s with policy.
- falsedan 8y agopolicy is just a way to record a decision-making process. Yet he issue is that getting the right information to be reasonably sure it’s safe to remove a checklist item is hard and made harder the longer ago the item was added
- dv_dt 8y agoI would say that a system too entrenched to consider removing anything, is fundamentally at risk of not being able to correctly analyze the system when adding anything to the checklist, and is itself at an increased risk to fail.
- bluGill 8y agoThe problem is it is easy to look an item and say "we no longer need this, remove it". It is much harder to be sure you are fixed all possible ways the problem could have happened. Maybe the system auto-signs the release - but does it in all cases? Is there a rare special case where the automatic process doesn't happen? Do you really know what tricks they have to pull in that branch office in India?
- dv_dt 8y agoThat's why the same consideration to adding an item needs to be put into removing an item, need to be put into reviewing if the checklist is relevant still at all. Broken down a different way, you need two systems: an automatic system so you're not reconsidering decisions that have good outcome (a checklist), and a considerate system that decides what goes into or out of the automatic system. If the considerate system is broken or just frozen, then you have a problem. How long it will take to manifest is a different question.
- eslaught 8y agoI think there's an asymmetry that you're not quite capturing here: when you add an item to the checklist, you have a specific concrete failure at hand (the process just failed, you had everyone on hand to do your root cause analysis, and you know exactly why). Later when you want to remove it, you (a) lose context because time has passed and memories are fallible and/or a different person is dealing with it, and (b) you have to have some reasonable assurance that you've covered other ways the same failure can occur. No amount of process can completely eliminate this asymmetry so at some point you're forced to make a tradeoff based on how risk averse you want to be.
- dv_dt 8y agoI can definitely concede there can be some asymmetry, but I think it's system dependent. As the Boeing 737 Max 8 shows, adding new items (in this case a new design element) is also fraught with risks. You have to get the root analysis correct when adding or removing elements. Adding has risks of unknown (not accounted for) like removing has risks of unknown (accounting forgotten). In the end, I guess I still believe the best strength is in good analysis at the consideration stage.