3 ms·
I think the correct way to view this is to try to find the points of maximum leverage. The root cause is always the Big Bang. We already know that, so it's kind
by asdfasgasdgasdg 7y ago
I think the correct way to view this is to try to find the points of maximum leverage. The root cause is always the Big Bang. We already know that, so it's kind of pointless to look for it. What you're interested in is what can be changed.
These will be different depending on your perspective/level of power. For a SWE, it will be something like, "If we had monitoring of this variable, we would have gotten notified faster." Or, "If we had this type of testing, we'd prevent this class of errors." For a VP, it will look more like, "If our incentive structure were changed this way, or if our dev infra team built this kind of tooling, my SWEs would be better enabled to prevent this outage." Often there will be more than one point of leverage, but you only have so much time in your life, so the question is which one is most important or likely to be effective.
Examples of points of non-leverage are, "Joe submitted this busted CL." That is in the past and is not changeable. If Joe is in a habit of submitting busted CLs, he might need counseling, a PIP, or failing all else, to be fired. However, typically the submission of broken CLs is a systemic problem, not a case of a single sub-par engineer. You have to find the leverage that will allow you to change the system.
- contingencies 7y agoAgreed, after identifying contributing factors or manageable interfaces process-oriented fixes are most powerful. That's why CI/CD is so awesome, RCS/VCS pre-commit and post-commit hooks allow you to automate the checklists you need to solve entire categories of issues.