6 ms·

> no-blame culture We have this where I work and I have mixed feelings about it. On one hand, it's certainly freeing that we don't get fired for mistakes. On

> no-blame culture We have this where I work and I have mixed feelings about it. On one hand, it's certainly freeing that we don't get fired for mistakes. On another, it's strange to feel that the people who objectively caused the headache for everyone else go without serious attention to remediation. IMO, we don't have to name names, but Root Cause Analysis should include the decision tree that the primary motivator chose. No blame culture also hampers a lot of insider-threat analysis.

robalfonso 4y ago
No-blame culture and no-consequence culture are not the same thing. Just because you don’t get fired doesn’t mean the org should allow someone the capability to make the same mistake, whether that’s access/permissions/change in org etc
Sebb767 4y ago
I think the idea behind no-blame culture is that a single person shouldn't be able to cause a massive incident on their own (assuming no malice), so finding a victim to put the blame on isn't helpful anyway. Also, if you start blaming people, everyone will go in permanent cover-my-ass-mode and this will usually just lead to some poor junior, who couldn't play the game yet, getting the full blow. Of course, letting people run rampant and acting everything is fine is not great, but there's a good chance it might be an organizational problem and, if not, a HR problem (which is not necessarily related to a specific incident).
saghm 4y ago
I think the idea behind no-blame is that there are basically three ways that an outage could be the fault of an individual: maliciousness, incompetence, or bad process. In the first two cases, the individual should be fired, and in the third case, blaming the individual is counterproductive as it not only doesn't do anything to mitigate the issue but also distracts from the real cause. The only gray area I can think of is where it's not not clear from a single incident whether an individual is actually incompetent or if they're "borderline competent" (maybe more competent at other aspects of the job role and it's a balance to figure out if they're net value is worth keeping around), but in that case, I think that's also probably better to be dealt with at the management level and keep out of the root cause analysis. > MO, we don't have to name names, but Root Cause Analysis should include the decision tree that the primary motivator chose. I definitely agree with this! I actually have seen cases where people have asked for revisions to a root cause analysis not due to a lack of naming individuals but a lack of sufficient explanation for what the anonymous individual actually did to cause the outage, which I think is fair feedback to give. > No blame culture also hampers a lot of insider-threat analysis. Yeah, if no-blame culture is being used as a shield for maliciousness or incompetence, I think that could have a harmful effect on things. I guess I always just interpreted "no-blame" as actually meaning "no blame for honest mistakes", but I guess that's too long to be catchy.
- chx 4y ago
  If incompetent people got in the position where they cause a production outage whose fault is that? Sure they should be moved / privileges stripped -- consequences are fine -- but once again, in my opinion, aside from maliciousness an outage is always an organizational mistake. I have twenty years of experience in the framework we work with and I can't roll a production hotfix without QA approval. What this comes down to is: if prod goes down, I should be able to focus on getting it back up and not worrying whether I have a job tomorrow. That's not conducive to success.
throwawaysleep 4y ago
Except that no one person even enormously missing something should be able to cause a major error.
aledalgrande 4y ago
no blame works without exceptions when the team makes it impossible to screw up big (test coverage, multiple reviews, modularity, canaries, DR strategy etc.) if you do retros and the same thing keeps coming up and it's always coming from the same person, then the manager will have to solve that
kache_ 4y ago
If you can take down the whole network, it isn't a single person's fuck up. Its the whole organization's fuck up.

robalfonso 4y ago

No-blame culture and no-consequence culture are not the same thing. Just because you don’t get fired doesn’t mean the org should allow someone the capability to make the same mistake, whether that’s access/permissions/change in org etc

Sebb767 4y ago

I think the idea behind no-blame culture is that a single person shouldn't be able to cause a massive incident on their own (assuming no malice), so finding a victim to put the blame on isn't helpful anyway. Also, if you start blaming people, everyone will go in permanent cover-my-ass-mode and this will usually just lead to some poor junior, who couldn't play the game yet, getting the full blow. Of course, letting people run rampant and acting everything is fine is not great, but there's a good chance it might be an organizational problem and, if not, a HR problem (which is not necessarily related to a specific incident).

saghm 4y ago

I think the idea behind no-blame is that there are basically three ways that an outage could be the fault of an individual: maliciousness, incompetence, or bad process. In the first two cases, the individual should be fired, and in the third case, blaming the individual is counterproductive as it not only doesn't do anything to mitigate the issue but also distracts from the real cause. The only gray area I can think of is where it's not not clear from a single incident whether an individual is actually incompetent or if they're "borderline competent" (maybe more competent at other aspects of the job role and it's a balance to figure out if they're net value is worth keeping around), but in that case, I think that's also probably better to be dealt with at the management level and keep out of the root cause analysis. > MO, we don't have to name names, but Root Cause Analysis should include the decision tree that the primary motivator chose. I definitely agree with this! I actually have seen cases where people have asked for revisions to a root cause analysis not due to a lack of naming individuals but a lack of sufficient explanation for what the anonymous individual actually did to cause the outage, which I think is fair feedback to give. > No blame culture also hampers a lot of insider-threat analysis. Yeah, if no-blame culture is being used as a shield for maliciousness or incompetence, I think that could have a harmful effect on things. I guess I always just interpreted "no-blame" as actually meaning "no blame for honest mistakes", but I guess that's too long to be catchy.

chx 4y ago
If incompetent people got in the position where they cause a production outage whose fault is that? Sure they should be moved / privileges stripped -- consequences are fine -- but once again, in my opinion, aside from maliciousness an outage is always an organizational mistake. I have twenty years of experience in the framework we work with and I can't roll a production hotfix without QA approval. What this comes down to is: if prod goes down, I should be able to focus on getting it back up and not worrying whether I have a job tomorrow. That's not conducive to success.

throwawaysleep 4y ago

Except that no one person even enormously missing something should be able to cause a major error.

aledalgrande 4y ago

no blame works without exceptions when the team makes it impossible to screw up big (test coverage, multiple reviews, modularity, canaries, DR strategy etc.) if you do retros and the same thing keeps coming up and it's always coming from the same person, then the manager will have to solve that

kache_ 4y ago

If you can take down the whole network, it isn't a single person's fuck up. Its the whole organization's fuck up.