4 ms·

The problem with 5 Whys? Taking it too literally, and then using this as basis for an argument criticizing the whole concept. To be fair, the idea of multiple

by beat 8y ago

The problem with 5 Whys? Taking it too literally, and then using this as basis for an argument criticizing the whole concept. To be fair, the idea of multiple root causes (and the difficulty 5 whys has in uncovering multiple causes) is important. But that's not grounds for total dismissal. Because the value of 5 whys isn't "ask why 5 times and you arrive at the root cause", but rather "Don't stop at the first answer you find". Since rather a lot of people stop at the first cause they see rather a lot of the time, it's a valuable approach.

alexandercrohde 8y ago
Agreed. The argument "tool is misusable and therefore has no value" is absurd. Of course any discussion framework is going to rely on intelligent people using it sincerely, and even then will only provide some additional insight. It's a tool to help people make 10% better conclusions.
- beat 8y ago
  I really should find a name for this sort of fallacy (I'm sure some clever person has a clever name for it). "This is not perfect, therefore it is useless". I've seen this particular fallacy quite a bit on HN lately, and it's annoying me to no end.
- agentultra 8y ago
  I read the conclusion much differently. It said that in a system with a high rate of error the 5W analysis is insufficient at directing investigators to a reasonable solution. A definition for reasonable in the case of healthcare is given. It suggested that context was important and that Toyota's engineering practices are sufficiently different from health care that the technique cannot be recommended. I think the same may be true for networked software services as well. At a certain scale "root cause" becomes a matter of opinion or interpretation. Attempting to use a 5W methodology will lead you to one of many different, possible, causes but will ignore other confounding factors depending on who is doing the investigation and how it is conducted. In my experience finding the root cause of a particularly nasty error in a production cloud system required a formal model to prove that a particular error was triggered under very specific circumstances. The error was not reproducible and a test case elusive and yet it frequently ate up our error budget. There was no way any kind of root-cause analysis could have detected that error. It was a behavior comprised of over a dozen steps where the conditions had to be right to trigger it. Humans are not good at finding things like that on their own and simply asking, "Why?" like a parrot is not helpful... the system is simply too complex. Instead of root cause I like to think in terms of factors. What is the probability that a given factor has a high impact on our SLOs? Let's fix that. If we don't understand what the source of the problem is let's write a formal specification and test our assumptions and find the factor. Most of all, build for o11y and make reliability an engineering concern.
  - beat 8y ago
    Of course 5 whys is no substitute for the scientific method. It isn't meant to be. Criticizing 5 ways because it's not good at science is unhelpful. It's a way to dig pretty deep into a problem very quickly with a simple heuristic. A 5 whys analysis of anything shouldn't take more than a couple of minutes. If your problem is going to need more than a couple of minutes of analysis, don't do 5 whys.
hinkley 8y ago
It's the XY problem. Stopping at 1 or 2 Whys almost always results in a large body of bandaids, duct tape and bailing wire. If you go three or four deep you start digging into cross cutting concerns that create the same class of error in multiple ways. By 5 you've probably found a useful culprit in 99% of cases. (Occasionally 2 Whys turns into a toxic environment based on shaming. Why are we down? Because Steve screwed up. Don't be like Steve!)

by beat 8y ago

alexandercrohde 8y ago

Agreed. The argument "tool is misusable and therefore has no value" is absurd. Of course any discussion framework is going to rely on intelligent people using it sincerely, and even then will only provide some additional insight. It's a tool to help people make 10% better conclusions.

beat 8y ago
I really should find a name for this sort of fallacy (I'm sure some clever person has a clever name for it). "This is not perfect, therefore it is useless". I've seen this particular fallacy quite a bit on HN lately, and it's annoying me to no end.
agentultra 8y ago
I read the conclusion much differently. It said that in a system with a high rate of error the 5W analysis is insufficient at directing investigators to a reasonable solution. A definition for reasonable in the case of healthcare is given. It suggested that context was important and that Toyota's engineering practices are sufficiently different from health care that the technique cannot be recommended. I think the same may be true for networked software services as well. At a certain scale "root cause" becomes a matter of opinion or interpretation. Attempting to use a 5W methodology will lead you to one of many different, possible, causes but will ignore other confounding factors depending on who is doing the investigation and how it is conducted. In my experience finding the root cause of a particularly nasty error in a production cloud system required a formal model to prove that a particular error was triggered under very specific circumstances. The error was not reproducible and a test case elusive and yet it frequently ate up our error budget. There was no way any kind of root-cause analysis could have detected that error. It was a behavior comprised of over a dozen steps where the conditions had to be right to trigger it. Humans are not good at finding things like that on their own and simply asking, "Why?" like a parrot is not helpful... the system is simply too complex. Instead of root cause I like to think in terms of factors. What is the probability that a given factor has a high impact on our SLOs? Let's fix that. If we don't understand what the source of the problem is let's write a formal specification and test our assumptions and find the factor. Most of all, build for o11y and make reliability an engineering concern.
- beat 8y ago
  Of course 5 whys is no substitute for the scientific method. It isn't meant to be. Criticizing 5 ways because it's not good at science is unhelpful. It's a way to dig pretty deep into a problem very quickly with a simple heuristic. A 5 whys analysis of anything shouldn't take more than a couple of minutes. If your problem is going to need more than a couple of minutes of analysis, don't do 5 whys.

hinkley 8y ago

It's the XY problem. Stopping at 1 or 2 Whys almost always results in a large body of bandaids, duct tape and bailing wire. If you go three or four deep you start digging into cross cutting concerns that create the same class of error in multiple ways. By 5 you've probably found a useful culprit in 99% of cases. (Occasionally 2 Whys turns into a toxic environment based on shaming. Why are we down? Because Steve screwed up. Don't be like Steve!)