3 ms·

Agreed. The argument "tool is misusable and therefore has no value" is absurd. Of course any discussion framework is going to rely on intelligent people using

Agreed. The argument "tool is misusable and therefore has no value" is absurd. Of course any discussion framework is going to rely on intelligent people using it sincerely, and even then will only provide some additional insight. It's a tool to help people make 10% better conclusions.

beat 8y ago
I really should find a name for this sort of fallacy (I'm sure some clever person has a clever name for it). "This is not perfect, therefore it is useless". I've seen this particular fallacy quite a bit on HN lately, and it's annoying me to no end.
agentultra 8y ago
I read the conclusion much differently. It said that in a system with a high rate of error the 5W analysis is insufficient at directing investigators to a reasonable solution. A definition for reasonable in the case of healthcare is given. It suggested that context was important and that Toyota's engineering practices are sufficiently different from health care that the technique cannot be recommended. I think the same may be true for networked software services as well. At a certain scale "root cause" becomes a matter of opinion or interpretation. Attempting to use a 5W methodology will lead you to one of many different, possible, causes but will ignore other confounding factors depending on who is doing the investigation and how it is conducted. In my experience finding the root cause of a particularly nasty error in a production cloud system required a formal model to prove that a particular error was triggered under very specific circumstances. The error was not reproducible and a test case elusive and yet it frequently ate up our error budget. There was no way any kind of root-cause analysis could have detected that error. It was a behavior comprised of over a dozen steps where the conditions had to be right to trigger it. Humans are not good at finding things like that on their own and simply asking, "Why?" like a parrot is not helpful... the system is simply too complex. Instead of root cause I like to think in terms of factors. What is the probability that a given factor has a high impact on our SLOs? Let's fix that. If we don't understand what the source of the problem is let's write a formal specification and test our assumptions and find the factor. Most of all, build for o11y and make reliability an engineering concern.
- beat 8y ago
  Of course 5 whys is no substitute for the scientific method. It isn't meant to be. Criticizing 5 ways because it's not good at science is unhelpful. It's a way to dig pretty deep into a problem very quickly with a simple heuristic. A 5 whys analysis of anything shouldn't take more than a couple of minutes. If your problem is going to need more than a couple of minutes of analysis, don't do 5 whys.

beat 8y ago

I really should find a name for this sort of fallacy (I'm sure some clever person has a clever name for it). "This is not perfect, therefore it is useless". I've seen this particular fallacy quite a bit on HN lately, and it's annoying me to no end.

agentultra 8y ago

I read the conclusion much differently. It said that in a system with a high rate of error the 5W analysis is insufficient at directing investigators to a reasonable solution. A definition for reasonable in the case of healthcare is given. It suggested that context was important and that Toyota's engineering practices are sufficiently different from health care that the technique cannot be recommended. I think the same may be true for networked software services as well. At a certain scale "root cause" becomes a matter of opinion or interpretation. Attempting to use a 5W methodology will lead you to one of many different, possible, causes but will ignore other confounding factors depending on who is doing the investigation and how it is conducted. In my experience finding the root cause of a particularly nasty error in a production cloud system required a formal model to prove that a particular error was triggered under very specific circumstances. The error was not reproducible and a test case elusive and yet it frequently ate up our error budget. There was no way any kind of root-cause analysis could have detected that error. It was a behavior comprised of over a dozen steps where the conditions had to be right to trigger it. Humans are not good at finding things like that on their own and simply asking, "Why?" like a parrot is not helpful... the system is simply too complex. Instead of root cause I like to think in terms of factors. What is the probability that a given factor has a high impact on our SLOs? Let's fix that. If we don't understand what the source of the problem is let's write a formal specification and test our assumptions and find the factor. Most of all, build for o11y and make reliability an engineering concern.

beat 8y ago
Of course 5 whys is no substitute for the scientific method. It isn't meant to be. Criticizing 5 ways because it's not good at science is unhelpful. It's a way to dig pretty deep into a problem very quickly with a simple heuristic. A 5 whys analysis of anything shouldn't take more than a couple of minutes. If your problem is going to need more than a couple of minutes of analysis, don't do 5 whys.