4 ms·

I read a really interesting article on this in my undergraduate software engineering course (an excerpt from Tufte's excellent "Visual Explanations" book dealin

by gknoy 9y ago

I read a really interesting article on this in my undergraduate software engineering course (an excerpt from Tufte's excellent "Visual Explanations" book dealing with the shuttle Challenger mishap). The main takeaway was not that it was malicious management, but rather that the _way the information was portrayed_ (slides) did a remarkably poor job of showing the relationship between temperature and booster failure. No one showed management a graph of, _"Here's a chart of failures vs temperature, and this is how cold is is today ..."_ Looking at the memos and slides that were presented up the chain, it's not very surprising that the correlation between temperature and failure was lost on its way up the bureaucracy.

LeifCarrotson 9y ago
Engineering says "This is a bad idea". Management says "We want this regardless." The problem is not that the engineers didn't convey what they wanted to say clearly, it's that management did not want to listen.
- mrisoli 9y ago
  Exactly this, speaking from personal experience, a PM's job is to make sure things are shipped, not escalate Engineering problems to increasingly non-technical people up in the chain. If engineering says "This is a bad idea", PMs will say "It's not perfect", for management that means "ship it".
pdonis 9y ago
Any reading of Tufte's claims about the Challenger launch decision should be accompanied by reading this article, one of whose authors is Roger Boisjoly, one of the Thiokol engineers who recommended against the launch: https://people.rit.edu/wlrgsh/FINRobison.pdf https://people.rit.edu/wlrgsh/FINRobison.pdf tl/dr: Tufte's analysis itself exemplifies the charge Tufte makes. Tufte presents the wrong information about the way the decision was made, in the wrong way, and that is what drives his conclusions. When the right information is presented in the right way, it is clear that the Thiokol engineers did everything they possibly could, and that it was indeed a case of management overruling good recommendations by engineers. The article also gives very important background information: Thiokol had previously recommended grounding the Shuttle until issues with the O-rings that had surfaced on previous flights could be understood. NASA refused. So during the discussion the night before the Challenger launch, Thiokol could not make a number of arguments that would have seemed simpler and easier, taken in isolation--because they had already made those arguments and had them rejected.
- gknoy 9y ago
  Oh, wow, thank you for sharing that!
pdonis 9y ago
> Looking at the memos and slides that were presented up the chain, it's not very surprising that the correlation between temperature and failure was lost on its way up the bureaucracy. One of the points made by the article by Boisjoly et al (linked to in my other comment just now) is that "the correlation between temperature and failure" is hindsight (and incorrect hindsight at that--see below). On the night before the Challenger launch, nobody knew exactly what the problem was. That was not for lack of trying on the engineers' part: they had repeatedly requested more data and more experiments with the O-rings, and been refused. The basis of the engineers' recommendation the night before the Challenger launch (which, as I noted in my previous comment, must be considered in the light of the fact that they had already recommended grounding the Shuttle until the O-ring issue could be understood, and been refused) was not "the O-rings will fail in sufficiently cold weather"; there was not sufficient data to show that. The basis was that, in the absence of an actual understanding of the issue with the O-rings, no launch should be attempted outside the range of temperatures previously experienced. The reason "correlation between temperature and failure" is not correct hindsight is simple: as noted in the Boisjoly et al article, both test data and flight data had shown failures of the O-ring assembly at higher temperatures as well as lower temperatures! To understand how this could be, one needs to understand the root problem with the O-ring assembly (another item that Tufte's analysis gets wrong). The root problem is that, as the SRB fires, the gap between sections, which is supposed to be sealed by the O-ring assembly, expands. The O-ring assembly also has to expand, fast enough to keep the gap sealed; if it doesn't, hot gas can escape. It is true that the speed at which the O-ring assembly can expand gets slower as the O-rings get colder. However, it is also true, looking at the data, that even at temperatures as high as 75 F (and possibly as high as 100 F--the data is not conclusive), the O-rings already were not expanding fast enough to keep the gap sealed! In other words, based on a proper understanding of the root problem, pretty much every Shuttle flight with this O-ring assembly was rolling the dice. And, of course, there were indications of a problem on a number of previous flights, which were raised as issues by the Thiokol engineers and ignored by NASA.

by gknoy 9y ago

LeifCarrotson 9y ago

Engineering says "This is a bad idea". Management says "We want this regardless." The problem is not that the engineers didn't convey what they wanted to say clearly, it's that management did not want to listen.

mrisoli 9y ago
Exactly this, speaking from personal experience, a PM's job is to make sure things are shipped, not escalate Engineering problems to increasingly non-technical people up in the chain. If engineering says "This is a bad idea", PMs will say "It's not perfect", for management that means "ship it".

pdonis 9y ago

Any reading of Tufte's claims about the Challenger launch decision should be accompanied by reading this article, one of whose authors is Roger Boisjoly, one of the Thiokol engineers who recommended against the launch: https://people.rit.edu/wlrgsh/FINRobison.pdf https://people.rit.edu/wlrgsh/FINRobison.pdf tl/dr: Tufte's analysis itself exemplifies the charge Tufte makes. Tufte presents the wrong information about the way the decision was made, in the wrong way, and that is what drives his conclusions. When the right information is presented in the right way, it is clear that the Thiokol engineers did everything they possibly could, and that it was indeed a case of management overruling good recommendations by engineers. The article also gives very important background information: Thiokol had previously recommended grounding the Shuttle until issues with the O-rings that had surfaced on previous flights could be understood. NASA refused. So during the discussion the night before the Challenger launch, Thiokol could not make a number of arguments that would have seemed simpler and easier, taken in isolation--because they had already made those arguments and had them rejected.

gknoy 9y ago
Oh, wow, thank you for sharing that!

pdonis 9y ago

> Looking at the memos and slides that were presented up the chain, it's not very surprising that the correlation between temperature and failure was lost on its way up the bureaucracy. One of the points made by the article by Boisjoly et al (linked to in my other comment just now) is that "the correlation between temperature and failure" is hindsight (and incorrect hindsight at that--see below). On the night before the Challenger launch, nobody knew exactly what the problem was. That was not for lack of trying on the engineers' part: they had repeatedly requested more data and more experiments with the O-rings, and been refused. The basis of the engineers' recommendation the night before the Challenger launch (which, as I noted in my previous comment, must be considered in the light of the fact that they had already recommended grounding the Shuttle until the O-ring issue could be understood, and been refused) was not "the O-rings will fail in sufficiently cold weather"; there was not sufficient data to show that. The basis was that, in the absence of an actual understanding of the issue with the O-rings, no launch should be attempted outside the range of temperatures previously experienced. The reason "correlation between temperature and failure" is not correct hindsight is simple: as noted in the Boisjoly et al article, both test data and flight data had shown failures of the O-ring assembly at higher temperatures as well as lower temperatures! To understand how this could be, one needs to understand the root problem with the O-ring assembly (another item that Tufte's analysis gets wrong). The root problem is that, as the SRB fires, the gap between sections, which is supposed to be sealed by the O-ring assembly, expands. The O-ring assembly also has to expand, fast enough to keep the gap sealed; if it doesn't, hot gas can escape. It is true that the speed at which the O-ring assembly can expand gets slower as the O-rings get colder. However, it is also true, looking at the data, that even at temperatures as high as 75 F (and possibly as high as 100 F--the data is not conclusive), the O-rings already were not expanding fast enough to keep the gap sealed! In other words, based on a proper understanding of the root problem, pretty much every Shuttle flight with this O-ring assembly was rolling the dice. And, of course, there were indications of a problem on a number of previous flights, which were raised as issues by the Thiokol engineers and ignored by NASA.