The Challenger explosion is a great case study. But in focusing on that chart from the Rogers Commission report, this piece reinforces what is basically a well-

The Challenger explosion is a great case study. But in focusing on that chart from the Rogers Commission report, this piece reinforces what is basically a well-told fable about the Challenger disaster. The piece says: "Below is the key graph of the O-ring test data that NASA analyzed before launch" and reproduces the famous chart. It continues, "NASA management used the data behind this first graph (among many other pieces of information) to justify their view the night before launch that there was no temperature effect on O-ring performance [...] But NASA management made one catastrophic mistake: this was not that chart they should have been looking at." I think these statements are pretty misleading without some major caveats. Tufte ("Visual and Statistical Thinking: Displays of Evidence for Making Decisions"; https://blogs.stockton.edu/hist4690/files/2012/06/Edward-Tufte-Visual-and-Statistical-Thinking.pdf https://blogs.stockton.edu/hist4690/files/2012/06/Edward-Tuf...) writes: "Most accounts of the Challenger reproduce a scatterplot that apparently demonstrates the analytical failure of the pre-launch debate. The graph depicts only launches with O-ring damage and their temperatures, omitting all damage-free launches (an absence of data points on the line of zero incidents of damage). First published in the shuttle commission report (PCSSCA, volume 1, 146), the chart is a favorite of statistics teachers. [...] The graph of the missing data-points is a vivid and poignant object lesson in how not to look at data when making an important decision. But it is too good to be true! First, the graph was not part of the pre-launch debate; it was not among the 13 charts used by Thiokol and NASA in deciding to launch. Rather, it was drawn after the accident by two staff members (the executive director and a lawyer) at the commission as their simulation of the poor reasoning in the pre-launch debate. Second, the graph implies that the pre-launch analysis examined 7 launches at 7 temperatures with 7 damage measurements. That is not true; only 2 cases of blow-by and 2 temperatures were linked up. The actual pre-launch analysis was much thinner than indicated by the commission scatterplot. Third, the damage scale is dequantified, only counting the number of incidents rather than measuring their severity. In short, whether for teaching statistics or for seeking to understand the practice of data graphics, why use an inaccurately simulated post-launch chart when we have the genuine 13 pre-launch decision charts right in hand?" (For a response to Tufte's essay, see https://people.rit.edu/wlrgsh/FINRobison.pdf https://people.rit.edu/wlrgsh/FINRobison.pdf, also cited elsewhere here.)