10 ms·

The Space Shuttle Challenger Explosion and the O-ring

169 points by sethbannon 10y ago

tps5 10y ago
Richard Feynman, on NASA's attitude toward the space shuttle program: "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled."
- delish 10y ago
  Another famous intellectual on humility toward Nature, Goethe: Nature understands no jesting; she is always true, always serious, always severe; she is always right, and errors and faults are always those of man. The man incapable of appreciating her, she despises; and only to the apt, the pure, and the true, does she resign herself and reveal her secrets.
- stcredzero 10y ago
  The gaming industry needs to understand that this applies to coding for games as well. It especially applies to multiplayer.
  - flamedoge 10y ago
    multiplayer? We do fool clients all the time with extrapolation, just to shave half of the latency. And when we get it wrong, we "correct" it retroactively.
    - stcredzero 10y ago
      The laws of nature are different in this context, but you have to follow them with the same diligence with the same harshness awaiting when you fail. In games, it's not what is logical or mathematically correct. It's what feels correct.
nemild 10y ago
I'm the author of this piece, happy to answer questions. I grew up with stories of the Challenger after my father - a statistician - and 2 of his co-authors were selected by National Academy of Sciences to study if the danger could have been predicted beforehand. They showed that the likelihood of failure was 13% at the launch temperature, but would have been negligible if NASA had waited just a few hours. (His co-author, Ed Fowlkes, was dying of AIDS at the time - and considered this paper one of his life's great achievements) Bad statistical inferences were a huge part of the launch story, and you can see more in Richard Feynman's critiques: https://en.wikipedia.org/wiki/Rogers_Commission_Report https://en.wikipedia.org/wiki/Rogers_Commission_Report Secondly, the effect I highlight (a biased data sample) is a key issue with news/social media - and can lead us to heavily flawed inferences if we don't correct for it. I'll dig deep into this in future posts with a substantial amount of data and visualizations.
- kevinpet 10y ago
  Feynman's actually observations are well worth reading for anyone who builds anything that may be vaguely considered engineering. http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/rogers-commission/Appendix-F.txt http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roger... The key items for me were: 1) While they had no expectation of erosion, and the design did not call for the o-rings to erode, once they observed them eroding, they retroactively invented a "margin of error" based on what fraction the o-rings eroded. This was not based on an actual understood process, and is akin to saying "well, the bridge didn't break when we drove that truck over it, so it must be okay" 2) The engineers actually knew the risk (~1% chance of loss per launch, not specific to the o-rings, compared with two actual losses of the shuttle over ~130 missions). Management used entirely invented numbers for the risk which were not justified.
  - JohnJamesRambo 10y ago
    Since 1 part in 100,000 would imply that one could put a Shuttle up each day for 300 years expecting to lose only one, we could properly ask "What is the cause of management's fantastic faith in the machinery?" I wonder if you told the management that, in those words, if they would have still believed such a ludicrous idea.
mavhc 10y ago
Of course if politics hadn't caused parts of the shuttle to be built far away from the launch pad, they wouldn't have needed O-rings in the first place
- greenshackle2 10y ago
  Yes, it is well known that American engineers do not use O-rings. In fact, Americans could have built the entire shuttle without using any parts at all.
  - snrplfth 10y ago
    The Shuttle Solid Rocket Boosters were mainly built in Utah, and many people assert that this choice of location was due to political patronage. The fact that these boosters were built inland, far from barge-capable waterways and distant from the launch site, meant that, rather than being completed as a single large piece at the factory, it arrived at the Kennedy Space Center as four pieces, which were then joined together with the O-rings in question. This made them vulnerable to a blowout of the kind experienced with Challenger; solid rocket boosters made in a single piece are generally much less vulnerable to this kind of failure.
    - greenshackle2 10y ago
      Fair enough, I don't know what I'm talking about. Thanks for explaining. (I thought OP meant no O-rings would've been used in the entire shuttle, not just these particular ones.)
- simonjgreen 10y ago
  Definite brinkmanship. This is comparable to performing a root cause analysis and finding the root cause to be a desire to go to space.
Splines 10y ago
This is why I always have my visualizations overlay both failure rates and usage. A given failure rate (or even worse, a failure count) doesn't tell you much about the system without also including the totals of both success and failure. I'm sure I could be more rigorous with this though. Is there a way to express a given failure rate in terms of certainty? As in, we have sampled the failure rate of a component with fixed parameters a, b, c, and we are x% certain of the failure rate? (Maybe I'm wording this wrong - I don't have much of a stats background).
- acidbaseextract 10y ago
  > This is why I always have my visualizations overlay both failure rates and usage. Do you have an example? I love having more viz tools in my toolbox!
  - Splines 10y ago
    Sorry for the late reply - where I work we use an in-house platform that uses Kendo for rendering charts. http://demos.telerik.com/kendo-ui/bar-charts/column http://demos.telerik.com/kendo-ui/bar-charts/column What I do isn't anything particularly sophisticated, I usually do a multi-axis chart with a line & column chart, with one axis corresponding to usage, and the other axis corresponding to failure rate. Imagine a basic multi-axis chart in Excel, except rendered in a browser, and you're 90% of the way there.
seliopou 10y ago
Tufte wrote an essay on how the data available suggested that there was a high likelihood of O-ring failure, but that the data and findings were poorly communicated. This led to the decision to launch, the subsequent failure of the O-rings and loss of life. This essay appears in the booklet "Visual and Statistical Thinking"[0], among other publications, along with an other essay on how the source of cholera was traced to contaminated drinking water in 19th century London by John Snow. He plotted cholera cases on a map, and looked at where the outbreaks were most frequent.[1] This also led to the discovery of the vector of cholera, which up until then was unknown or at least misattributed. Both are great reads. [0]: https://www.sfu.ca/cmns/courses/2012/801/1-Readings/Tufte%20Visual%20and%20Statistical%20Thinking.pdf https://www.sfu.ca/cmns/courses/2012/801/1-Readings/Tufte%20... [1]: https://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outbreak#/media/File:Snow-cholera-map-1.jpg https://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outb...
- justin66 10y ago
  Some of the engineers at Thiokol whose work Tufte criticized have responded over the years. There's at least one painfully academic paper out there, although if you wanted to you could start here: http://www.onlineethics.org/Topics/ProfPractice/Exemplars/BehavingWell/RB-intro/RepMisrep.aspx#misrep http://www.onlineethics.org/Topics/ProfPractice/Exemplars/Be... https://eagereyes.org/criticism/tufte-and-the-truth-about-the-challenger https://eagereyes.org/criticism/tufte-and-the-truth-about-th... It's been longer since I read Feynman but I recall his assessment as being a lot more grounded, and fairer to the engineers.
  - masklinn 10y ago
    > It's been longer since I read Feynman but I recall his assessment as being a lot more grounded, and fairer to the engineers. Feynman laid the vast majority of the blame on management ("NASA officials" in his Appendix F), noting that engineers had fairly realistic views of the matter (and failure rate estimates) and IIRC that they'd tried to raise concerns but those had gotten lost climbing the manglement ladder. The one unit for which he had nothing but praise was the Software Group: > To summarize then, the computer software checking system and attitude is of the highest quality. There appears to be no process of gradually fooling oneself while degrading standards so characteristic of the Solid Rocket Booster or Space Shuttle Main Engine safety systems. Nothing that they had to constantly resist manglement trying to mangle: > To be sure, there have been recent suggestions by management to curtail such elaborate and expensive tests as being unnecessary at this late date in Shuttle history.
cvoss 10y ago
My university has a course (maybe in the business or statistics departments? I'm not sure, as I didn't take it) that has a project where the professor gives this very pre-launch data to groups of students in a totally different context: It is presented as Formula One race data, and the task is for the students (the racing team) to decide whether or not to pull out of the important race based on current weather conditions and the potential safety implications for the driver. The next class period, only after the teams propose their decided course of action, it is revealed where the data really came from. I imagine it's quite jarring, especially for those who decided to proceed, albeit with different risks in mind.
- nemild 10y ago
  This is often taught in business schools and one case is called Carter Racing: http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-A.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma...
  - LeifCarrotson 10y ago
    That's only the first file in the series. There are -B.pdf and -C.pdf files as well. http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-B.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-C.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... And here's a convenient link to all three, apparently an Apache feature I didn't know about: http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-D.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... > Multiple Choices > The document name you requested (/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-D.pdf) could not be found on this server. However, we found documents with names similar to the one you requested. > Available documents: > http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-A.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... (mistyped character) > http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-B.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... (mistyped character) > http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-C.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... (mistyped character) > Apache Server at heller.brandeis.edu Port 80 Also, each document has the following item in the footer. I suspect that Brandeis.edu is violating their license agreement by hosting these, and also that the license agreement was designed by someone who really doesn't like the Internet or computers: > Not to be reproduced, modified, stored, or transmitted without prior written permission of the copyright holder or agent.
Pigo 10y ago
I was only 5 when this happened, but I remember what a blow this was to my school. Were past missions, like the Apollo's, dismissive of concerns like this as well and just lucky? Or was the shuttle missions just more complex, with more points of failure to be concerned about?
- digi_owl 10y ago
  No data to back it up with, but i think during the space race things were new enough that they often didn't know the chances. But the time of the shuttle, NASA had moved from a prestigious project to pork barrel politics. End result was that managers and politicians were endlessly overruling the engineers.
- jartelt 10y ago
  Since the shuttle was designed to do repeated take offs and landings, it was very complicated. Of course the Apollo mission were super complicated as well, but the only had to design for one launch. This is part of the reason why the shuttle program was so expensive.
- billsmithaustin 10y ago
  The Apollo 1 cabin caught on fire on the launch pad. All three astronauts died.
  - mturmon 10y ago
    And Apollo 13 was a near loss, although it worked out ok.
- dbcurtis 10y ago
  My understanding is that the attitude of NASA management changed from the Apollo era's "Prove to me we are good to go." to one of "Prove to me we can't go." Some might put it: Gene Kranz retired. Source: Anecdotes from an old friend who is a quality assurance engineer. He was one of the boys on the ground in Huston that brought the Apollo 13 crew home.
lordnacho 10y ago
Am I reading this wrong or did they launch the thing at a temperature way outside the range where they normally launch shuttles? Wouldn't there be a whole bunch of different stats measured, which would all say that you should be cautious when trying a region far away from what you know? In any case, a very good example of how stats is unintuitive. I hadn't guessed about the missing "no error" data until I read it. I'm sure there's many more little things like that. Simpson's paradox, those kinds of things.
- masklinn 10y ago
  > Am I reading this wrong or did they launch the thing at a temperature way outside the range where they normally launch shuttles? No, that's correct, the launch was the coldest yet and reached temperatures at which the O-rings had lost their flexibility and couldn't spring back fast enough to seal. In fact an iconic scene from the Challenger hearings was Feynman showing (on TV!) the loss of ductility after having dunked an O-ring in ice-water.
dispose13432 10y ago
It's an interesting example where just because something is done by the "public" doesn't make it safer than done privately. Everyone knows about regulatory capture (when companies manage their safety regulators). Normally it's private organizations pushing public safety regulators. Here, on the other hand, it was a public organization that took risks. The reason is simple. Every organization has certain needs. Boeing needs to make planes (make $), Ford needs to make cars (make $), etc. Safety is an annoying thing they need to get over with ASAP to get to their primary purpose (make $). Nasa needed to launch, and (at least for the managers) it became an acceptable risk. If it flies 20 times and blows once, they win. So should there be a NASA and a NASAOC (NASA oversight committee, to check on them)? Then the organization on top of both (Congress, the President) will choose which one to listen. This is the general problem of self-policing. And the only way to get around this is by having multiple, independent, providers. So if NASA doesn't think SpaceX is safe enough, they can shut down the contract while still having access to space. If Nasa had that in 1986, the Shuttle would have been (rightfully) decommissioned then and there. Unfortunately, it required _another_ accident before anything moved. And the lesson can be learned to matters outside space.
mixmastamyk 10y ago
I remember that day vividly, I was in jr. high school. It's hard to describe how traumatic it was; the school teacher on board made it ten times worse. I would compare it to 9/11 for those that aren't old enough to remember. As we learned later about the rubber o-rings failing in cold weather, the solution has always been framed as not launching in those conditions. But, why not use a different material, or design away the o-rings to avoid the problem in the first place? Edit: question is already answered here: https://news.ycombinator.com/item?id=13239241 https://news.ycombinator.com/item?id=13239241
- iamatworknow 10y ago
  >I remember that day vividly, I was in jr. high school. It's hard to describe how traumatic it was; the school teacher on board made it ten times worse. I would compare it to 9/11 for those that aren't old enough to remember. It's interesting (albeit depressing) to think about how every generation in the television (and now internet) age seems to have at least one of these events that occurs right on the border of adulthood; when you've become old enough to have a sense of the world and the personal impact of the event kind of jolts you into reality, so to speak. For my dad it was JFK's assassination and for me it was 9/11. It's also notable that when Columbia disintegrated upon re-entry in 2003 the media and public at large didn't seem to pay it much attention at all (or at least I don't remember it being such a big deal).
  - dispose13432 10y ago
    >It's also notable that when Columbia disintegrated upon re-entry in 2003 the media and public at large didn't seem to pay it much attention at all (or at least I don't remember it being such a big deal). Probably because it wasn't broadcast live.
    - sharksauce 10y ago
      It certainly was broadcast live -- every major TV news organization in the US broadcast the re-entry. Of course Columbia was very high altitude when it broke apart and the cameras (fortunately in my opinion) couldn't get very close-in shots. There's a point to be made that people were more prepared for a possible disaster given the reports of damage to the shuttle that the crew provided days ahead, compared to the absolute surprise and shock at Challenger, and the horrifyingly clear camera footage provided at a lower altitude.
keithwinstein 10y ago
The Challenger explosion is a great case study. But in focusing on that chart from the Rogers Commission report, this piece reinforces what is basically a well-told fable about the Challenger disaster. The piece says: "Below is the key graph of the O-ring test data that NASA analyzed before launch" and reproduces the famous chart. It continues, "NASA management used the data behind this first graph (among many other pieces of information) to justify their view the night before launch that there was no temperature effect on O-ring performance [...] But NASA management made one catastrophic mistake: this was not that chart they should have been looking at." I think these statements are pretty misleading without some major caveats. Tufte ("Visual and Statistical Thinking: Displays of Evidence for Making Decisions"; https://blogs.stockton.edu/hist4690/files/2012/06/Edward-Tufte-Visual-and-Statistical-Thinking.pdf https://blogs.stockton.edu/hist4690/files/2012/06/Edward-Tuf...) writes: "Most accounts of the Challenger reproduce a scatterplot that apparently demonstrates the analytical failure of the pre-launch debate. The graph depicts only launches with O-ring damage and their temperatures, omitting all damage-free launches (an absence of data points on the line of zero incidents of damage). First published in the shuttle commission report (PCSSCA, volume 1, 146), the chart is a favorite of statistics teachers. [...] The graph of the missing data-points is a vivid and poignant object lesson in how not to look at data when making an important decision. But it is too good to be true! First, the graph was not part of the pre-launch debate; it was not among the 13 charts used by Thiokol and NASA in deciding to launch. Rather, it was drawn after the accident by two staff members (the executive director and a lawyer) at the commission as their simulation of the poor reasoning in the pre-launch debate. Second, the graph implies that the pre-launch analysis examined 7 launches at 7 temperatures with 7 damage measurements. That is not true; only 2 cases of blow-by and 2 temperatures were linked up. The actual pre-launch analysis was much thinner than indicated by the commission scatterplot. Third, the damage scale is dequantified, only counting the number of incidents rather than measuring their severity. In short, whether for teaching statistics or for seeking to understand the practice of data graphics, why use an inaccurately simulated post-launch chart when we have the genuine 13 pre-launch decision charts right in hand?" (For a response to Tufte's essay, see https://people.rit.edu/wlrgsh/FINRobison.pdf https://people.rit.edu/wlrgsh/FINRobison.pdf, also cited elsewhere here.)
- aidenn0 10y ago
babesh 10y ago
Institutional failure: don't forget the second shuttle disaster on reentry (Columbia) with the known issues with tiles getting knocked off by foam. Badly engineered contraption.
joering2 10y ago
Just so we on the same page - if you read enough about this sad story, you know damn well it wasn't an explosion or the faulty/frozen O-ring that killed those brave souls - it was a horrible amount of bureaucracy and ignorance towards engineers, who rang warning bells long before Challenger's liftoff.
skmurphy 10y ago
The full data chart reminders me of Abraham Wald asking the question "where do we never see damage on a returning plane?" to understand that those areas meant complete loss of the aircraft and would most benefit from more armor.
amoruso 10y ago
I'll add another recommendation for Tufte's writings on the Challenger explosion. It should be required reading for all engineers. People who criticize Tufte for oversimplifying miss the point entirely. It's not about analysis, it's about communication. It's one thing for domain specialist to have a complex, multi-dimensional understanding of their specialty; extracting a relevant summary for non-experts is something else entirely. If you've ever been in a meeting where you had trouble getting your point across, you should read this. Make diagrams like Tufte does to get your point across, make more detailed ones as backups if you need to dive deep into details. http://williamwolff.org/wp-content/uploads/2013/01/tufte-challenger-1997.pdf http://williamwolff.org/wp-content/uploads/2013/01/tufte-cha...
- 7952 10y ago
  I think there is a tendancy to trivialise the difficulty of communication on large projects. I have had people tell me that "it is not rocket science". Well actually, maybe it is much harder than that. A small team can design a rocket engine. Getting a small team of managers to know all the right facts is very hard and on many projects seemingly impossible. And that is on projects where you can have very large margins of error. Obviously that is not always possible building things with strict mass limits.
rietta 10y ago
The Challenger explosion is one of my first vivid memories from my childhood as my mom was letting me watch the launch on TV as a 3 year old. I also had an understanding of death at the time as I asked my mom if they were in heaven (which was as advanced as my understanding was at the time). This is a fascinating read 30 years later.

169 points by sethbannon 10y ago

tps5 10y ago

Richard Feynman, on NASA's attitude toward the space shuttle program: "For a successful technology, reality must take precedence over public relations, for nature cannot be fooled."

delish 10y ago
Another famous intellectual on humility toward Nature, Goethe: Nature understands no jesting; she is always true, always serious, always severe; she is always right, and errors and faults are always those of man. The man incapable of appreciating her, she despises; and only to the apt, the pure, and the true, does she resign herself and reveal her secrets.
stcredzero 10y ago
The gaming industry needs to understand that this applies to coding for games as well. It especially applies to multiplayer.
- flamedoge 10y ago
  multiplayer? We do fool clients all the time with extrapolation, just to shave half of the latency. And when we get it wrong, we "correct" it retroactively.
  - stcredzero 10y ago
    The laws of nature are different in this context, but you have to follow them with the same diligence with the same harshness awaiting when you fail. In games, it's not what is logical or mathematically correct. It's what feels correct.

nemild 10y ago

I'm the author of this piece, happy to answer questions. I grew up with stories of the Challenger after my father - a statistician - and 2 of his co-authors were selected by National Academy of Sciences to study if the danger could have been predicted beforehand. They showed that the likelihood of failure was 13% at the launch temperature, but would have been negligible if NASA had waited just a few hours. (His co-author, Ed Fowlkes, was dying of AIDS at the time - and considered this paper one of his life's great achievements) Bad statistical inferences were a huge part of the launch story, and you can see more in Richard Feynman's critiques: https://en.wikipedia.org/wiki/Rogers_Commission_Report https://en.wikipedia.org/wiki/Rogers_Commission_Report Secondly, the effect I highlight (a biased data sample) is a key issue with news/social media - and can lead us to heavily flawed inferences if we don't correct for it. I'll dig deep into this in future posts with a substantial amount of data and visualizations.

kevinpet 10y ago
Feynman's actually observations are well worth reading for anyone who builds anything that may be vaguely considered engineering. http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/rogers-commission/Appendix-F.txt http://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roger... The key items for me were: 1) While they had no expectation of erosion, and the design did not call for the o-rings to erode, once they observed them eroding, they retroactively invented a "margin of error" based on what fraction the o-rings eroded. This was not based on an actual understood process, and is akin to saying "well, the bridge didn't break when we drove that truck over it, so it must be okay" 2) The engineers actually knew the risk (~1% chance of loss per launch, not specific to the o-rings, compared with two actual losses of the shuttle over ~130 missions). Management used entirely invented numbers for the risk which were not justified.
- JohnJamesRambo 10y ago
  Since 1 part in 100,000 would imply that one could put a Shuttle up each day for 300 years expecting to lose only one, we could properly ask "What is the cause of management's fantastic faith in the machinery?" I wonder if you told the management that, in those words, if they would have still believed such a ludicrous idea.

mavhc 10y ago

Of course if politics hadn't caused parts of the shuttle to be built far away from the launch pad, they wouldn't have needed O-rings in the first place

greenshackle2 10y ago
Yes, it is well known that American engineers do not use O-rings. In fact, Americans could have built the entire shuttle without using any parts at all.
- snrplfth 10y ago
  The Shuttle Solid Rocket Boosters were mainly built in Utah, and many people assert that this choice of location was due to political patronage. The fact that these boosters were built inland, far from barge-capable waterways and distant from the launch site, meant that, rather than being completed as a single large piece at the factory, it arrived at the Kennedy Space Center as four pieces, which were then joined together with the O-rings in question. This made them vulnerable to a blowout of the kind experienced with Challenger; solid rocket boosters made in a single piece are generally much less vulnerable to this kind of failure.
  - greenshackle2 10y ago
    Fair enough, I don't know what I'm talking about. Thanks for explaining. (I thought OP meant no O-rings would've been used in the entire shuttle, not just these particular ones.)
simonjgreen 10y ago
Definite brinkmanship. This is comparable to performing a root cause analysis and finding the root cause to be a desire to go to space.

Splines 10y ago

This is why I always have my visualizations overlay both failure rates and usage. A given failure rate (or even worse, a failure count) doesn't tell you much about the system without also including the totals of both success and failure. I'm sure I could be more rigorous with this though. Is there a way to express a given failure rate in terms of certainty? As in, we have sampled the failure rate of a component with fixed parameters a, b, c, and we are x% certain of the failure rate? (Maybe I'm wording this wrong - I don't have much of a stats background).

acidbaseextract 10y ago
> This is why I always have my visualizations overlay both failure rates and usage. Do you have an example? I love having more viz tools in my toolbox!
- Splines 10y ago
  Sorry for the late reply - where I work we use an in-house platform that uses Kendo for rendering charts. http://demos.telerik.com/kendo-ui/bar-charts/column http://demos.telerik.com/kendo-ui/bar-charts/column What I do isn't anything particularly sophisticated, I usually do a multi-axis chart with a line & column chart, with one axis corresponding to usage, and the other axis corresponding to failure rate. Imagine a basic multi-axis chart in Excel, except rendered in a browser, and you're 90% of the way there.

seliopou 10y ago

Tufte wrote an essay on how the data available suggested that there was a high likelihood of O-ring failure, but that the data and findings were poorly communicated. This led to the decision to launch, the subsequent failure of the O-rings and loss of life. This essay appears in the booklet "Visual and Statistical Thinking"[0], among other publications, along with an other essay on how the source of cholera was traced to contaminated drinking water in 19th century London by John Snow. He plotted cholera cases on a map, and looked at where the outbreaks were most frequent.[1] This also led to the discovery of the vector of cholera, which up until then was unknown or at least misattributed. Both are great reads. [0]: https://www.sfu.ca/cmns/courses/2012/801/1-Readings/Tufte%20Visual%20and%20Statistical%20Thinking.pdf https://www.sfu.ca/cmns/courses/2012/801/1-Readings/Tufte%20... [1]: https://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outbreak#/media/File:Snow-cholera-map-1.jpg https://en.wikipedia.org/wiki/1854_Broad_Street_cholera_outb...

justin66 10y ago
Some of the engineers at Thiokol whose work Tufte criticized have responded over the years. There's at least one painfully academic paper out there, although if you wanted to you could start here: http://www.onlineethics.org/Topics/ProfPractice/Exemplars/BehavingWell/RB-intro/RepMisrep.aspx#misrep http://www.onlineethics.org/Topics/ProfPractice/Exemplars/Be... https://eagereyes.org/criticism/tufte-and-the-truth-about-the-challenger https://eagereyes.org/criticism/tufte-and-the-truth-about-th... It's been longer since I read Feynman but I recall his assessment as being a lot more grounded, and fairer to the engineers.
- masklinn 10y ago
  > It's been longer since I read Feynman but I recall his assessment as being a lot more grounded, and fairer to the engineers. Feynman laid the vast majority of the blame on management ("NASA officials" in his Appendix F), noting that engineers had fairly realistic views of the matter (and failure rate estimates) and IIRC that they'd tried to raise concerns but those had gotten lost climbing the manglement ladder. The one unit for which he had nothing but praise was the Software Group: > To summarize then, the computer software checking system and attitude is of the highest quality. There appears to be no process of gradually fooling oneself while degrading standards so characteristic of the Solid Rocket Booster or Space Shuttle Main Engine safety systems. Nothing that they had to constantly resist manglement trying to mangle: > To be sure, there have been recent suggestions by management to curtail such elaborate and expensive tests as being unnecessary at this late date in Shuttle history.

cvoss 10y ago

My university has a course (maybe in the business or statistics departments? I'm not sure, as I didn't take it) that has a project where the professor gives this very pre-launch data to groups of students in a totally different context: It is presented as Formula One race data, and the task is for the students (the racing team) to decide whether or not to pull out of the important race based on current weather conditions and the potential safety implications for the driver. The next class period, only after the teams propose their decided course of action, it is revealed where the data really came from. I imagine it's quite jarring, especially for those who decided to proceed, albeit with different risks in mind.

nemild 10y ago
This is often taught in business schools and one case is called Carter Racing: http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-A.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma...
- LeifCarrotson 10y ago
  That's only the first file in the series. There are -B.pdf and -C.pdf files as well. http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-B.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-C.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... And here's a convenient link to all three, apparently an Apache feature I didn't know about: http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-D.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... > Multiple Choices > The document name you requested (/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-D.pdf) could not be found on this server. However, we found documents with names similar to the one you requested. > Available documents: > http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-A.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... (mistyped character) > http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-B.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... (mistyped character) > http://heller.brandeis.edu/executive-education/maine-2012/may/pdfs/BHLP-102-READING-Carter-C.pdf http://heller.brandeis.edu/executive-education/maine-2012/ma... (mistyped character) > Apache Server at heller.brandeis.edu Port 80 Also, each document has the following item in the footer. I suspect that Brandeis.edu is violating their license agreement by hosting these, and also that the license agreement was designed by someone who really doesn't like the Internet or computers: > Not to be reproduced, modified, stored, or transmitted without prior written permission of the copyright holder or agent.

Pigo 10y ago

I was only 5 when this happened, but I remember what a blow this was to my school. Were past missions, like the Apollo's, dismissive of concerns like this as well and just lucky? Or was the shuttle missions just more complex, with more points of failure to be concerned about?

digi_owl 10y ago
No data to back it up with, but i think during the space race things were new enough that they often didn't know the chances. But the time of the shuttle, NASA had moved from a prestigious project to pork barrel politics. End result was that managers and politicians were endlessly overruling the engineers.
jartelt 10y ago
Since the shuttle was designed to do repeated take offs and landings, it was very complicated. Of course the Apollo mission were super complicated as well, but the only had to design for one launch. This is part of the reason why the shuttle program was so expensive.
billsmithaustin 10y ago
The Apollo 1 cabin caught on fire on the launch pad. All three astronauts died.
- mturmon 10y ago
  And Apollo 13 was a near loss, although it worked out ok.
dbcurtis 10y ago
My understanding is that the attitude of NASA management changed from the Apollo era's "Prove to me we are good to go." to one of "Prove to me we can't go." Some might put it: Gene Kranz retired. Source: Anecdotes from an old friend who is a quality assurance engineer. He was one of the boys on the ground in Huston that brought the Apollo 13 crew home.

lordnacho 10y ago

Am I reading this wrong or did they launch the thing at a temperature way outside the range where they normally launch shuttles? Wouldn't there be a whole bunch of different stats measured, which would all say that you should be cautious when trying a region far away from what you know? In any case, a very good example of how stats is unintuitive. I hadn't guessed about the missing "no error" data until I read it. I'm sure there's many more little things like that. Simpson's paradox, those kinds of things.

masklinn 10y ago
> Am I reading this wrong or did they launch the thing at a temperature way outside the range where they normally launch shuttles? No, that's correct, the launch was the coldest yet and reached temperatures at which the O-rings had lost their flexibility and couldn't spring back fast enough to seal. In fact an iconic scene from the Challenger hearings was Feynman showing (on TV!) the loss of ductility after having dunked an O-ring in ice-water.

dispose13432 10y ago

It's an interesting example where just because something is done by the "public" doesn't make it safer than done privately. Everyone knows about regulatory capture (when companies manage their safety regulators). Normally it's private organizations pushing public safety regulators. Here, on the other hand, it was a public organization that took risks. The reason is simple. Every organization has certain needs. Boeing needs to make planes (make $), Ford needs to make cars (make $), etc. Safety is an annoying thing they need to get over with ASAP to get to their primary purpose (make $). Nasa needed to launch, and (at least for the managers) it became an acceptable risk. If it flies 20 times and blows once, they win. So should there be a NASA and a NASAOC (NASA oversight committee, to check on them)? Then the organization on top of both (Congress, the President) will choose which one to listen. This is the general problem of self-policing. And the only way to get around this is by having multiple, independent, providers. So if NASA doesn't think SpaceX is safe enough, they can shut down the contract while still having access to space. If Nasa had that in 1986, the Shuttle would have been (rightfully) decommissioned then and there. Unfortunately, it required _another_ accident before anything moved. And the lesson can be learned to matters outside space.

mixmastamyk 10y ago

I remember that day vividly, I was in jr. high school. It's hard to describe how traumatic it was; the school teacher on board made it ten times worse. I would compare it to 9/11 for those that aren't old enough to remember. As we learned later about the rubber o-rings failing in cold weather, the solution has always been framed as not launching in those conditions. But, why not use a different material, or design away the o-rings to avoid the problem in the first place? Edit: question is already answered here: https://news.ycombinator.com/item?id=13239241 https://news.ycombinator.com/item?id=13239241

iamatworknow 10y ago
>I remember that day vividly, I was in jr. high school. It's hard to describe how traumatic it was; the school teacher on board made it ten times worse. I would compare it to 9/11 for those that aren't old enough to remember. It's interesting (albeit depressing) to think about how every generation in the television (and now internet) age seems to have at least one of these events that occurs right on the border of adulthood; when you've become old enough to have a sense of the world and the personal impact of the event kind of jolts you into reality, so to speak. For my dad it was JFK's assassination and for me it was 9/11. It's also notable that when Columbia disintegrated upon re-entry in 2003 the media and public at large didn't seem to pay it much attention at all (or at least I don't remember it being such a big deal).
- dispose13432 10y ago
  >It's also notable that when Columbia disintegrated upon re-entry in 2003 the media and public at large didn't seem to pay it much attention at all (or at least I don't remember it being such a big deal). Probably because it wasn't broadcast live.
  - sharksauce 10y ago
    It certainly was broadcast live -- every major TV news organization in the US broadcast the re-entry. Of course Columbia was very high altitude when it broke apart and the cameras (fortunately in my opinion) couldn't get very close-in shots. There's a point to be made that people were more prepared for a possible disaster given the reports of damage to the shuttle that the crew provided days ahead, compared to the absolute surprise and shock at Challenger, and the horrifyingly clear camera footage provided at a lower altitude.

keithwinstein 10y ago

The Challenger explosion is a great case study. But in focusing on that chart from the Rogers Commission report, this piece reinforces what is basically a well-told fable about the Challenger disaster. The piece says: "Below is the key graph of the O-ring test data that NASA analyzed before launch" and reproduces the famous chart. It continues, "NASA management used the data behind this first graph (among many other pieces of information) to justify their view the night before launch that there was no temperature effect on O-ring performance [...] But NASA management made one catastrophic mistake: this was not that chart they should have been looking at." I think these statements are pretty misleading without some major caveats. Tufte ("Visual and Statistical Thinking: Displays of Evidence for Making Decisions"; https://blogs.stockton.edu/hist4690/files/2012/06/Edward-Tufte-Visual-and-Statistical-Thinking.pdf https://blogs.stockton.edu/hist4690/files/2012/06/Edward-Tuf...) writes: "Most accounts of the Challenger reproduce a scatterplot that apparently demonstrates the analytical failure of the pre-launch debate. The graph depicts only launches with O-ring damage and their temperatures, omitting all damage-free launches (an absence of data points on the line of zero incidents of damage). First published in the shuttle commission report (PCSSCA, volume 1, 146), the chart is a favorite of statistics teachers. [...] The graph of the missing data-points is a vivid and poignant object lesson in how not to look at data when making an important decision. But it is too good to be true! First, the graph was not part of the pre-launch debate; it was not among the 13 charts used by Thiokol and NASA in deciding to launch. Rather, it was drawn after the accident by two staff members (the executive director and a lawyer) at the commission as their simulation of the poor reasoning in the pre-launch debate. Second, the graph implies that the pre-launch analysis examined 7 launches at 7 temperatures with 7 damage measurements. That is not true; only 2 cases of blow-by and 2 temperatures were linked up. The actual pre-launch analysis was much thinner than indicated by the commission scatterplot. Third, the damage scale is dequantified, only counting the number of incidents rather than measuring their severity. In short, whether for teaching statistics or for seeking to understand the practice of data graphics, why use an inaccurately simulated post-launch chart when we have the genuine 13 pre-launch decision charts right in hand?" (For a response to Tufte's essay, see https://people.rit.edu/wlrgsh/FINRobison.pdf https://people.rit.edu/wlrgsh/FINRobison.pdf, also cited elsewhere here.)

aidenn0 10y ago

babesh 10y ago

Institutional failure: don't forget the second shuttle disaster on reentry (Columbia) with the known issues with tiles getting knocked off by foam. Badly engineered contraption.

joering2 10y ago

Just so we on the same page - if you read enough about this sad story, you know damn well it wasn't an explosion or the faulty/frozen O-ring that killed those brave souls - it was a horrible amount of bureaucracy and ignorance towards engineers, who rang warning bells long before Challenger's liftoff.

skmurphy 10y ago

The full data chart reminders me of Abraham Wald asking the question "where do we never see damage on a returning plane?" to understand that those areas meant complete loss of the aircraft and would most benefit from more armor.

amoruso 10y ago

I'll add another recommendation for Tufte's writings on the Challenger explosion. It should be required reading for all engineers. People who criticize Tufte for oversimplifying miss the point entirely. It's not about analysis, it's about communication. It's one thing for domain specialist to have a complex, multi-dimensional understanding of their specialty; extracting a relevant summary for non-experts is something else entirely. If you've ever been in a meeting where you had trouble getting your point across, you should read this. Make diagrams like Tufte does to get your point across, make more detailed ones as backups if you need to dive deep into details. http://williamwolff.org/wp-content/uploads/2013/01/tufte-challenger-1997.pdf http://williamwolff.org/wp-content/uploads/2013/01/tufte-cha...

7952 10y ago
I think there is a tendancy to trivialise the difficulty of communication on large projects. I have had people tell me that "it is not rocket science". Well actually, maybe it is much harder than that. A small team can design a rocket engine. Getting a small team of managers to know all the right facts is very hard and on many projects seemingly impossible. And that is on projects where you can have very large margins of error. Obviously that is not always possible building things with strict mass limits.

rietta 10y ago

The Challenger explosion is one of my first vivid memories from my childhood as my mom was letting me watch the launch on TV as a 3 year old. I also had an understanding of death at the time as I asked my mom if they were in heaven (which was as advanced as my understanding was at the time). This is a fascinating read 30 years later.