6 ms·

I'm not saying NASA didn't fuck up by any means. I'd just like to point out that hindsight bias is a very real effect that can have huge consequences when plann

I'm not saying NASA didn't fuck up by any means. I'd just like to point out that hindsight bias is a very real effect that can have huge consequences when planning for future events: http://lesswrong.com/lw/il/hindsight_bias/ http://lesswrong.com/lw/il/hindsight_bias/ >Viewing history through the lens of hindsight, we vastly underestimate the cost of effective safety precautions. In 1986, the Challenger exploded for reasons traced to an O-ring losing flexibility at low temperature. There were warning signs of a problem with the O-rings. But preventing the Challenger disaster would have required, not attending to the problem with the O-rings, but attending to every warning sign which seemed as severe as the O-ring problem, without benefit of hindsight. It could have been done, but it would have required a general policy much more expensive than just fixing the O-Rings. >Shortly after September 11th 2001, I thought to myself, and now someone will turn up minor intelligence warnings of something-or-other, and then the hindsight will begin. Yes, I'm sure they had some minor warnings of an al Qaeda plot, but they probably also had minor warnings of mafia activity, nuclear material for sale, and an invasion from Mars. >Because we don't see the cost of a general policy, we learn overly specific lessons. After September 11th, the FAA prohibited box-cutters on airplanes—as if the problem had been the failure to take this particular "obvious" precaution. We don't learn the general lesson: the cost of effective caution is very high because you must attend to problems that are not as obvious now as past problems seem in hindsight.

yosefk 11y ago
The letter demonstrates foresight, not hindsight; a very long list of equally severe and plausible warnings is needed to make the point that one is unlikely to single out this one letter without the benefit of hindsight.
- jsprogrammer 11y ago
  Yes, there is a serious gap of logic in the linked article. In fact, it appears that if the O-Rings were better checked and improved, as the letter indicated they must be, the Challenger would not have exploded as it did.
- banku_brougham 11y ago
  This is the crux. This letter is clearly very ballsy in a corporate context, I doubt there is a stack of similar memos in the Morton Thiokol archives which if acted upon as a group would have bogged down the launch. But at a minimum, making this argument requires at least one similar example. Without that alternate case, languorous management is the most likely culprit.
VikingCoder 11y ago
I think your comment is great, but I have to disagree with one part in particular. I don't agree that it was a "some minor warning" when President Clinton told President Bush explicitly; "In his campaign, Bush had said he thought the biggest security issue was Iraq and a national missile defense. I told him that in my opinion, the biggest security problem was Osama bin Laden."
- njloof 11y ago
  Clearly Clinton was wrong; Iraq was the biggest security issue in America by the end of the Bush administration.
  - snowwrestler 11y ago
    Reminds of this great quote from Alan Kay: "The best way to predict the future is to invent it."
- yellowstuff 11y ago
  "Osama Bin Laden is dangerous" is pretty far from actionable intelligence, though. I think it's an example of hindsight bias to think that if President Bush had been more prudent in dealing with Bin Laden then the 9/11 attacks would have been prevented.
  - VikingCoder 11y ago
    Your paraphrase is one thing. When the current Commander in Chief of the United States of America, the most powerful military in the world, says "the biggest security problem [is] Osama bin Laden," that's something entirely different. And frankly, I trust the opinion of Richard Clarke more than I trust yours OR MINE.
    - Houshalter 11y ago
      Except after 9/11 he did say that, and sent the entire military after him. And it took 10 years to find him.
revelation 11y ago
How many letters to VPs and pre-launch holdups were there, then? Surely you can't make this argument without detailing that?
mikeash 11y ago
What were the other warning signs which seemed as severe as the O-ring problem without the benefit of hindsight? Your quote claims that it would have been much more expensive to address all of these things, but what's the support for that statement? My (extremely limited, to be sure) understanding is that there wasn't anything else that severe. The O-ring problem was dire, and the only reason it didn't get attention was because schedule pressure and go-fever caused management to minimize the problem. Note that the warning signs mentioned in the quote were not just theoretical test results, but also several cases where fire was getting where it was not supposed to be during actual flights. Further, there was strong evidence that these incidents were linked to cold weather. "Don't fly when it's really cold" wouldn't have been a particularly expensive mitigation, either, even if you argue that suspending flights until the problem was investigated and fixed was not reasonable. I mean, you're launching from Florida, a restriction not to launch in temperatures under 50F wouldn't have been a terrible burden. How many other warning sings were as severe as the O-rings and as easy to mitigate? My money is on "zero" in which case this argument is just so much noise.
- pdonis 11y ago
  > My (extremely limited, to be sure) understanding is that there wasn't anything else that severe. That's my understanding as well. Boisjoly wrote a detailed essay describing the thought process that they went through to arrive at their conclusion that the O-rings were a critical problem (the other three authors are a professor and two students at RIT): https://people.rit.edu/wlrgsh/FINRobison.pdf https://people.rit.edu/wlrgsh/FINRobison.pdf The essay is a response to a criticism by Edward Tufte that the engineers did not present the issue properly during the telcon the night before the launch, and that this was the reason that NASA did not agree to scrub the launch. It makes clear that, at least in the minds of the engineers, there was no other issue even close to being this critical. Granted, they were concerned with only one subsystem, the SRBs--but the fact that this issue also dominated the telcon the night before the launch indicates that there was no other issue even close to being that critical in any other subsystem either. [Edit: It's interesting, btw, that in the essay linked to above, Boisjoly accuses Tufte of precisely the thing Yudkowsky is talking about in his article: hindsight bias. He argues that Tufte did not recognize or allow for the fact that the engineers at the time had limited information, but talks as though they knew everything that was known in hindsight.]

yosefk 11y ago

The letter demonstrates foresight, not hindsight; a very long list of equally severe and plausible warnings is needed to make the point that one is unlikely to single out this one letter without the benefit of hindsight.

jsprogrammer 11y ago
Yes, there is a serious gap of logic in the linked article. In fact, it appears that if the O-Rings were better checked and improved, as the letter indicated they must be, the Challenger would not have exploded as it did.
banku_brougham 11y ago
This is the crux. This letter is clearly very ballsy in a corporate context, I doubt there is a stack of similar memos in the Morton Thiokol archives which if acted upon as a group would have bogged down the launch. But at a minimum, making this argument requires at least one similar example. Without that alternate case, languorous management is the most likely culprit.

VikingCoder 11y ago

I think your comment is great, but I have to disagree with one part in particular. I don't agree that it was a "some minor warning" when President Clinton told President Bush explicitly; "In his campaign, Bush had said he thought the biggest security issue was Iraq and a national missile defense. I told him that in my opinion, the biggest security problem was Osama bin Laden."

njloof 11y ago
Clearly Clinton was wrong; Iraq was the biggest security issue in America by the end of the Bush administration.
- snowwrestler 11y ago
  Reminds of this great quote from Alan Kay: "The best way to predict the future is to invent it."
yellowstuff 11y ago
"Osama Bin Laden is dangerous" is pretty far from actionable intelligence, though. I think it's an example of hindsight bias to think that if President Bush had been more prudent in dealing with Bin Laden then the 9/11 attacks would have been prevented.
- VikingCoder 11y ago
  Your paraphrase is one thing. When the current Commander in Chief of the United States of America, the most powerful military in the world, says "the biggest security problem [is] Osama bin Laden," that's something entirely different. And frankly, I trust the opinion of Richard Clarke more than I trust yours OR MINE.
  - Houshalter 11y ago
    Except after 9/11 he did say that, and sent the entire military after him. And it took 10 years to find him.

revelation 11y ago

How many letters to VPs and pre-launch holdups were there, then? Surely you can't make this argument without detailing that?

mikeash 11y ago

What were the other warning signs which seemed as severe as the O-ring problem without the benefit of hindsight? Your quote claims that it would have been much more expensive to address all of these things, but what's the support for that statement? My (extremely limited, to be sure) understanding is that there wasn't anything else that severe. The O-ring problem was dire, and the only reason it didn't get attention was because schedule pressure and go-fever caused management to minimize the problem. Note that the warning signs mentioned in the quote were not just theoretical test results, but also several cases where fire was getting where it was not supposed to be during actual flights. Further, there was strong evidence that these incidents were linked to cold weather. "Don't fly when it's really cold" wouldn't have been a particularly expensive mitigation, either, even if you argue that suspending flights until the problem was investigated and fixed was not reasonable. I mean, you're launching from Florida, a restriction not to launch in temperatures under 50F wouldn't have been a terrible burden. How many other warning sings were as severe as the O-rings and as easy to mitigate? My money is on "zero" in which case this argument is just so much noise.

pdonis 11y ago
> My (extremely limited, to be sure) understanding is that there wasn't anything else that severe. That's my understanding as well. Boisjoly wrote a detailed essay describing the thought process that they went through to arrive at their conclusion that the O-rings were a critical problem (the other three authors are a professor and two students at RIT): https://people.rit.edu/wlrgsh/FINRobison.pdf https://people.rit.edu/wlrgsh/FINRobison.pdf The essay is a response to a criticism by Edward Tufte that the engineers did not present the issue properly during the telcon the night before the launch, and that this was the reason that NASA did not agree to scrub the launch. It makes clear that, at least in the minds of the engineers, there was no other issue even close to being this critical. Granted, they were concerned with only one subsystem, the SRBs--but the fact that this issue also dominated the telcon the night before the launch indicates that there was no other issue even close to being that critical in any other subsystem either. [Edit: It's interesting, btw, that in the essay linked to above, Boisjoly accuses Tufte of precisely the thing Yudkowsky is talking about in his article: hindsight bias. He argues that Tufte did not recognize or allow for the fact that the engineers at the time had limited information, but talks as though they knew everything that was known in hindsight.]