4 ms·

Maybe, but I don't think anyone can deny that it is an appalling slide. I count 3 spelling and grammatical errors alone. It looks like the bad "before" example

by zelos 5y ago

Maybe, but I don't think anyone can deny that it is an appalling slide. I count 3 spelling and grammatical errors alone. It looks like the bad "before" example in a presentation skills workshop. This was created by engineers working on life and death issues involving billions of dollars of hardware.

bumby 5y ago
I agree with you, but to be charitable, this was a real-time and evolving situation where I'm willing to bet the slide was expected to be finished "yesterday"
- phicoh 5y ago
  This seems to be the kind of slide made by someone without a lot of experience presenting work. The other issue is, some people really, really, don't want to speculate. In this case it seems that the person who made the slide probably assumed that the tile could be broken with a high enough probility. But because it was outside all available data, the slide says that we don't really know. Of course, anybody in a position to make such a go-no-go decision should have enough experience talking to engineers, and seeing this effect in action to recognize the slide for what it is. It is really weird to conclude that based on absence of data, it is probably safe.
  - bumby 5y ago
    >It is really weird to conclude that based on absence of data, it is probably safe. Considering that's exactly what happened nearly 20 years earlier with Challenger, it seems to be more common and likely the result of a number of cognitive biases. We read these with some hindsight and are disconnected from all the other pressures (schedule, budget, peer, etc.) they are dealing with at the time.
    - mirker 5y ago
      One factor for why is that bringing bad news may poorly reflect on the organization, and therefore the person’s career.
      - bumby 5y ago
        It takes some intestinal fortitude to be in a role that is tasked with communicating information people don't want to hear. It's part of the reason NASA created it's "Safety and Mission Assurance" organization after this incident and gave them a completely different chain of command. In theory, that mitigates some of the career threat, but in practice it may be different.
    - phicoh 5y ago
      That points to a far more fundamental problem. Related to information processing higher up in the organisation. Just making better slides is unlikely to solve that problem.
      - bumby 5y ago
        Probably correct, and I have doubts that those types of problems are easily fixed because they're rooted in human psychology. It's interesting to me that the "big" incidents seem to occur every 15-20 years, almost as if there is a new professional cohort who has to learn the hard way. I do think clear communication is a necessary, but insufficient, element of fixing that problem.
        phicoh 5y ago
        One thing I wonder about with these kinds of accidents: to what extend does operational experience work its way back to requirements of components. For example, if regularly pieces of foam are hitting the tiles after launch, was that part of the specs for the tiles to handle that? Did anybody go back, take a worst case scenario of a piece foam hitting a tile (size, speed, etc.) and verify that the tiles could handle such an impact?
        bumby 5y ago
        They'll generally use a Failure Mode Effects Analysis (FMEA). So in this example, designers would identify all the ways a tile could fail and the consequence and probability of that failure. They then go through the process of mitigating it. The order of precedence for mitigations is 1) remove the hazard, 2) engineer around the hazard, 3) administrative controls (like standard procedures), 4) personal protective equipment. The iterate around this until the risk is within an acceptable range. All those mitigations become requirements. So let's say they identify a tile failure mode as "tile struck by object". They assign a worst-case severity to that. Let's say they knew how bad it could be and they assign a severity as "loss of crew." Then they have to identify all the ways the tile could be struck and assign probabilities to that even happening. They use a matrix that maps the severity and probability to arrive at a risk classification. If the classification is higher than their threshold, they add mitigations that either reduce the severity or the probability (or both) until it's within an acceptable risk range. There's lots that can go wrong with this process, though. You obviously have to be able to identify the failure modes. Is there some off-the-wall failure that nobody could foresee? Maybe. Then you have to have good enough data to objectively determine the risk. In this case, I wonder if all the previous foam strikes led them to discredit the risk as being improbable/negligible to cause that failure mode. Add to that, the PowerPoint seemed to imply the model they used is too conservative (it was believed to overestimate the actual penetration). I know people involved on some hypervelocity testing of the foam and they were legitimately surprised at the way the foam acted when it was fired at higher speeds. So in this case, the risk was probably unknown beforehand, although they assumed they understood the risk sufficiently. To quote Mark Twain, "What gets us into trouble is not what we don't know. It's what we know for sure that just ain't so." That's just one system on an immensely complex machine. It's easy to sit back with hindsight and say "Well, they shouldn't have made a decision until they did additional testing to get the data." But if they did that to every system on the Shuttle, it likely wouldn't have left the ground. In practice, engineers deal with all kinds of other cost and schedule constraints.
deleted 5y ago
[deleted]
cxr 5y ago
> Maybe, but I don't think anyone can deny that it is an appalling slide. I count 3 spelling and grammatical errors alone. The slide from the post that you're referring to (which is 16:9) is a fabricated, probably created for this post, and is not a faithful copy of the actual deck that was presented in 2003. The actual slide—complete with a Boeing watermark, sans some of the errors and presentation issues we can see in the blog post, and in 4:3, of course—can be seen in Tufte's book. Who knows why the slide in this post was fabricated (and why the author failed to indicate this fact anywhere).

by zelos 5y ago

bumby 5y ago

I agree with you, but to be charitable, this was a real-time and evolving situation where I'm willing to bet the slide was expected to be finished "yesterday"

phicoh 5y ago
This seems to be the kind of slide made by someone without a lot of experience presenting work. The other issue is, some people really, really, don't want to speculate. In this case it seems that the person who made the slide probably assumed that the tile could be broken with a high enough probility. But because it was outside all available data, the slide says that we don't really know. Of course, anybody in a position to make such a go-no-go decision should have enough experience talking to engineers, and seeing this effect in action to recognize the slide for what it is. It is really weird to conclude that based on absence of data, it is probably safe.
- bumby 5y ago
  >It is really weird to conclude that based on absence of data, it is probably safe. Considering that's exactly what happened nearly 20 years earlier with Challenger, it seems to be more common and likely the result of a number of cognitive biases. We read these with some hindsight and are disconnected from all the other pressures (schedule, budget, peer, etc.) they are dealing with at the time.
  - mirker 5y ago
    One factor for why is that bringing bad news may poorly reflect on the organization, and therefore the person’s career.
    - bumby 5y ago
      It takes some intestinal fortitude to be in a role that is tasked with communicating information people don't want to hear. It's part of the reason NASA created it's "Safety and Mission Assurance" organization after this incident and gave them a completely different chain of command. In theory, that mitigates some of the career threat, but in practice it may be different.
  - phicoh 5y ago
    That points to a far more fundamental problem. Related to information processing higher up in the organisation. Just making better slides is unlikely to solve that problem.
    - bumby 5y ago
      Probably correct, and I have doubts that those types of problems are easily fixed because they're rooted in human psychology. It's interesting to me that the "big" incidents seem to occur every 15-20 years, almost as if there is a new professional cohort who has to learn the hard way. I do think clear communication is a necessary, but insufficient, element of fixing that problem.
      - phicoh 5y ago
        One thing I wonder about with these kinds of accidents: to what extend does operational experience work its way back to requirements of components. For example, if regularly pieces of foam are hitting the tiles after launch, was that part of the specs for the tiles to handle that? Did anybody go back, take a worst case scenario of a piece foam hitting a tile (size, speed, etc.) and verify that the tiles could handle such an impact?
        bumby 5y ago
        They'll generally use a Failure Mode Effects Analysis (FMEA). So in this example, designers would identify all the ways a tile could fail and the consequence and probability of that failure. They then go through the process of mitigating it. The order of precedence for mitigations is 1) remove the hazard, 2) engineer around the hazard, 3) administrative controls (like standard procedures), 4) personal protective equipment. The iterate around this until the risk is within an acceptable range. All those mitigations become requirements. So let's say they identify a tile failure mode as "tile struck by object". They assign a worst-case severity to that. Let's say they knew how bad it could be and they assign a severity as "loss of crew." Then they have to identify all the ways the tile could be struck and assign probabilities to that even happening. They use a matrix that maps the severity and probability to arrive at a risk classification. If the classification is higher than their threshold, they add mitigations that either reduce the severity or the probability (or both) until it's within an acceptable risk range. There's lots that can go wrong with this process, though. You obviously have to be able to identify the failure modes. Is there some off-the-wall failure that nobody could foresee? Maybe. Then you have to have good enough data to objectively determine the risk. In this case, I wonder if all the previous foam strikes led them to discredit the risk as being improbable/negligible to cause that failure mode. Add to that, the PowerPoint seemed to imply the model they used is too conservative (it was believed to overestimate the actual penetration). I know people involved on some hypervelocity testing of the foam and they were legitimately surprised at the way the foam acted when it was fired at higher speeds. So in this case, the risk was probably unknown beforehand, although they assumed they understood the risk sufficiently. To quote Mark Twain, "What gets us into trouble is not what we don't know. It's what we know for sure that just ain't so." That's just one system on an immensely complex machine. It's easy to sit back with hindsight and say "Well, they shouldn't have made a decision until they did additional testing to get the data." But if they did that to every system on the Shuttle, it likely wouldn't have left the ground. In practice, engineers deal with all kinds of other cost and schedule constraints.

deleted 5y ago

[deleted]

cxr 5y ago

> Maybe, but I don't think anyone can deny that it is an appalling slide. I count 3 spelling and grammatical errors alone. The slide from the post that you're referring to (which is 16:9) is a fabricated, probably created for this post, and is not a faithful copy of the actual deck that was presented in 2003. The actual slide—complete with a Boeing watermark, sans some of the errors and presentation issues we can see in the blog post, and in 4:3, of course—can be seen in Tufte's book. Who knows why the slide in this post was fabricated (and why the author failed to indicate this fact anywhere).