15 ms·
I think the last paragraph really says it all: > Before the very first shuttle flight, NASA estimated that the chance of death was between 1 in 500 and 1 in 5,
by citilife 8y ago
I think the last paragraph really says it all:
> Before the very first shuttle flight, NASA estimated that the chance of death was between 1 in 500 and 1 in 5,000. Later, after the agency had compiled data from shuttle flights, it went back and came up with a very different number. The chance of death was actually 1 in 12.
The fact is, these numbers are made up because the systems are too complicated to calculate based on individual components. Until they have something like 50+ launches (after the kinks are worked out), we wont really know the actual chances of blowing up a rocket are. Further, how well do the escape capsule(s) work? If they work the majority of the time (saving astronauts from he explosion), that'd also dramatically drive down the risk.
If say, there was a 1 in 30 chance of a rocket exploding using the method described, but the escape capsule failed to save the astronauts 1 out of 10 times. Then the real risk of killing the astronauts is 1 in 300. Unfortunately, we simply don't have either of those numbers, so the actual "risk" level is essentially made up.
- jessaustin 8y agoSince the actual disaster record for the shuttle was 2/135, they must have done some other adjustments to come up with 1/12. Still, at this point it seems very likely that SpaceX will be safer than the shuttle.
- craftyguy 8y ago2/135 is failed flights/total flights. Over the life of the shuttle[0], there were 355 astronauts who flew, 14 died. So for every ~25 astronauts who flew on the shuttle, 1 died.. 0) https://www.space.com/12376-nasa-space-shuttle-program-facts-statistics.html https://www.space.com/12376-nasa-space-shuttle-program-facts...
- ebbv 8y agoOh my lord you guys that is not how probabilities work. You can’t look at what actually happened and then say “Well that was the chance.” If I flip a coin 20 times and happen to end up with a 15:5 split that doesn’t mean the chance of heads is 3:1. Also the context of that 1 in 12 in the quote is important. They’re saying that was the chance on the first flight in retrospect. The shuttle and it’s supporting facilities and processes didn’t stay static, they improved over time.
- craftyguy 8y agoI didn't say it was a probability. Quite literally, if you look at how many people flew and how many of those died, 1 in 25 died. There was a 100% 'chance' of 1 in 25 dying on the shuttle because that's what actually happened. There's no probability there.
- drb91 8y agoWait, so a ratio by itself is meaningless without showing how the ratio was derived? If only we could verify the 1/12!
- whatshisface 8y agoIf statistics didn't converge to probabilities in the high-n limit then science would be impossible. In the 15:5 coin example, the conclusion is skewed by a massive prior that coins are fair. When we're talking about things like space shuttle launches, it's not reasonable to expect our understanding of the system to be so good that we can boldly forge through with estimates that disagree with results.
- joshuamorton 8y ago>Oh my lord you guys that is not how probabilities work. You can’t look at what actually happened and then say “Well that was the chance.” If I flip a coin 20 times and happen to end up with a 15:5 split that doesn’t mean the chance of heads is 3:1. Erm, over a large sample size, yes this is exactly how things work. Sure 15/5 may be too small, but 150/50 isn't. That's well after the point where its reasonable to believe that your coin is rigged. As the other user mentioned, if you don't have the prior that the coin is fair, a 15/5 outcome may (and does) indeed imply that heads are more likely than tails. In fact, 15/5 would imply a less than 1% chance of a fair coin, given uniform priors. (Beta[15, 5] -> 1st percentile is 50.175)
- pps43 8y agoBut in this case the samples are small. If you flip a coin once and got tails, what is the probability of heads?
- TangoTrotFox 8y agoTake an example of why this number is probably not as meaningful as you might think. We have only one astronaut. And this astronaut flies on a ship 1000 times. And then on the 1000th time it blows up. Well we had 1 astronaut that flew and he died. So it would be fair to say that 100% of astronauts that flew on our imaginary ship died on it, but it would lack much of any meaning. Accounting for repeats, there were 833 individuals that flew on the shuttle and 14 died. The ratio there is about 60:1 which is predictably comparable to the ratio of successful:failed flights.
- delbel 8y agoThe Challenger Crew somehow died and also they are actually still alive. This is probably due to a timeline shift, for example there is no "Jiffy" peanut butter in this timeline, aka Mandela effect. Unfortunately we do not know what is causing this yet. https://1.bp.blogspot.com/-8NO8Dkx_yY8/V5loXQBvWkI/AAAAAAAIaRk/LZurjZJp5wcYYjwVEVixFJ39mMiU-83lQCLcB/s640/13654201_1288270541184295_8806315802829802667_n.jpg https://1.bp.blogspot.com/-8NO8Dkx_yY8/V5loXQBvWkI/AAAAAAAIa...
- jacquesm 8y agoThat's one very twisted way of computing the chance of dying on the shuttle. If you launched the same crew over and over again they would all have died. If you used 'fresh' people for every launch then the chance according to your computation would have been lower. And yet, through all of that the underlying machinery would be identical and that is the thing we're trying to evaluate here. So even though you can compute that particular number it is not a very useful number to calculate. A better way would be to look at the 2/135 number to determine vehicle safety and to use the number of astronauts on board to get to a generic figure and for each individual astronaut to use the 2/135 multiplied by the number of flights they took to figure out their chances. Of course these are still very low numbers and the error bars will therefore be large.
- jimrandomh 8y agoThe 1/12 failure chance was for the first flight, which is inherently riskier than subsequent flights.
- Robotbeat 8y agoThe first flights were more risky than later ones. They made improvements. But there were many close calls (including main engine failure). But SpaceX necessarily WILL be safer because they're using a proven vehicle and a more robust capsule. Plus a launch abort system. Shuttle flew crewed on the very first launch.
- MPSimmons 8y agoThere's also the concept of the "Drift into Failure", which I believe was coined by Sydney Dekker (https://www.amazon.com/Drift-into-Failure-Components-Understanding/dp/1409422216 https://www.amazon.com/Drift-into-Failure-Components-Underst...), where small compromises in operations eventually cause very important security and reliability protections to be eroded away.
- Robotbeat 8y agoi tend to think this is a strength of the Russian R7/Soyuz: the vast majority of the R7 launch family launches are uncrewed (and with a lower level of "mission assurance," i.e. less double-checking), therefore any systemic problems (i.e. due to "drift into failure," etc) are almost certainly going to be caught in an uncrewed launch first. Likewise, uncrewed Progress cargo spacecraft missions use a very similar platform as the Soyuz spacecraft, I think on the same line, so problems with the spacecraft can also be found on uncrewed missions. Falcon 9 is planned to launch maybe twice a year crewed, compared to 20 or 30 uncrewed launches (including like 2 or 3 uncrewed Dragon launches), meaning any new systemic problems with the launch vehicle will almost certainly pop up in an uncrewed launch before they result in loss of crew. Shuttle ONLY launched uncrewed, therefore any systemic problem which resulted in a loss of vehicle necessarily resulted in loss-of-crew.
- fao_ 8y ago> Shuttle ONLY launched uncrewed, therefore any systemic problem which resulted in a loss of vehicle necessarily resulted in loss-of-crew. Don't you mean that shuttle only launched crewed? Either that or I think I do not have the same definition as you do for "uncrewed" -- I take it to mean that there is no crew on board?
- pizza 8y agoWaiting for an accident to occur before determining that the modifications that allowed it to happen were unsafe -- because the estimates are "essentially made up" -- is just naive.
- marcosdumay 8y agoOn practice you will want some independent protection systems, and fix problems as they appear on each of those. On the real world, what is naive is expecting you did in fact know all the failure modes beforehand. The entire problem with rockets is that you can't fit many such systems into them. So the accidents happen.
- afterburner 8y ago> If say, there was a 1 in 30 chance of a rocket exploding using the method described, but the escape capsule failed to save the astronauts 1 out of 10 times. Then the real risk of killing the astronauts is 1 in 300. That's the same kind of mistake NASA made, just multiplying stuff like it was a high school textbook question. What if the very things that make a rocket explode also make the escape capsule fail?
- Retric 8y agoEscape capsules are not used when the rocket is ok. So, 1 in 10 chance of failure would likely include dammage from an exploding rocket. NASA's problem was their two failures did not result from any one component of the original design failing. One was used out of spec and the foam had been changed from the initial design.
- EthanHeilman 8y ago>Escape capsules are not used when the rocket is ok. So, 1 in 10 chance of failure would likely include dammage from an exploding rocket. I agree that a very effective escape system, as it appears the f9 has, makes the rocket far safer. However understanding failure probabilities is much complex than this thread is making it sound. For instance you could have a escape system failure that triggers the exploding rocket or a software system failure which causes both the rocket to explode and the escape system to fail. Take a look at 'Engineering a safer world' for an idea of how NASA and other parties actually calculate these probabilities: https://mitpress.mit.edu/books/engineering-safer-world https://mitpress.mit.edu/books/engineering-safer-world
- Retric 8y agoI think you missed my point. If you have the actual probabilities, which include cascade failures and one of the systems is a safety backup system not in normal use then multiplication is reasonably though not completely accurate. Though, you do need to consider the failure of the safety system when the normal system is operating correctly, but that's also much easier to avoid so likely very very low. Cascade failures are a far more complex situation and thus harder to predict. So again, it depends on what the failure rate is describing not just what the number is.
- garmaine 8y ago> The fact is, these numbers are made up because the systems are too complicated to calculate based on individual components No, you can perfectly well calculate component-by-component and come up with the 1 in 500 number. That's what NASA did (at great expense). What they failed to realize was this was only considering known-unknowns, and actual experience using the stack is required to learn about the unknown-unknowns which dominated the analysis.
- cjslep 8y agoThis response deserves to be listed higher. I participated in risk assessment analyses, specifically for nuclear powerplants, specifically for extremely rare wind events. The nuclear industry is like the space one and has had to learn about unknown unknowns the hard way.
- garmaine 8y agoYup. It comes up in other contexts too. In project risk assessment (“will we deliver on time?”) it is known as the inside view vs the outside view. The inside view is: here are all the parts we need to make, and how long each creation and integration step by itself should take, with known risks factored in. The outside view is: ok but the last 3 times we did something on this scale it took 2+ years when we thought it would take just 6 months. You can use an inside-view analysis to get a minimum risk assessment. Something would have been very wrong in the analysis if we didn’t lose 1 in 500, since that’s the risk we expected from known factors. And the outside view can give you a reasonable handle on maximums: objectively the risk of shuttle loss was about 1:75 or so, because that is what actually happened. If we were to have continued the shuttle program instead of cannibalizing it for the now defunct Constellation, we could have confidence that failures going forward would be at least 1:75 or better as we fix things, assuming regressions are rare. The failure by NASA was in treating the inside view known-unknowns risk assessment as a maximum (“risk of loss is NO MORE THAN...”) when it was really a minimum (“risk of loss is AT LEAST...”).
- Karishma1234 8y agoYes. This is a great reply. I believe in such super complicated systems there will be a long list of tail risks, probably too hard to compute. Also at this stage of space exploration we should be focusing on rapid development of technology while keeping humans safe but not putting too much emphasis on safety that we might want to see in our cars.
- TrainedMonkey 8y agoAgree with point on rapid development. Engineering is an iterative process, trying to come up with a perfect system is an anti-pattern. Make something that works and then never stop improving it. Russian Soyuz, SpaceX Merlin, iPhone, ICE, and microprocessors are all examples of where you can get with iterative approach.
- cagenut 8y agoeasy for you (or I) to say
- phkahler 8y agoOn a related note, I don't see how BFS atop BFR would have that escape capability. OTOH they seem to be aiming for reliability rather than figuring out how to fail safely.
- Balgair 8y agoI used to do reliability analyses for a large aerospace contractor, I may have some insights. The stuff I worked on was not HSF (Human Space Flight) just automated sats and the like. Each and every single part is spoken for and tallied. Each resistor on a PCB, each explosive bolt, each screw and panel. All of it has a lifetime curve, typically a Bathtub plot, to an extreme degree. Resistors were tested in vast batches, over vast I-V ranges, in hard radiation and vacuum, in vibe, for long periods. You calculate it all out. Something approaching the far upper reaches of the expected 'black swans' is tested and the part is proofed against it. Our stuff was just comms and the like, granted, but even then, the expected lifetime of each satellite was fairly well known out to about 20 years in orbit. A typical figure for a 15 year mission was a 75% chance of it making it the whole way, including launch. It's checked to an extreme degree. I've had people come back and correct an un-dotted 'i' symbol in a handwritten notebook. Really. HSF is easily 100x higher tolerances. Some stuff, you have to live with, some stuff (like the panels and the o-rings) is too far fetched for even NASA to think up beforehand. And if you've ever been to The Cape, you know they will never forgive themselves. They try their very best to think it all up. They really really do. NASA may be full of brains, but damned if they aren't all real heroes there on the ground. Those folk have heart more than anything else. Per Aspera Ad Astra does not fucking cut it.
- sitkack 8y ago> o-rings Were a known weakness.
- CamperBob2 8y agoExactly. That's the last example anyone should cite when arguing that the process has the interests of safe human spaceflight in mind.
- f4rker 8y agoIt's ok facts don't matter
- Balgair 8y agoMy mistake! Thanks for the correction!
- deleted 8y ago[deleted]
- dvdkhlng 8y agoRelated video: Video of Soyuz launch escape system used in a real emergency: https://www.youtube.com/watch?v=UyFF4cpMVag https://www.youtube.com/watch?v=UyFF4cpMVag This is probably how SpaceX imagines astronauts would be saved in case of a fueling problem.
- tim333 8y agoIt reminds me a bit of Feynman investigating the Challenger disaster: >Engineers at marshal estimate it as 1/300, while NASA management, to whom these engineers report, claims it is 1/100,000. An independent engineer consulting for NASA thought 1 or 2 per 100 a reasonable estimate The estimates seem to depend on who is estimating https://science.ksc.nasa.gov/shuttle/missions/51-l/docs/rogers-commission/Appendix-F.txt https://science.ksc.nasa.gov/shuttle/missions/51-l/docs/roge... He had to fight publish that: >Feynman was so critical of flaws in NASA's "safety culture" that he threatened to remove his name from the report unless it included his personal observations on the reliability of the shuttle, which appeared as Appendix F.