4 ms·
Eh, aerospace involves a lot more testing and specification and generally being careful in engineering practices and still failure for new products is expected.
by colechristensen 5y ago
Eh, aerospace involves a lot more testing and specification and generally being careful in engineering practices and still failure for new products is expected.
You launch your first few rockets expecting them to explode because even with all of the care, it's just not reasonable to expect that you got everything right in simulation, test, and design.
This is why the public response to "failures" from SpaceX, NASA, etc. etc. are so frustrating, people don't understand the nature of product testing. You don't get to see a washing machine or microchip or car or whatever else do its equivalent of exploding on the pad, but they all do and much worse.
The first flight of a rocket not exploding is like spending days writing code, compiling it once without errors and running it in production with no issues. It's basically unimaginable and when it does happen you sit and wonder what's actually going wrong that you don't know about yet.
Once you do get a working configuration, you don't change it or make only very safe changes. You rely on what's worked before because things are actually exactly the same, not constantly changing like so much software today. This leads to a lot of conservatism that people don't understand (why is military hardware using X, it's so outdated!)
- bumby 5y agoI think this is a fair take when you're talking about failures of nascent designs because of unknown failure modes. But it's frustrating when there are failures from known failure modes that just had process escapes. A couple examples: - CST-100 was sent into orbit with software mapped to valves incorrectly. There is no credible reason this shouldn't have been tested thoroughly on the ground first. - Falcon 9 had a failure due to a tank support strut that had strength characteristics a fraction of its design spec. Good supplier quality control should have vetted the material prior to its use. (SpaceX has since added material tests). Finding unknown causes of failures help us learn more about the science of the industry. But not catching known failure modes due to lack of process control is a different animal.
- hn_throwaway_99 5y ago> But not catching known failure modes due to lack of process control is a different animal. I agree, but for any complicated product like a rocket launch there are probably many many tens of thousands/hundreds of thousands/millions of things like this where one small uncaught error can cause mission failure. At some point it's just statistics, you'd expect to see some catastrophic failures across some percentage of products early in their lifecycle, regardless of how good your process control is.
- bumby 5y agoYes, there is reliability analysis involved which invokes probability. However, that's not the case with the items I mentioned above. It would be one thing if SpaceX tested the material coupon and found it within specs, but by sheer probability, the strut had a portion of its material structure that was anomalous. That's what you're talking about. What I'm talking about is there are specific process steps that would have caught those mistakes above, and those processes were just not performed. Neither of the cases mentioned above are the result of "We did everything right, but the probability gods were just not on our side."
- hn_throwaway_99 5y ago> It would be one thing if SpaceX tested the material coupon and found it within specs, but by sheer probability, the strut had a portion of its material structure that was anomalous. That's what you're talking about. Actually, that's not what I'm talking about. What I'm talking about is lessons from QA processes and the chance that any particular bug escapes from one QA process to the next. The idea being, suppose you have: * one million different "components" in a system * each of those components have, say, 10 possible failure modes * There are 100 of these new complex products a year. Obviously the numbers above are all made up, but the idea would be in that example you'd have a billion (1,000,000 X 10 X 100) possible failure modes. Any QA filter, might be expected to catch, say, 95% of remaining bugs. So you run as many filters as you can, but at some point you're still left with the small probability for new bugs, and they only way to remediate those bugs is to figure out what went wrong after the fact.
- bumby 5y agoNot every bug creates a failure, so there's not necessarily a need to be completely bug-free. The point is to focus efforts on the bugs that can cause failure and design mitigations for them. If your system has a billion potential software failure modes, my hope is that a project manager will say the software is too complex and needs to be redesigned. I think this is where some in the software crowd get it wrong in their understanding of how software reliability in safety-critical applications is performed. There are a lot of project managers who deliberately avoid using software as a mitigation because of the complexity issue (see the 737Max use of software as a mitigation for a good example of where this practice can go wrong). So take the software example from above on CST-100. The valve mapping error should have been a known failure mode ("We command valve A and it doesn't respond." or "We command valve X and valve A responds"). Those failures are very straightforward to test for with simulations on the ground, yet somehow those errors made it into a flight configuration. Arguably, the software timer issue on that flight falls into a similar category, but the mitigations there are a bit more complex.
- deleted 5y ago[deleted]
- Bayart 5y ago> It's basically unimaginable and when it does happen you sit and wonder what's actually going wrong that you don't know about yet. Every time that happens I'm terrified and overcome with a quiet, pervasive sense of existential dread.