5 ms·

The article suggests using a probabilistic failure model instead of a large a safety factor, and explains how a safety factor established in the 1930s affected

The article suggests using a probabilistic failure model instead of a large a safety factor, and explains how a safety factor established in the 1930s affected the cost of the Space Shuttle. But spacecraft might be a special case, where any additional weight is so expensive, and you also expect the models to be especially accurate and manufacturing to be extremely precise. For more everyday civil engineering, I think the safety factor "covers up" a lot of systemic inaccuracies everywhere in the system, from modeling to design to manufacturing to unintended uses. Some of those you might account for in a probabilistic model. But it's very difficult to probabilistically model errors in the model itself, as the financial industry found out the hard way. When driving over a bridge built the way suggested here, how comfortable can we be that certain stresses aren't correlated in ways that the engineers didn't anticipate? Or that a certain distribution isn't actually as well approximated by a Gaussian as it was assumed to be? Intuitively, it's a lot harder to be wildly wrong with the factor of safety approach. To put this another way, a more complex way to reason about safety necessarily has more moving parts, and is thus more likely to be wrong. So in effect, adopting more complicated safety models introduces a safety risk all on its own. I think that needs to be considered as well.

warrenm 5y ago
Did you get to the end of the article? He addresses this: >A bigger issue, and the one I think has prevented more widespread adoption, is that probabilistic design doesn’t account for fluke events -- the unknowables. If you don’t know what could happen, you obviously can’t assign that event a probability. >The ideal approach might be a hybrid. Probabilistic design could be responsible for covering simplifications and a reduced safety factor could cover the unknowables. Of course, there’s no simple way to determine how much of the current factor covers simplifications, so reducing the factor would still be a risky endeavor.
wiredfool 5y ago
I'll go out on a pretty small limb and say that the vast majority of Civil Engineering failings are not a matter of an incorrect safety factor, but are things that are explicitly not part of it. 1) Blunders. (Many places. You do the math wrong, or approve the wrong shop drawing, and no factor of safety is going to save you). (See the Hyatt Regency Walkway Failure) 2) Inadequate Geotech Info. (Basically every dam failure ever) 3) Genuinely new behavior. (Tacoma Narrows) 4) Contractors. (I-90 Bridge Sinking) 5) Deferred Maintenance. (Fatigue on bridges, Minneapolis)
- ghaff 5y ago
  Agreed. Although I'd probably also argue that a safety factor probably papers over those kind of problems in many many cases.
  - londons_explore 5y ago
    But the question is, if you used a probabilistic approach, and tried to model, even very roughly, those things (probability contractor bodges the job in a way inspection doesn't notice: 10%), then would you end up with a safer bridge for the same money spent?
    - ghaff 5y ago
      It feels like you're picking numbers out of the air (or basing them on historical experience) either way. Unless you actually have historical data on certain types of problems--but then it seems like you're pretty much back to a safety/fudge factor.
      - londons_explore 5y ago
        But a large number of guestimated fudge factors all added up will approach the true value as long as there is no bias. The same does not apply for factors of safety - a 1.5 FoS is always between 0 and 50% too much.
        deleted 5y ago
        [deleted]
    - wiredfool 5y ago
      That can be an unbounded black swan event. There are distributions where there is no mean value. The difference between a square section and welded channels. The difference between 53F and 27F. The difference between putting the waterproofing on before or after the post tensioning anchors. Leaving watertight doors open during a storm.
- 5y ago
lamontcg 5y ago
Or more simply: planes that get hit with massive unexpected turbulence shouldn't just drop out of the sky and kill everyone on board. And to paraphrase the great philosopher Donald Rumsfeld, it is all about the unknown-unknowns.
- saalweachter 5y ago
  Any plane that can land missing half a wing on one engine is over-engineered, from a certain point of view.
  - warrenm 5y ago
    So... you'd rather it crash and kill everyone onboard?
  - bfrog 5y ago
    Unless it's meant to take that sort of damage and survive intentionally due to its usage such as the A-10
- benhurmarcel 5y ago
  Massive turbulence is taken into account in the loads calculation (JAR 25.341). It's not supposed to be in the safety factor.
WalterBright 5y ago
Aircraft have a smallish safety factor, because of the weight. They make up for it with much more careful design, manufacture, and maintenance along with frequent inspections.
- mturmon 5y ago
  Yes. I linked below a slide set by a Lee Petersen, JPL engineer who was instrumental in setting up the uncertainty accounting for the Mars lander EDL system - both Curiosity and Perseverance. (Among other uses of UQ for engineered systems at JPL.) In the slides he’s deliberately contrasting a “MUF” (model uncertainty factor) approach that uses safety factors, possibly stacked as indicated in the OP, versus a “BE+U” (best estimate plus uncertainty) approach. The latter uses verified physics-based models, validation of model predictions versus experimental tests, and uncertainty quantification of sources of error, to get a best estimate of whatever quantity is critical to safe operation, and an uncertainty. A final margin can then be added onto that, see slide 13. The mess at the system level that can result from a stack of ad hoc MUFs applied at the subsystem level is shown on slide 15. Incidentally, many of the comments nearby seem to think the application of domain specific safety factors is still state of the art. This just isn’t true any more for high risk systems. https://cstools.asme.org/csconnect/Filedownload.cfm?thisfile=39465.pdf&dir=CommitteeFiles&44348.0865972 https://cstools.asme.org/csconnect/Filedownload.cfm?thisfile...
abduhl 5y ago
Most modern bridges you drive over are designed the probabilistic way that is suggested in the article. Bridge design followed vertical construction as material science and manufacturing got better for steel and concrete. The probabilistic approaches haven’t been adopted much in engineering fields that deal with too many unknowns. I’m actually incredibly surprised that there was no mention of the technical terms for these approaches: Allowable Stress Design, Load and Resistance Factor Design, and Yeah That Looks Right Design. LRFD is highly probability and materials testing based. ASD is a hybrid approach of old factors of safety and some probabilistic theory. YTLRD is based on the long and storied history of guys who have been doing it this way since before you were born, no matter when that was.
- wiredfool 5y ago
  And they're all used, to some extent. If your Wizzy design based on the latest everything doesn't pass the grumpy old partner's YTLRD review, you're going to redo it till you do. (In my case, that was Bill. He was one of those guys who knew where to put the $50k mark)
justincredible 5y ago
I don't think the probabilistic failure model is suggested, merely presented. The author concludes that departing from the stated safety factor is necessarily taking on additional risk.
whack 5y ago
> The article suggests using a probabilistic failure model instead of a large a safety factor That wasn't my takeaway from the article, and the article discusses the same problems you've brought up. To quote: > On the flip side, it requires more information as well. There must be good data or the result will be as arbitrary as the factor of safety, without the benefit of decades of experience... A bigger issue, and the one I think has prevented more widespread adoption, is that probabilistic design doesn’t account for fluke events -- the unknowables. If you don’t know what could happen, you obviously can’t assign that event a probability... The ideal approach might be a hybrid. Probabilistic design could be responsible for covering simplifications and a reduced safety factor could cover the unknowables. Of course, there’s no simple way to determine how much of the current factor covers simplifications, so reducing the factor would still be a risky endeavor... For my projects, I intend to embrace the empirical nature of safety factors and not think too hard about it. If a factor already exists for the area I’m exploring, I’ll use that

warrenm 5y ago

Did you get to the end of the article? He addresses this: >A bigger issue, and the one I think has prevented more widespread adoption, is that probabilistic design doesn’t account for fluke events -- the unknowables. If you don’t know what could happen, you obviously can’t assign that event a probability. >The ideal approach might be a hybrid. Probabilistic design could be responsible for covering simplifications and a reduced safety factor could cover the unknowables. Of course, there’s no simple way to determine how much of the current factor covers simplifications, so reducing the factor would still be a risky endeavor.

wiredfool 5y ago

I'll go out on a pretty small limb and say that the vast majority of Civil Engineering failings are not a matter of an incorrect safety factor, but are things that are explicitly not part of it. 1) Blunders. (Many places. You do the math wrong, or approve the wrong shop drawing, and no factor of safety is going to save you). (See the Hyatt Regency Walkway Failure) 2) Inadequate Geotech Info. (Basically every dam failure ever) 3) Genuinely new behavior. (Tacoma Narrows) 4) Contractors. (I-90 Bridge Sinking) 5) Deferred Maintenance. (Fatigue on bridges, Minneapolis)

ghaff 5y ago
Agreed. Although I'd probably also argue that a safety factor probably papers over those kind of problems in many many cases.
- londons_explore 5y ago
  But the question is, if you used a probabilistic approach, and tried to model, even very roughly, those things (probability contractor bodges the job in a way inspection doesn't notice: 10%), then would you end up with a safer bridge for the same money spent?
  - ghaff 5y ago
    It feels like you're picking numbers out of the air (or basing them on historical experience) either way. Unless you actually have historical data on certain types of problems--but then it seems like you're pretty much back to a safety/fudge factor.
    - londons_explore 5y ago
      But a large number of guestimated fudge factors all added up will approach the true value as long as there is no bias. The same does not apply for factors of safety - a 1.5 FoS is always between 0 and 50% too much.
      - deleted 5y ago
        [deleted]
  - wiredfool 5y ago
    That can be an unbounded black swan event. There are distributions where there is no mean value. The difference between a square section and welded channels. The difference between 53F and 27F. The difference between putting the waterproofing on before or after the post tensioning anchors. Leaving watertight doors open during a storm.
5y ago

lamontcg 5y ago

Or more simply: planes that get hit with massive unexpected turbulence shouldn't just drop out of the sky and kill everyone on board. And to paraphrase the great philosopher Donald Rumsfeld, it is all about the unknown-unknowns.

saalweachter 5y ago
Any plane that can land missing half a wing on one engine is over-engineered, from a certain point of view.
- warrenm 5y ago
  So... you'd rather it crash and kill everyone onboard?
- bfrog 5y ago
  Unless it's meant to take that sort of damage and survive intentionally due to its usage such as the A-10
benhurmarcel 5y ago
Massive turbulence is taken into account in the loads calculation (JAR 25.341). It's not supposed to be in the safety factor.

WalterBright 5y ago

Aircraft have a smallish safety factor, because of the weight. They make up for it with much more careful design, manufacture, and maintenance along with frequent inspections.

mturmon 5y ago
Yes. I linked below a slide set by a Lee Petersen, JPL engineer who was instrumental in setting up the uncertainty accounting for the Mars lander EDL system - both Curiosity and Perseverance. (Among other uses of UQ for engineered systems at JPL.) In the slides he’s deliberately contrasting a “MUF” (model uncertainty factor) approach that uses safety factors, possibly stacked as indicated in the OP, versus a “BE+U” (best estimate plus uncertainty) approach. The latter uses verified physics-based models, validation of model predictions versus experimental tests, and uncertainty quantification of sources of error, to get a best estimate of whatever quantity is critical to safe operation, and an uncertainty. A final margin can then be added onto that, see slide 13. The mess at the system level that can result from a stack of ad hoc MUFs applied at the subsystem level is shown on slide 15. Incidentally, many of the comments nearby seem to think the application of domain specific safety factors is still state of the art. This just isn’t true any more for high risk systems. https://cstools.asme.org/csconnect/Filedownload.cfm?thisfile=39465.pdf&dir=CommitteeFiles&44348.0865972 https://cstools.asme.org/csconnect/Filedownload.cfm?thisfile...

abduhl 5y ago

Most modern bridges you drive over are designed the probabilistic way that is suggested in the article. Bridge design followed vertical construction as material science and manufacturing got better for steel and concrete. The probabilistic approaches haven’t been adopted much in engineering fields that deal with too many unknowns. I’m actually incredibly surprised that there was no mention of the technical terms for these approaches: Allowable Stress Design, Load and Resistance Factor Design, and Yeah That Looks Right Design. LRFD is highly probability and materials testing based. ASD is a hybrid approach of old factors of safety and some probabilistic theory. YTLRD is based on the long and storied history of guys who have been doing it this way since before you were born, no matter when that was.

wiredfool 5y ago
And they're all used, to some extent. If your Wizzy design based on the latest everything doesn't pass the grumpy old partner's YTLRD review, you're going to redo it till you do. (In my case, that was Bill. He was one of those guys who knew where to put the $50k mark)

justincredible 5y ago

I don't think the probabilistic failure model is suggested, merely presented. The author concludes that departing from the stated safety factor is necessarily taking on additional risk.

whack 5y ago

> The article suggests using a probabilistic failure model instead of a large a safety factor That wasn't my takeaway from the article, and the article discusses the same problems you've brought up. To quote: > On the flip side, it requires more information as well. There must be good data or the result will be as arbitrary as the factor of safety, without the benefit of decades of experience... A bigger issue, and the one I think has prevented more widespread adoption, is that probabilistic design doesn’t account for fluke events -- the unknowables. If you don’t know what could happen, you obviously can’t assign that event a probability... The ideal approach might be a hybrid. Probabilistic design could be responsible for covering simplifications and a reduced safety factor could cover the unknowables. Of course, there’s no simple way to determine how much of the current factor covers simplifications, so reducing the factor would still be a risky endeavor... For my projects, I intend to embrace the empirical nature of safety factors and not think too hard about it. If a factor already exists for the area I’m exploring, I’ll use that