5 ms·
How Complex Systems Fail (1998)
- tpoacher 4y agoHow complex systems fail: Like most things. Slowly at first, then all at once.
- teddyh 4y agos/^/How /
- Zealotux 4y agoSimple systems also fails!
- bob1029 4y agoThe root cause thing I would push back on. Management of complex systems is never a done deal, so there is always the possibility you missed some tiny gap in your process that can still take you out entirely. A good example of this being the Texas grid in 2021.
- kwhitefoot 4y agoYes, but that tiny gap is no more the root cause than any other one of many other decisions that preceded the catastrophe.
- aliasxneo 4y agoI believe the point is to not look for a single root cause.
- redtexture 4y agoNot a tiny gap at all. For which there had been numerous warnings for years by industry observers. That system does not draw upon outside of network utilities, to avoid Federal regulation, hence has limited reserve resources. And the Texas system did not pay providers to have standby reserves. Thus a fragile system, easy to be subject to failure. Edit: Texas Was Warned a Decade Ago Its Grid Was Unready for Cold (Bloomberg) https://www.bloomberg.com/news/articles/2021-02-17/texas-was-warned-a-decade-ago-its-grid-was-unprepared-for-cold https://www.bloomberg.com/news/articles/2021-02-17/texas-was...
- fatneckbeardz 4y agothe thing is that we all see this system as a failure, but the people who make the decisions, and who run that system, see it as a success. their primary goal is not to provide reliable power, their primary goal is profit and ideology. and they have successfully done both those things. a bunch of people died, nothing will change, and they will face no consequences. texas will remain off the grid and the next winter storm the same thing will happen. in their book, they are a success. thats the whole thing about complex systems. at some point, human beings disagree on what the priorities are, so they disagree on what failure is, they disagree on what maintenance is, and they disagree on what "proper function" is. complex systems are always connected to complex vested interests and flows of power and money. so people say things like "the boeing 787 failed". . . did it though? It killed hundres of people , but the executives in charge of it made huge profits and faced zero consequences. Boeing stock price is fine, and it will not face any meaningful punishment or consequences from the government. nor will it face any meaningful consequences from the legal system, which is irrevocably twisted in favor of big corporations like them. From these peoples perspective, the 787 killing hundreds of people is not a failure, its just something that happened that they can hire PR people to deal with. it wont interrupt cash flow (or, at least it wont interrupt their personal bonuses and personal wealth that much) so its basically irrelevant to them.
- redtexture 4y agoNot a gap. A planned system, as you indicate. Prior poster indicated the item was a mere gap. The system operator does not treat the problem as a gap, and also has had no authority to enforce appropriate remedation at the generator-plant level, and the gas-supply level to reduce the cold-caused shutdowns encountered. --- The Texas Electric Grid Failure Was a Warm-up: One year after the deadly blackout, officials have done little to prevent the next one—which could be far worse. By Russell Gold, Texas Monthly (February 2022) https://www.texasmonthly.com/news-politics/texas-electric-grid-failure-warm-up/ https://www.texasmonthly.com/news-politics/texas-electric-gr...
- greyface- 4y ago2020, 84 comments https://news.ycombinator.com/item?id=25550685 https://news.ycombinator.com/item?id=25550685 2019, 33 comments https://news.ycombinator.com/item?id=20380055 https://news.ycombinator.com/item?id=20380055 2017, 21 comments https://news.ycombinator.com/item?id=15002683 https://news.ycombinator.com/item?id=15002683 2014, 16 comments https://news.ycombinator.com/item?id=8282923 https://news.ycombinator.com/item?id=8282923
- dang 4y agoThanks! Macroexpanded: How Complex Systems Fail (1998) - https://news.ycombinator.com/item?id=25550685 https://news.ycombinator.com/item?id=25550685 - Dec 2020 (83 comments) How Complex Systems Fail (1998) [pdf] - https://news.ycombinator.com/item?id=20380055 https://news.ycombinator.com/item?id=20380055 - July 2019 (33 comments) How complex systems fail (2002) [pdf] - https://news.ycombinator.com/item?id=15002683 https://news.ycombinator.com/item?id=15002683 - Aug 2017 (19 comments) How Complex Systems Fail (1998) [pdf] - https://news.ycombinator.com/item?id=14127543 https://news.ycombinator.com/item?id=14127543 - April 2017 (7 comments) How Complex Systems Fail (1998) - https://news.ycombinator.com/item?id=8282923 https://news.ycombinator.com/item?id=8282923 - Sept 2014 (16 comments) How Complex Systems Fail [pdf] - https://news.ycombinator.com/item?id=926735 https://news.ycombinator.com/item?id=926735 - Nov 2009 (1 comment)
- eternityforest 4y agoThe real URL is how.complexsystems.fail Point 2 is "Complex systems are heavily and successfully defended against failure" Complex systems do fail. But airplanes are still extremely safe. Because people stacked on even more complex systems, often involving worldwide change in response to an accident that happened once. You constantly hear about how safe it is to fly. And yet hardly anyone seems to learn fron their successes. When you stop accepting failure and are willing to disrupt everything if it saves even one life, you can do a lot. Complex systems may be unreliable, but with enough work, it seems we can sometimes make the overall picture safer than not having them. I can't firmware update all of mankind to never leave a baby in a hot car. But they can put sensors on seats and continually do studies to be sure it's working. Complex systems are sometimes more controllable than people or simple systems. The choice sometimes seems to be "Add complexity, do nothing, or do something that nobody will accept"
- hutzlibu 4y ago"I can't firmware update all of mankind to never leave a baby in a hot car. But they can put sensors on seats and continually do studies to be sure it's working." We can also put mandatory sensors in peoples bodies, to make sure they act and live allright. But I think this would be overcomplicating things.
- eternityforest 4y agoComplexity wouldn't be the problem, the issue would be violation of people's bodies. Something like a car seat sensor is just a consumer product safety regulation that does not imply any extreme expense, danger, or violation, except to very extreme anti-tech or anti-regulation people. It's a further development of the same trend as headlight or seatbelt related laws. Plus, it protects people who have no way of protecting themselves, from mistakes that are made by people who have actively been prevented all their life (via confidence culture) from having the tools to prevent making them.
- hutzlibu 4y ago"Plus, it protects people who have no way of protecting themselves, from mistakes that are made by people who have actively been prevented all their life" Let me put it that way, idiots who let their babies in a hot car actually would also need a million of other mandated sensors and I would not trust them with small lives, or a dangerous fast and heavy bullet like a car in the first place. But the default should be trust and not micromanaging people assuming they are all idiots. Because this keeps people as idiots.
- _plg_ 4y agoThe Career, Accomplishments, and Impact of Richard I. Cook: A Life in Many Acts (September 12, 2022) https://www.adaptivecapacitylabs.com/blog/2022/09/12/richard-cook-a-life-in-many-acts/ https://www.adaptivecapacitylabs.com/blog/2022/09/12/richard...
- dqpb 4y agoThis does not seem very rigorous. Can someone point to a better coverage of this topic?
- thraxil 4y agoIt's not intended to be rigorous. The context here is that Richard I. Cook, one of the main figures in safety and resilience engineering, who's published many, many papers on these topics died recently. The "How Complex Systems Fail" paper is intended to be a bit pithy and light; more an attempt at summarizing years of wisdom. See: https://www.adaptivecapacitylabs.com/blog/2022/09/12/richard-cook-a-life-in-many-acts/ https://www.adaptivecapacitylabs.com/blog/2022/09/12/richard...
- dqpb 4y agoWell, this sounds wrong to me: > Catastrophe requires multiple failures – single point failures are not enough My experience is that a single failure causes a cascade of subsequent failures. This topic is very interesting, but this post is more of a teaser of topics than a real explanation.
- advisedwang 4y agoPlaces where a single failure in an otherwise perfectly functioning system can cause catastrophic outcomes are relatively easy to identify, relatively easy to argue need to be fixed and relatively easy to fix. As a result mature, complex systems have generally developed safety mechanisms for such issues. Once you have done that you need at least two failures (underlying issue + safety, hot+cold, or two interacting systems). I would suspect that your experience of single modes of failure being present are one of the following * Immature system (e.g. a startup) * One where failure is acceptable and so engineering isn't invested in solving these issues (i.e. the author is talking about disasters that kill people, not causing a few mins of ads not getthing shown) * Extreme organizational dysfunction (talking criminal negligence type stuff)
- dqpb 4y ago
- x32n23nr 4y agoI highly recommend the book Normal Accidents by Charles Perrow. Perrow argues that multiple and unexpected failures are built into society's complex and tightly coupled systems
- eternalban 4y agoEngineering a safer world - Systems thinking applied to thinking, by Nancy Leveson (MIT - 2009) was recommended in a previous discussion as a more comprehensive and systemic treatment: https://news.ycombinator.com/item?id=14131981 https://news.ycombinator.com/item?id=14131981 http://sunnyday.mit.edu/safer-world.pdf http://sunnyday.mit.edu/safer-world.pdf STAMP ("System-Theoretic Accident Model and Processes") is reviewed here: https://www.sciencedirect.com/science/article/abs/pii/S0950423014001193 https://www.sciencedirect.com/science/article/abs/pii/S09504... And there is a course (lecture notes look great): https://ocw.mit.edu/courses/16-63j-system-safety-spring-2016/pages/syllabus/ https://ocw.mit.edu/courses/16-63j-system-safety-spring-2016...
- 0xbadcafebee 4y agoNumbers 1, 4-8, and 11-18, are all Truisms. The rest are not: "2. Complex systems are heavily and successfully defended against failure" Many complex systems are weakly defended, sometimes not at all. Sometimes the defense is accidental or incidental. Sometimes they are heavily yet unsuccessfully defended. Never attribute to defense that which can be attributed to purely random chance, ignorance, convenience, and avoidance of responsibility. "3. Catastrophe requires multiple failures – single point failures are not enough." Catastrophe definitely can and does happen from single points of failure. It's just that in highly defended systems, multiple failures are common. "9. Human operators have dual roles: as producers & as defenders against failure." These can be distinct roles, but in practice that requires extra money, staffing, etc which makes it rare. However, there are systems in which defense becomes its own role, often because the producers suck at it or don't want to do it, or are just really busy. "10. All practitioner actions are gambles." On the fence about this one. I would say all practitioner changes are gambles. A practitioner looking at a pressure gauge dial is an action, but it isn't a gamble. Unless the gauge needle sticks, and reading it was a critical action... I suppose you could say all actions are gambles, and changes are much more risky gambles, and non-change actions are likely to be seen as non-risky.
- tempie_deleteme 4y agowhy read about it when we can experience it by being alive today? to better understand the times, of course...