5 ms·
I don't think this list hits any fundamental truths. The great depression doesn't have parallels to software failures beyond the fact that complex systems fail.
by roenxi 3y ago
I don't think this list hits any fundamental truths. The great depression doesn't have parallels to software failures beyond the fact that complex systems fail. And many of the lessons are vague and unactionable - "Put an end to information hoarding within orgs/teams" for example, says nothing. The Atlassian copy that section links to also says nothing. A lot of the lessons lack meaty learnings, and good luck to anyone trying to put everything in to practice simultaneously.
Makes a fun list of big disasters though and I respect this guy's eye for website design. The site design was probably stronger than the link content, there is a lot to like about it.
- jacquesm 3y agoComplex systems fail, but they don't all fail in the same way and analyzing how they fail can help in engineering new and hopefully more robust complex systems. I'm a huge fan of Risk Digest and there isn't a disaster small enough that we can't learn from it. Obviously the larger the disaster the more complex the failure and the harder to analyze the root cause. But one interesting takeaway for me from this list is that all of them were preventable and in all but few of the cases the root cause may have been the trigger but the setup of the environment is what allowed the fault to escalate in the way that it did. In a resilient system faults happen as well, but they do not propagate. And that's the big secret to designing reliable systems.
- roenxi 3y ago> ...one interesting takeaway for me from this list is that all of them were preventable... Every disaster is preventable. Everything on the list was happening in human-engineered environments - as do most things that affect humans. The human race has been the master of its own destiny since the 1900s. The questions are how far before the disaster we need to look to find somewhere to act and what needed to be given up to change the flow of events. But that doesn't have any implications for software engineering. Studying a software failure post mortem will be a lot more useful than studying 9/11.
- jacquesm 3y ago> Every disaster is preventable. No, there is such a thing as residual risk and there are always disasters that you can't prevent such as natural disasters. But even then you can have risk mitigation and strategies for dealing with the aftermath of an incident to limit the effects. > Everything on the list was happening in human-engineered environments - as do most things that affect humans. That is precisely why they were picked and make for good examples. > The human race has been the master of its own destiny since the 1900s. That isn't true and it likely will never be true. We are tied 1:1 to the fate of our star and may well go down with it. There is a small but non-zero chance that we can change our destiny but I wouldn't bet on it. And even then in the even longer term it still won't matter. We are passengers, the best we can do is be good stewards of the ship we've inherited. > The questions are how far before the disaster we need to look to find somewhere to act and what needed to be given up to change the flow of events. Indeed. So in the case of each of the items listed the RCA gives a point in time where the accident given the situation as it existed was no longer a theoretical possibility but an event in progress. Situation and responses determined how far it got and in each of the cases outlined you can come up with a whole slew of ways in which the risk could have been reduced and possibly how the whole thing may have been averted once the root cause had triggered. But that doesn't mean that the root cause doesn't matter, it matters a lot. But the root cause isn't always a major thing. An O-ring, a horseshoe... > But that doesn't have any implications for software engineering. If that is your takeaway then for you it indeed probably does not. But I see such things in software engineering every other week or so and I think there are many lessons from these events that apply to software engineering. As do the people that design reliable systems, which is why many of us are arguing for liability for software. Because once producers of software are held liable for their product a large number of the bad practices and avoidable incidents (not just security) would become subject to the Darwinian selection process: bad producers would go out of business. > Studying a software failure post mortem will be a lot more useful than studying 9/11. You can learn lots of things from other fields, if you are open to learning in general. Myopically focusing on your own field is useful and can get you places but it will always result in 'deep' approaches, never in 'wide' approaches and for a really important system both of these approaches are valid and complementary. To make your life easier the author has listed in the right hand column which items from the non-software disasters carry over into the software world, which I think is a valuable service. A middlebrow dismissal of that effort is throwing away an opportunity to learn, for free, from incidents that have all made the history books. And if you don't learn from your own and others' mistakes then you are bound to repeat that history. Software isn't special in this sense. Not at all. What is special is the arrogance of some software people who believe that their field is so special that they can ignore the lessons from the world around them. And as a corollary: that they can ignore all the lessons already learned in software systems in the past. We are in an eternal cycle of repeating past mistakes with newer and shinier tools and we urgently need to break out of it.