4 ms·

The real URL is how.complexsystems.fail Point 2 is "Complex systems are heavily and successfully defended against failure" Complex systems do fail. But airpla

The real URL is how.complexsystems.fail Point 2 is "Complex systems are heavily and successfully defended against failure" Complex systems do fail. But airplanes are still extremely safe. Because people stacked on even more complex systems, often involving worldwide change in response to an accident that happened once. You constantly hear about how safe it is to fly. And yet hardly anyone seems to learn fron their successes. When you stop accepting failure and are willing to disrupt everything if it saves even one life, you can do a lot. Complex systems may be unreliable, but with enough work, it seems we can sometimes make the overall picture safer than not having them. I can't firmware update all of mankind to never leave a baby in a hot car. But they can put sensors on seats and continually do studies to be sure it's working. Complex systems are sometimes more controllable than people or simple systems. The choice sometimes seems to be "Add complexity, do nothing, or do something that nobody will accept"

hutzlibu 4y ago
"I can't firmware update all of mankind to never leave a baby in a hot car. But they can put sensors on seats and continually do studies to be sure it's working." We can also put mandatory sensors in peoples bodies, to make sure they act and live allright. But I think this would be overcomplicating things.
- eternityforest 4y ago
  Complexity wouldn't be the problem, the issue would be violation of people's bodies. Something like a car seat sensor is just a consumer product safety regulation that does not imply any extreme expense, danger, or violation, except to very extreme anti-tech or anti-regulation people. It's a further development of the same trend as headlight or seatbelt related laws. Plus, it protects people who have no way of protecting themselves, from mistakes that are made by people who have actively been prevented all their life (via confidence culture) from having the tools to prevent making them.
  - hutzlibu 4y ago
    "Plus, it protects people who have no way of protecting themselves, from mistakes that are made by people who have actively been prevented all their life" Let me put it that way, idiots who let their babies in a hot car actually would also need a million of other mandated sensors and I would not trust them with small lives, or a dangerous fast and heavy bullet like a car in the first place. But the default should be trust and not micromanaging people assuming they are all idiots. Because this keeps people as idiots.
    - eternityforest 4y ago
      This isn't a matter of "Keeping people as idiots". It's a matter of natural capability. For some people there's a constant risk that at any moment your brain will glitch out and drop any file at any time. Even your most ingrained habits aren't reliable. No matter how important something is it can disappear from your mind like it was never there. It's not a matter of skills you learn, it's a matter of someone who has never in their life been able to master a "When X I will do Y" process, and likely has never even been made aware that there's any issue beyond not trying hard enough or not caring. I hear "You obviously don't care or you'd remember" pretty much all the time. They care about their kids. They have probably been taught their whole life that leaving them in a hot car is something that only happens if you don't care. When it happens they are as shocked as the rest of the world. We don't have the option to choose who has kids and who doesn't. They do, and they're going to, and if you try to take them away to some horrid state orphanage they will probably have even worse lives. Tech is a factor we can control. Maybe it will make things better, maybe worse, but that's what the studies are for.
dijit 4y ago
I really see your point here; but I have to caution: Airplanes are "exactly as simple" as they need to be. There is a lot that goes into their design to simplify things greatly; you're probably thinking of complicated computer systems that are used in planes. But those computer systems are incredibly simple compared to what we use or build atop of: as simple as they have to be in order to be fully understood.
- eternityforest 4y ago
  They're still far more complicated than a layman might guess after years of hearing that simple is always better. I'm guessing things like fly by wire would be automatically assumed to be unsafe by a lot of people. When cars get features that are anything like what goes into planes, people tend to get upset and say "I'm not an idiot, you should make cars expensive and complicated just because some drivers can't stop crashing without a computer".
Ferret7446 4y ago
> are willing to disrupt everything if it saves even one life I feel like you're walking away with the wrong lesson. Disrupting everything is a great way to blow up complex systems. You want to change things gradually, ensuring that the human side can keep up.
- eternityforest 4y ago
  A lot of the time what happens is the human side doesn't need to keep up. They'll say "This actuator fails if driven past it's limit in hot weather after a rainstorm, and we have data showing that people can overdrive it accidentally in this condition". Then they'll replace all affected actuators even if it costs millions . Or they'll add a software patch to keep you from overdriving it. What they don't do is say "It's probably fine, people just need to be more careful". If someone made a mistake once, someone else can make it again. Systems have to be built for the people who will actually use them, not theoretical elite users. On occasion the technical fix has it's own dangers that need to be evaluated and you can't find any substitute for operators doing the right thing(See Gare de Lyon for the perfect example of multiple human errors by different people interacting with complex safety systems). But only some careful analysis will tell you what's more dangerous.

hutzlibu 4y ago

"I can't firmware update all of mankind to never leave a baby in a hot car. But they can put sensors on seats and continually do studies to be sure it's working." We can also put mandatory sensors in peoples bodies, to make sure they act and live allright. But I think this would be overcomplicating things.

eternityforest 4y ago
Complexity wouldn't be the problem, the issue would be violation of people's bodies. Something like a car seat sensor is just a consumer product safety regulation that does not imply any extreme expense, danger, or violation, except to very extreme anti-tech or anti-regulation people. It's a further development of the same trend as headlight or seatbelt related laws. Plus, it protects people who have no way of protecting themselves, from mistakes that are made by people who have actively been prevented all their life (via confidence culture) from having the tools to prevent making them.
- hutzlibu 4y ago
  "Plus, it protects people who have no way of protecting themselves, from mistakes that are made by people who have actively been prevented all their life" Let me put it that way, idiots who let their babies in a hot car actually would also need a million of other mandated sensors and I would not trust them with small lives, or a dangerous fast and heavy bullet like a car in the first place. But the default should be trust and not micromanaging people assuming they are all idiots. Because this keeps people as idiots.
  - eternityforest 4y ago
    This isn't a matter of "Keeping people as idiots". It's a matter of natural capability. For some people there's a constant risk that at any moment your brain will glitch out and drop any file at any time. Even your most ingrained habits aren't reliable. No matter how important something is it can disappear from your mind like it was never there. It's not a matter of skills you learn, it's a matter of someone who has never in their life been able to master a "When X I will do Y" process, and likely has never even been made aware that there's any issue beyond not trying hard enough or not caring. I hear "You obviously don't care or you'd remember" pretty much all the time. They care about their kids. They have probably been taught their whole life that leaving them in a hot car is something that only happens if you don't care. When it happens they are as shocked as the rest of the world. We don't have the option to choose who has kids and who doesn't. They do, and they're going to, and if you try to take them away to some horrid state orphanage they will probably have even worse lives. Tech is a factor we can control. Maybe it will make things better, maybe worse, but that's what the studies are for.

dijit 4y ago

I really see your point here; but I have to caution: Airplanes are "exactly as simple" as they need to be. There is a lot that goes into their design to simplify things greatly; you're probably thinking of complicated computer systems that are used in planes. But those computer systems are incredibly simple compared to what we use or build atop of: as simple as they have to be in order to be fully understood.

eternityforest 4y ago
They're still far more complicated than a layman might guess after years of hearing that simple is always better. I'm guessing things like fly by wire would be automatically assumed to be unsafe by a lot of people. When cars get features that are anything like what goes into planes, people tend to get upset and say "I'm not an idiot, you should make cars expensive and complicated just because some drivers can't stop crashing without a computer".

Ferret7446 4y ago

> are willing to disrupt everything if it saves even one life I feel like you're walking away with the wrong lesson. Disrupting everything is a great way to blow up complex systems. You want to change things gradually, ensuring that the human side can keep up.

eternityforest 4y ago
A lot of the time what happens is the human side doesn't need to keep up. They'll say "This actuator fails if driven past it's limit in hot weather after a rainstorm, and we have data showing that people can overdrive it accidentally in this condition". Then they'll replace all affected actuators even if it costs millions . Or they'll add a software patch to keep you from overdriving it. What they don't do is say "It's probably fine, people just need to be more careful". If someone made a mistake once, someone else can make it again. Systems have to be built for the people who will actually use them, not theoretical elite users. On occasion the technical fix has it's own dangers that need to be evaluated and you can't find any substitute for operators doing the right thing(See Gare de Lyon for the perfect example of multiple human errors by different people interacting with complex safety systems). But only some careful analysis will tell you what's more dangerous.