3 ms·

The thing that worries me most is that, based on mere amateur observations in the media, the number of incidents involving user interfaces, repeated changes to

by yason 8y ago

The thing that worries me most is that, based on mere amateur observations in the media, the number of incidents involving user interfaces, repeated changes to procedures, and software fixes to "unexpected" corner-cases seems to have been in the increase. To the uninitiated, it seems that problems in airline incidents used to revolve around mechanical failures, bad wiring, sensor malfunctions, toasted electronics causing smell of smoke and such. Plain and simple. Issue a fix in the design and apply to all planes of the same type, and you'll end up reducing the failure surface bit by bit. I might be totally wrong about this because I don't track actual data. But if I've spotted the trend right it's scary because the problems are shifting into our space, i.e. software engineering. And being a software engineer, I know we're pretty much fucked at that point. We've only ever managed to write reliable and trustworthy software when we've split it into very tiny pieces that we can verify and kept the number of pieces small. Maybe aviation computers used to be much simpler that they could be verified more throughoutly. Maybe airplanes used to have less features and they could keep the complexity sufficiently down and functionality orthogonal. Maybe there was enough human glue in between the systems so that there was a live sanity-check during flights and pilots could react properly if the computers didn't agree on something. But now I sense a new category of error conditions that are eerily similar to what we've had in non-critical software for decades where assumptions are laid on top of other assumptions, and when they fail the whole stack comes crashing down. Only this time there might be a whole plane coming down instead of getting a curious SIGSEGV on the screen with a blinking cursor. It might start as an innocent "couldn't access flight plan because of wifi went down" but such interdependencies between certified and uncertified systems grow exponentially and this will snowball into the unmanageable very soon.

tomaskafka 8y ago
If you split a complex systems into a tiny pieces, you imo just shift a locus of problems from individual modules into their more and more complex network of interactions and emergent behaviors.
- yason 8y ago
  Hence you keep the number of pieces small to avoid the exponential interactions. You only ever implement what is really needed instead of letting feature creep and excess convenience work their way into your plane. It's not like N times more of that sweet complexity is required to fly a plane today than what was needed in the 80's or 90's.
poof131 8y ago
I think you are on the right track about one of the underlying problems. From another comment on a different thread. > My company's error rates have increased on the "common fleet" by almost triple. [1] The aircraft manufacturers and carriers want to get the most change with the least impact: updated jets with better gas mileage that don’t require pilots to retrain. It seems the FAA and other regulators have become too lenient in letting the manufacturers put significant changes into aircraft without requiring them to have different type ratings or at least add new training requirements.[2] Not communicating the MCAS changes seems to indicate a broken process at Boeing too.[3] But you are also observing the Swiss cheese model of safety.[4] As the easier individual issues are fixed (mechanical) more difficult compound issues cause the accident, where multiple things (pilot training, bad communication, new system) all need to line up. [1] https://news.ycombinator.com/item?id=19376565 https://news.ycombinator.com/item?id=19376565 [2] https://en.wikipedia.org/wiki/Type_rating https://en.wikipedia.org/wiki/Type_rating [3] https://news.ycombinator.com/item?id=19377220 https://news.ycombinator.com/item?id=19377220 [4] https://en.wikipedia.org/wiki/Swiss_cheese_model https://en.wikipedia.org/wiki/Swiss_cheese_model

by yason 8y ago

tomaskafka 8y ago

If you split a complex systems into a tiny pieces, you imo just shift a locus of problems from individual modules into their more and more complex network of interactions and emergent behaviors.

yason 8y ago
Hence you keep the number of pieces small to avoid the exponential interactions. You only ever implement what is really needed instead of letting feature creep and excess convenience work their way into your plane. It's not like N times more of that sweet complexity is required to fly a plane today than what was needed in the 80's or 90's.

poof131 8y ago

I think you are on the right track about one of the underlying problems. From another comment on a different thread. > My company's error rates have increased on the "common fleet" by almost triple. [1] The aircraft manufacturers and carriers want to get the most change with the least impact: updated jets with better gas mileage that don’t require pilots to retrain. It seems the FAA and other regulators have become too lenient in letting the manufacturers put significant changes into aircraft without requiring them to have different type ratings or at least add new training requirements.[2] Not communicating the MCAS changes seems to indicate a broken process at Boeing too.[3] But you are also observing the Swiss cheese model of safety.[4] As the easier individual issues are fixed (mechanical) more difficult compound issues cause the accident, where multiple things (pilot training, bad communication, new system) all need to line up. [1] https://news.ycombinator.com/item?id=19376565 https://news.ycombinator.com/item?id=19376565 [2] https://en.wikipedia.org/wiki/Type_rating https://en.wikipedia.org/wiki/Type_rating [3] https://news.ycombinator.com/item?id=19377220 https://news.ycombinator.com/item?id=19377220 [4] https://en.wikipedia.org/wiki/Swiss_cheese_model https://en.wikipedia.org/wiki/Swiss_cheese_model