4 ms·

No. Just compare the difficulty of getting a regular driver's license to that of a license to fly commercial airliners. It's safe to say that _almost_ every pil

No. Just compare the difficulty of getting a regular driver's license to that of a license to fly commercial airliners. It's safe to say that _almost_ every pilot can control a plane without incident when sensors / automated systems fail, so long as they are given full manual control. Automation overriding pilot inputs during sensor failure is a recipe for disaster.

jakobegger 8y ago
> Automation overriding pilot inputs during sensor failure is a recipe for disaster. In 2009 an Airbus A330-200 crashed into the Atlantic ocean killing more than 200 people because a confused pilot kept pulling the plane up, despite loud stall warnings from automatic safety systems [1]. In the discussions of that event, one commenter asked, "Why did [the plane] not override the pilots' inputs and force a pitch-down?" [2] Designing safety critical systems is difficult. If the automated system fails, the pilot needs to be able to take over manual control. But at the same time, if the pilot does something stupid, the automatic system should prevent that. I'm not sure how to square that. [1]: https://www.telegraph.co.uk/technology/9231855/Air-France-Flight-447-Damn-it-were-going-to-crash.html https://www.telegraph.co.uk/technology/9231855/Air-France-Fl... [2]: https://news.ycombinator.com/item?id=8452080 https://news.ycombinator.com/item?id=8452080
- AstralStorm 8y ago
  This is equivalent to a confused deputy variant of a Byzantine failure. Since you already have allegedly redundant pilots and alarm systems and hopefully sensors, the remaining part is figuring out which failure detector has ultimately failed to work and why. Also if redundant pilots are failing, it's a systematic failure in training, fatal to airlines and whoever designs training simulators. The most potentially fatal to Boeing scenario is a systematic failure or negligence in terms of redundancy. Hampering pilot training is bad enough. Another potential failure point is a software issue which means negligence in safety critical software engineering.
- mnw21cam 8y ago
  One of the confounding elements of that particular crash was the fact that when the angle of attack increased above a certain (crazy) amount, the stall warnings went away, due to the inability of the system to cope with an angle of attack that high. This confused the pilots into thinking they were going "crazy fast", when actually they were going "crazy slow".
  - richardkmichael 8y ago
    Yes, I point out AF447 as a perfect and horrific example of a problem which is really HCI (human computer interaction) / UX. The lack of stall warning meant two different things: "not stalling", and "stalling so extremely it can't be sure, so don't report it"; as you say. I wondered if there should be a different sounding alarm for "can't determine". Or, the pitch of stall alarm could rise, such that pilots could tell "which side of the stall" the plane is on, so that entering stall from one side would side start with a very high pitch or very low pitch. (I'd guess there's a better way..) So, in this case, it's more subtlety more tragic than you describe, because the pilot did correct the AoA, and nose down, but when he did that, the stall warning resumed (albeit from the extreme other "side" of the interval), confusing him: no alarm, alarm resumes, so he thought he "re-stalled" it and he pulled up again. This kept him mentally "stuck" in the extreme stall. They needed to "pass right through" the stall warning window, and come out the other side. They had sufficient altitude for this to succeed.
- StreamBright 8y ago
  I disagree. Flying a plane is pretty much a simple state machine with very well defined states. We have hundred thousands of flight hours available and we can also build a very realistic simulator which is used extensively to train pilots. It is should not be a challenging task to build a reliable auto-pilot and in fact we have already one that covers most of the aspects of flying. On the top of this, most of the flight related incidents are human errors.
  - noselasd 8y ago
    It doesn't sound that plausible that we can account for all the various things that can happen to a plane in something as complex as the earths atmosphere with a "simple state machine" . Weather can do really wacky things.
    - StreamBright 8y ago
      You are saying that it is impossible to implement auto-pilot? How does human pilot fly something planes? You think that human mind can deal with infinite states?
      - nradov 8y ago
        Yes the human mind can deal with an effectively infinite number of states (and let's not get into a pointless pedantic discussion about the true meaning of "infinite"). Until we figure out how to build a true AGI then no autopilot will be able to do that.
        StreamBright 8y ago
        No it can't. Flying an airplane on Earth is a very limited exercise in every possible dimension and it can be easily automated without AGIs.
        nradov 8y ago
        You're simply wrong and have no idea how flight control actually works. In particular failure modes aren't simple at all; real machines can break in a huge number of ways and no one could possibly predict them all. Recommend you take some actual graduate level aeronautical engineering courses and then reconsider the issue.
chopin 8y ago
The system needs to realize that there is a sensor failure. If the Lion Air crash is due to a faulty sensor Boeing needs to answer some serious questions: 1. Why did they not inform operators about changed system behavior? 2. Why wasn't the system capable of detecting sensor failure and/or act accordingly? In the automation industry one goes to great length to be able to tell good sensors apart from bad ones. Why was that not the case here? If it is not possible to tell bad sensors from good ones, why wasn't there (sufficient) redundancy (=multiple sensors) which at least deactivate the system on conflicting input?
- FabHK 8y ago
  Re 2: There are two AoA sensors. However, good question - why aren't there 3 (so you can have majority quorum in case of one failure), and why wasn't automation disabled once a discrepancy between the 2 existing ones was detected?
  - Serow225 8y ago
    Cost.
  - Zak 8y ago
    I've seen a similar failure in a van. The van was equipped with antilock brakes, which override the driver's control input and reduce braking force when the speed sensors on one or more wheels detect that the wheel has stopped too suddenly, as if skidding. One of the sensors had malfunctioned, resulting in an "ABS" warning light, and a fault code readable with a tool indicating which sensor was malfunctioning. Did it disable the ABS? No. Instead, it continued to rely on the faulty sensor information, activating ABS under moderate deceleration on dry pavement, greatly reducing the available braking force and creating a severe danger. I think there may be a bias among people designing safety equipment to always try to provide the intended safety benefit, even when part of the system isn't working. The problem is it's easy to lose sight of how the safety feature fits into the overall picture. A pilot can almost always safely fly a plane that doesn't use automation to recover from a stall, but not necessarily one that uses the trim to point the nose at the ground in spite of control inputs to the contrary.

jakobegger 8y ago

> Automation overriding pilot inputs during sensor failure is a recipe for disaster. In 2009 an Airbus A330-200 crashed into the Atlantic ocean killing more than 200 people because a confused pilot kept pulling the plane up, despite loud stall warnings from automatic safety systems [1]. In the discussions of that event, one commenter asked, "Why did [the plane] not override the pilots' inputs and force a pitch-down?" [2] Designing safety critical systems is difficult. If the automated system fails, the pilot needs to be able to take over manual control. But at the same time, if the pilot does something stupid, the automatic system should prevent that. I'm not sure how to square that. [1]: https://www.telegraph.co.uk/technology/9231855/Air-France-Flight-447-Damn-it-were-going-to-crash.html https://www.telegraph.co.uk/technology/9231855/Air-France-Fl... [2]: https://news.ycombinator.com/item?id=8452080 https://news.ycombinator.com/item?id=8452080

AstralStorm 8y ago
This is equivalent to a confused deputy variant of a Byzantine failure. Since you already have allegedly redundant pilots and alarm systems and hopefully sensors, the remaining part is figuring out which failure detector has ultimately failed to work and why. Also if redundant pilots are failing, it's a systematic failure in training, fatal to airlines and whoever designs training simulators. The most potentially fatal to Boeing scenario is a systematic failure or negligence in terms of redundancy. Hampering pilot training is bad enough. Another potential failure point is a software issue which means negligence in safety critical software engineering.
mnw21cam 8y ago
One of the confounding elements of that particular crash was the fact that when the angle of attack increased above a certain (crazy) amount, the stall warnings went away, due to the inability of the system to cope with an angle of attack that high. This confused the pilots into thinking they were going "crazy fast", when actually they were going "crazy slow".
- richardkmichael 8y ago
  Yes, I point out AF447 as a perfect and horrific example of a problem which is really HCI (human computer interaction) / UX. The lack of stall warning meant two different things: "not stalling", and "stalling so extremely it can't be sure, so don't report it"; as you say. I wondered if there should be a different sounding alarm for "can't determine". Or, the pitch of stall alarm could rise, such that pilots could tell "which side of the stall" the plane is on, so that entering stall from one side would side start with a very high pitch or very low pitch. (I'd guess there's a better way..) So, in this case, it's more subtlety more tragic than you describe, because the pilot did correct the AoA, and nose down, but when he did that, the stall warning resumed (albeit from the extreme other "side" of the interval), confusing him: no alarm, alarm resumes, so he thought he "re-stalled" it and he pulled up again. This kept him mentally "stuck" in the extreme stall. They needed to "pass right through" the stall warning window, and come out the other side. They had sufficient altitude for this to succeed.
StreamBright 8y ago
I disagree. Flying a plane is pretty much a simple state machine with very well defined states. We have hundred thousands of flight hours available and we can also build a very realistic simulator which is used extensively to train pilots. It is should not be a challenging task to build a reliable auto-pilot and in fact we have already one that covers most of the aspects of flying. On the top of this, most of the flight related incidents are human errors.
- noselasd 8y ago
  It doesn't sound that plausible that we can account for all the various things that can happen to a plane in something as complex as the earths atmosphere with a "simple state machine" . Weather can do really wacky things.
  - StreamBright 8y ago
    You are saying that it is impossible to implement auto-pilot? How does human pilot fly something planes? You think that human mind can deal with infinite states?
    - nradov 8y ago
      Yes the human mind can deal with an effectively infinite number of states (and let's not get into a pointless pedantic discussion about the true meaning of "infinite"). Until we figure out how to build a true AGI then no autopilot will be able to do that.
      - StreamBright 8y ago
        No it can't. Flying an airplane on Earth is a very limited exercise in every possible dimension and it can be easily automated without AGIs.
        nradov 8y ago
        You're simply wrong and have no idea how flight control actually works. In particular failure modes aren't simple at all; real machines can break in a huge number of ways and no one could possibly predict them all. Recommend you take some actual graduate level aeronautical engineering courses and then reconsider the issue.

chopin 8y ago

The system needs to realize that there is a sensor failure. If the Lion Air crash is due to a faulty sensor Boeing needs to answer some serious questions: 1. Why did they not inform operators about changed system behavior? 2. Why wasn't the system capable of detecting sensor failure and/or act accordingly? In the automation industry one goes to great length to be able to tell good sensors apart from bad ones. Why was that not the case here? If it is not possible to tell bad sensors from good ones, why wasn't there (sufficient) redundancy (=multiple sensors) which at least deactivate the system on conflicting input?

FabHK 8y ago
Re 2: There are two AoA sensors. However, good question - why aren't there 3 (so you can have majority quorum in case of one failure), and why wasn't automation disabled once a discrepancy between the 2 existing ones was detected?
- Serow225 8y ago
  Cost.
- Zak 8y ago
  I've seen a similar failure in a van. The van was equipped with antilock brakes, which override the driver's control input and reduce braking force when the speed sensors on one or more wheels detect that the wheel has stopped too suddenly, as if skidding. One of the sensors had malfunctioned, resulting in an "ABS" warning light, and a fault code readable with a tool indicating which sensor was malfunctioning. Did it disable the ABS? No. Instead, it continued to rely on the faulty sensor information, activating ABS under moderate deceleration on dry pavement, greatly reducing the available braking force and creating a severe danger. I think there may be a bias among people designing safety equipment to always try to provide the intended safety benefit, even when part of the system isn't working. The problem is it's easy to lose sight of how the safety feature fits into the overall picture. A pilot can almost always safely fly a plane that doesn't use automation to recover from a stall, but not necessarily one that uses the trim to point the nose at the ground in spite of control inputs to the contrary.