8 ms·

Flight disruption warning as Airbus requests modifications to 6k planes

235 points by nrhrjrjrjtntbt 11mo ago

ChrisArchitect 11mo ago
More discussion: https://news.ycombinator.com/item?id=46082296 https://news.ycombinator.com/item?id=46082296
loloquwowndueo 11mo ago
At least they didn’t wait for a crash before doing this :/
- Iridescent_ 11mo ago
  The proper reaction when you have a potential issue in your engineering
  - loloquwowndueo 11mo ago
    Sorry forgot to add “unlike that other aerospace company”
    - nickff 11mo ago
      Airbus is not immune to design & manufacturing issues with fatal consequences, they’re just not too-of-mind these days. A similar issue seems to have ‘cropped up’ on this flight: https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72 There was a television show (episode) about another design issue (which was fatal) some time ago: https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447
      - hexbin010 11mo ago
        > https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447 Quoting your link, "Final Report" section: > Temporary inconsistency between the measured speeds, likely as a result of the obstruction of the pitot tubes by ice crystals, caused autopilot disconnection and [flight control mode] reconfiguration to "alternate law (ALT)". - The crew made inappropriate control inputs that destabilized the flight path. - The crew failed to follow appropriate procedure for loss of displayed airspeed information. - The crew were late in identifying and correcting the deviation from the flight path. - The crew lacked understanding of the approach to stall. - The crew failed to recognize the aircraft had stalled, and consequently did not make inputs that would have made recovering from the stall possible. Note the numerous "the crew"
        Arainach 11mo ago
        It's often easy to blame the humans in the loop, but if the UX is poor or the procedures too complicated, then it's a systems fault even if the humans technically didn't "follow procedure".
        ern 11mo ago
        The relief second officer basically pulled up when the stall protection had been disabled and by the time the other pilot and captain realized what was happening it was too late to save the plane. There is a design flaw though: the sidesticks in modern Airbus planes are independent, so the other pilot didn’t get any tactile feedback when the second officer was pulling back.
        deleted 11mo ago
        [deleted]
        berti 11mo ago
        You do get an audible "DUAL INPUT DUAL INPUT" warning and some lights though [1]. It is never allowable to make sidestick inputs unless you are the single designated "pilot flying", but people can sometimes break down under stress of course. [1] https://safetyfirst.airbus.com/app/themes/mh_newsdesk/documents/archives/dual-side-stick-inputs.pdf https://safetyfirst.airbus.com/app/themes/mh_newsdesk/docume...
        nickff 11mo ago
        Crews saved multiple 737-MAXs, but the public has focused on the aircraft whose crews were less effective.
        dghlsakjg 11mo ago
        Faulting the crew is a common thing in almost all air incidents. In this case the crew absolutely could have saved the plane, but the plane did not help them at all. Part of the sales pitch of the Airbus is that the computer does A LOT of handholding for the pilots. In many configurations, including the one that the plane was flying in at the start of the incident, the inputs that caused the crash would have been harmless. In that incident the airspeed feed was lost to the computer and it literally changed the flight controls and turned off the safety limits, and none of the three people in the cockpit noticed. When an Airbus changes flight control modes, it does not keep inputs idempotent. Something harmless under one set of "laws" could crash the plane under another set of laws. In this case, what the pilot with the working control stick was doing would not have caused a crash, except that the computer had taken off the training wheels without anyone noticing. As a result of changing the primary controls one pilot was able to unintentionally place the plane in an unrecoverable state without the other pilots even noticing that he was making control inputs. Tack on that the computer intentionally disregarded the stall warning emanating from the AOA sensor as erroneous at a certain point and did not alert the pilots that the plane was stalled. You are taught from day one of flight training that if you hear the stall alarm you push the power in, and push the nose down until the alarm stops. In this case the stall warning came on, and then as the stall got worse, it turned itself off, with the computer under the mistaken belief that the plane could not actually be that far stalled. So the one alarm that they are trained to respond to in a certain way to recover the plane from a stall was silenced. If I was flying and I heard the stall alarm, then heard it stop, I would assume that I was no longer stalled, not that the plane was so far stalled that the stall alarm was convinced it had broken itself. So yes, the pilots flew the aircraft into the ground, but the computer suffered a partial failure and then changed how the primary flight controls operated. Imagine if the brake pedal, steering wheel, and accelerator all started responding to inputs differently when your car had a sensor issue. That causes the cruise control to fail. Add in that the cruise control failure turns off ABS, auto-brakes, lane assist, and stability control for some reason. Oh yeah, there's a steering control on the other side of the car on the armrest and the person sitting there can now make steering inputs, but it won't give feedback in your steering wheel, and also your steering wheel still can be manipulated when the other guy is steering, but it is completely disconnected from the tires while the other guy is steering. All of the controls are also more sensitive now, and allow you to do things that wouldn't have been possible a few seconds ago. Also, its a storm in the middle of the night, so you don't have a good visual reference for speed. So now your car is slipping, at night, in a storm, lights are flashing everywhere, nothing makes sense since the instruments are not reading correctly. However, the car is working exactly as described in the manual. When the car ends up in a ditch, the investigation will find that the cause of the crash was driver error since the car was operating exactly as it was designed. Worth noting that Boeing (and just about every other aircraft on earth) has linked flight controls between the two pilot's positions that always behave in the exact same way so this type of failure could have never happened on a 737 for example. At the end of the day, this was pilot error, but more in a "You're holding it wrong, I didn't design it wrong" kind of way. After all, there were three people with a combined 20k flying hours, including thousand of hours in that design. If three extremely qualified pilots that have literal years of experience in that cockpit, who are rigorously trained and tested on a regular basis for emergencies in that cockpit, can fly the thing into the ground due to a cascade from a single human error... maybe the design of the user interface needs a look. You also conveniently skipped over the parts of the wikipedia article where they charged the manufacturer with manslaughter, and documented dozens of similar incidents, and the entire section outlining the Human Computer Interface concerns.
    - petcat 11mo ago
      As if Airbus hasn't suffered horrific crashes of their airplanes killing hundreds of people
- limagnolia 11mo ago
  It was only discovered after a flight experienced the issue, though. It could have been much more serious.
- nrhrjrjrjtntbt 11mo ago
  Well "I ain't going" didn't let a mere crash or two stop them.
  - SteveNuts 11mo ago
    Boeing?
- signatoremo 11mo ago
  But what does it say about their QA or lack of it? Just to be clear, I’m not faulting Airbus. I take issues with the shallow snark at Boeing. The JetBlue incident was serious. Airbus isn’t immune to controversies , like AF447 or Habsheem air show crash in 1988
  - 725686 11mo ago
    a problem on 1 flight in a gazillion, and you complain about QA?
    - signatoremo 11mo ago
      Yes? How do you know it’s not? They roll back to a previous version. How do they know that version isn’t prone to the same issue?
      - tjr 11mo ago
        Not involved with this particular matter. What I would want to see is logs of the behavior of the failing subsystem and details of the failing environment. This may be able to to be reproduced in an environmental testing lab, a systems rig lab, or possibly even in a completely virtual avionics test environment. If stimulating the subsystem with the same environmental input results in the error as experienced on the plane, then a fix can be worked from there. And likewise, a rollback to a previous version could be tested against the same environment.
  - gman83 11mo ago
    As software developers, we should perhaps refrain from criticizing aeronautical engineers' QA standards.
    - dylan604 11mo ago
      "push to prod, let the users debug for us" would at least, I'd hope, offer lower ticket prices for said users.
    - 11mo ago
loeg 11mo ago
The live feed buries the only useful information at the very bottom of the article: > The plane manufacturer says it has found that intense radiation from the Sun could corrupt data crucial to flight controls. > It’s thought most will be able to undergo a simple software update. > The issue was discovered after a JetBlue aircraft en-route from Mexico to the United States in October experienced a ‘sudden drop in altitude’. > The plane made an emergency landing, with reports at the time suggesting 15 to 20 people suffered minor injuries. > It’s thought the incident was caused by intense solar radiation, which corrupted data in a computer used to help control the aircraft.
- piker 11mo ago
  Interesting how radiation issues could be solved in software.
  - lazide 11mo ago
    Finally turning on the ECC RAM option?
  - mandeepj 11mo ago
    Maybe there's a range that requires a change? Now imagine, if it was over the air update, then maybe there would be no disruption?
  - matja 11mo ago
    Perhaps it's improving the checksum algorithm on network packets, or even ... adding one.
    - amelius 11mo ago
      Makes you wonder, if/how _passengers_ are directly protected against the radiation
      - fragmede 11mo ago
        They're not. Excessive high altitude flight increases your chance of developing melanoma. https://pmc.ncbi.nlm.nih.gov/articles/PMC9447865/ https://pmc.ncbi.nlm.nih.gov/articles/PMC9447865/
lysace 11mo ago
Is this the graph of the root cause? https://www.swpc.noaa.gov/products/solar-cycle-progression https://www.swpc.noaa.gov/products/solar-cycle-progression
- btown 11mo ago
  IMO saying that predictable solar cycles are a “root cause” is like calling Bobby Tables’ mom (https://xkcd.com/327/ https://xkcd.com/327/) the root cause of a SQL injection vulnerability.
dboreham 11mo ago
Something seems off when the same nonsensical report gets through the world's journalists (ok...arts graduates, but none of them is married to an engineer??), is published everywhere, then many hours elapse and still nobody has been able to make sense of it.
smetannik 11mo ago
Meanwhile Boeing would probably ignore any issue as any incident was already included in the annual budget.
- not_a_bot_4sho 11mo ago
  What sources are you basing this on?
  - yeahthereiss 11mo ago
    Do you understand humor?
worik 11mo ago
> interference from intense solar radiation, which corrupted data in a computer which controls the aircraft's elevation Has anybody kept count of "fly by wire" failures in aircraft? It fills me with dread that a computer programme is between the pilot's controls and the control surfaces. I am amazed that it works at all.
- gregoriol 11mo ago
  Mecanical failures happen too
  - chopin 11mo ago
    But now you have both...
BartjeD 11mo ago
Thank god we still have responsible businesses
adhambadr 11mo ago
Didn’t spacex fix this problem by just adding a ridiculous redundancy of components? i.e the solution might be to install more HW to parallelise ?

235 points by nrhrjrjrjtntbt 11mo ago

ChrisArchitect 11mo ago

More discussion: https://news.ycombinator.com/item?id=46082296 https://news.ycombinator.com/item?id=46082296

loloquwowndueo 11mo ago

At least they didn’t wait for a crash before doing this :/

Iridescent_ 11mo ago
The proper reaction when you have a potential issue in your engineering
- loloquwowndueo 11mo ago
  Sorry forgot to add “unlike that other aerospace company”
  - nickff 11mo ago
    Airbus is not immune to design & manufacturing issues with fatal consequences, they’re just not too-of-mind these days. A similar issue seems to have ‘cropped up’ on this flight: https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72 There was a television show (episode) about another design issue (which was fatal) some time ago: https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447
    - hexbin010 11mo ago
      > https://en.wikipedia.org/wiki/Air_France_Flight_447 https://en.wikipedia.org/wiki/Air_France_Flight_447 Quoting your link, "Final Report" section: > Temporary inconsistency between the measured speeds, likely as a result of the obstruction of the pitot tubes by ice crystals, caused autopilot disconnection and [flight control mode] reconfiguration to "alternate law (ALT)". - The crew made inappropriate control inputs that destabilized the flight path. - The crew failed to follow appropriate procedure for loss of displayed airspeed information. - The crew were late in identifying and correcting the deviation from the flight path. - The crew lacked understanding of the approach to stall. - The crew failed to recognize the aircraft had stalled, and consequently did not make inputs that would have made recovering from the stall possible. Note the numerous "the crew"
      - Arainach 11mo ago
        It's often easy to blame the humans in the loop, but if the UX is poor or the procedures too complicated, then it's a systems fault even if the humans technically didn't "follow procedure".
      - ern 11mo ago
        The relief second officer basically pulled up when the stall protection had been disabled and by the time the other pilot and captain realized what was happening it was too late to save the plane. There is a design flaw though: the sidesticks in modern Airbus planes are independent, so the other pilot didn’t get any tactile feedback when the second officer was pulling back.
        deleted 11mo ago
        [deleted]
        berti 11mo ago
        You do get an audible "DUAL INPUT DUAL INPUT" warning and some lights though [1]. It is never allowable to make sidestick inputs unless you are the single designated "pilot flying", but people can sometimes break down under stress of course. [1] https://safetyfirst.airbus.com/app/themes/mh_newsdesk/documents/archives/dual-side-stick-inputs.pdf https://safetyfirst.airbus.com/app/themes/mh_newsdesk/docume...
      - nickff 11mo ago
        Crews saved multiple 737-MAXs, but the public has focused on the aircraft whose crews were less effective.
      - dghlsakjg 11mo ago
        Faulting the crew is a common thing in almost all air incidents. In this case the crew absolutely could have saved the plane, but the plane did not help them at all. Part of the sales pitch of the Airbus is that the computer does A LOT of handholding for the pilots. In many configurations, including the one that the plane was flying in at the start of the incident, the inputs that caused the crash would have been harmless. In that incident the airspeed feed was lost to the computer and it literally changed the flight controls and turned off the safety limits, and none of the three people in the cockpit noticed. When an Airbus changes flight control modes, it does not keep inputs idempotent. Something harmless under one set of "laws" could crash the plane under another set of laws. In this case, what the pilot with the working control stick was doing would not have caused a crash, except that the computer had taken off the training wheels without anyone noticing. As a result of changing the primary controls one pilot was able to unintentionally place the plane in an unrecoverable state without the other pilots even noticing that he was making control inputs. Tack on that the computer intentionally disregarded the stall warning emanating from the AOA sensor as erroneous at a certain point and did not alert the pilots that the plane was stalled. You are taught from day one of flight training that if you hear the stall alarm you push the power in, and push the nose down until the alarm stops. In this case the stall warning came on, and then as the stall got worse, it turned itself off, with the computer under the mistaken belief that the plane could not actually be that far stalled. So the one alarm that they are trained to respond to in a certain way to recover the plane from a stall was silenced. If I was flying and I heard the stall alarm, then heard it stop, I would assume that I was no longer stalled, not that the plane was so far stalled that the stall alarm was convinced it had broken itself. So yes, the pilots flew the aircraft into the ground, but the computer suffered a partial failure and then changed how the primary flight controls operated. Imagine if the brake pedal, steering wheel, and accelerator all started responding to inputs differently when your car had a sensor issue. That causes the cruise control to fail. Add in that the cruise control failure turns off ABS, auto-brakes, lane assist, and stability control for some reason. Oh yeah, there's a steering control on the other side of the car on the armrest and the person sitting there can now make steering inputs, but it won't give feedback in your steering wheel, and also your steering wheel still can be manipulated when the other guy is steering, but it is completely disconnected from the tires while the other guy is steering. All of the controls are also more sensitive now, and allow you to do things that wouldn't have been possible a few seconds ago. Also, its a storm in the middle of the night, so you don't have a good visual reference for speed. So now your car is slipping, at night, in a storm, lights are flashing everywhere, nothing makes sense since the instruments are not reading correctly. However, the car is working exactly as described in the manual. When the car ends up in a ditch, the investigation will find that the cause of the crash was driver error since the car was operating exactly as it was designed. Worth noting that Boeing (and just about every other aircraft on earth) has linked flight controls between the two pilot's positions that always behave in the exact same way so this type of failure could have never happened on a 737 for example. At the end of the day, this was pilot error, but more in a "You're holding it wrong, I didn't design it wrong" kind of way. After all, there were three people with a combined 20k flying hours, including thousand of hours in that design. If three extremely qualified pilots that have literal years of experience in that cockpit, who are rigorously trained and tested on a regular basis for emergencies in that cockpit, can fly the thing into the ground due to a cascade from a single human error... maybe the design of the user interface needs a look. You also conveniently skipped over the parts of the wikipedia article where they charged the manufacturer with manslaughter, and documented dozens of similar incidents, and the entire section outlining the Human Computer Interface concerns.
  - petcat 11mo ago
    As if Airbus hasn't suffered horrific crashes of their airplanes killing hundreds of people
limagnolia 11mo ago
It was only discovered after a flight experienced the issue, though. It could have been much more serious.
nrhrjrjrjtntbt 11mo ago
Well "I ain't going" didn't let a mere crash or two stop them.
- SteveNuts 11mo ago
  Boeing?
signatoremo 11mo ago
But what does it say about their QA or lack of it? Just to be clear, I’m not faulting Airbus. I take issues with the shallow snark at Boeing. The JetBlue incident was serious. Airbus isn’t immune to controversies , like AF447 or Habsheem air show crash in 1988
- 725686 11mo ago
  a problem on 1 flight in a gazillion, and you complain about QA?
  - signatoremo 11mo ago
    Yes? How do you know it’s not? They roll back to a previous version. How do they know that version isn’t prone to the same issue?
    - tjr 11mo ago
      Not involved with this particular matter. What I would want to see is logs of the behavior of the failing subsystem and details of the failing environment. This may be able to to be reproduced in an environmental testing lab, a systems rig lab, or possibly even in a completely virtual avionics test environment. If stimulating the subsystem with the same environmental input results in the error as experienced on the plane, then a fix can be worked from there. And likewise, a rollback to a previous version could be tested against the same environment.
- gman83 11mo ago
  As software developers, we should perhaps refrain from criticizing aeronautical engineers' QA standards.
  - dylan604 11mo ago
    "push to prod, let the users debug for us" would at least, I'd hope, offer lower ticket prices for said users.
  - 11mo ago

loeg 11mo ago

The live feed buries the only useful information at the very bottom of the article: > The plane manufacturer says it has found that intense radiation from the Sun could corrupt data crucial to flight controls. > It’s thought most will be able to undergo a simple software update. > The issue was discovered after a JetBlue aircraft en-route from Mexico to the United States in October experienced a ‘sudden drop in altitude’. > The plane made an emergency landing, with reports at the time suggesting 15 to 20 people suffered minor injuries. > It’s thought the incident was caused by intense solar radiation, which corrupted data in a computer used to help control the aircraft.

piker 11mo ago
Interesting how radiation issues could be solved in software.
- lazide 11mo ago
  Finally turning on the ECC RAM option?
- mandeepj 11mo ago
  Maybe there's a range that requires a change? Now imagine, if it was over the air update, then maybe there would be no disruption?
- matja 11mo ago
  Perhaps it's improving the checksum algorithm on network packets, or even ... adding one.
  - amelius 11mo ago
    Makes you wonder, if/how _passengers_ are directly protected against the radiation
    - fragmede 11mo ago
      They're not. Excessive high altitude flight increases your chance of developing melanoma. https://pmc.ncbi.nlm.nih.gov/articles/PMC9447865/ https://pmc.ncbi.nlm.nih.gov/articles/PMC9447865/

lysace 11mo ago

Is this the graph of the root cause? https://www.swpc.noaa.gov/products/solar-cycle-progression https://www.swpc.noaa.gov/products/solar-cycle-progression

btown 11mo ago
IMO saying that predictable solar cycles are a “root cause” is like calling Bobby Tables’ mom (https://xkcd.com/327/ https://xkcd.com/327/) the root cause of a SQL injection vulnerability.

dboreham 11mo ago

Something seems off when the same nonsensical report gets through the world's journalists (ok...arts graduates, but none of them is married to an engineer??), is published everywhere, then many hours elapse and still nobody has been able to make sense of it.

smetannik 11mo ago

Meanwhile Boeing would probably ignore any issue as any incident was already included in the annual budget.

not_a_bot_4sho 11mo ago
What sources are you basing this on?
- yeahthereiss 11mo ago
  Do you understand humor?

worik 11mo ago

> interference from intense solar radiation, which corrupted data in a computer which controls the aircraft's elevation Has anybody kept count of "fly by wire" failures in aircraft? It fills me with dread that a computer programme is between the pilot's controls and the control surfaces. I am amazed that it works at all.

gregoriol 11mo ago
Mecanical failures happen too
- chopin 11mo ago
  But now you have both...

BartjeD 11mo ago

Thank god we still have responsible businesses

adhambadr 11mo ago

Didn’t spacex fix this problem by just adding a ridiculous redundancy of components? i.e the solution might be to install more HW to parallelise ?