4 ms·

> went into a sudden dive yesterday due to a 'technical issue' injuring 50 people, some of which were "thrown into the roof" Reminiscent of this: https://en.wi

> went into a sudden dive yesterday due to a 'technical issue' injuring 50 people, some of which were "thrown into the roof" Reminiscent of this: https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72 The 2008 incident, this time on an Airbus, is interesting to HN readership as an unexplained hardware bit flip in one of the flight systems that calculates the angle of attack generated garbled data. This system was triple-redundant; however, it subsequently triggered a then-unknown software defect in the flight computer's algorithm for discarding such erroneous data. Some sort of edge case that wasn't considered. The result was the plane entered a dive in a manner eerily similar to the MCAS incident.

atoav 3y ago
To me a unaccounted edge case in a tripple redundant system that can be fixed by an update forever feels different than Boeing routinely missing screws and not documenting them. Don't get me wrong, both things should never happen, but the first can even happen to very good teams with stellar safety and testing culture, while the latter is indicative of a broad problem. Missing an edge case nobody knew existed is not the same as not doing a very obvious thing that you should be able to do.
- wannacboatmovie 3y ago
  Agreed, I was responding to the GP's mention of the recent 787 incident, a bit offtopic but more technically interesting; I don't believe it made the front page.
DanielHB 3y ago
Wow, how do you even test bit flips from the procurer side (Airbus in this case)? I work in industrial manufacturing (although not as mission-critical as a plane) and suppliers have surprisingly amount of problems in their parts and we can't (and shouldn't) need to stress-test every single part we buy from 3rd parties. Just last month we found out one of the bluetooth chips we use is giving kernel panics seemingly randomly once a week. It can only be fixed with a power-cycle. Supplier just shrugged and asked us to run some diagnostics software, we gave up on using bluetooth for this product instead (we had a physical display as an extra addon, but now it is mandatory). Note this is a million-dollar product and we were lucky to have caught this problem before release. We actually accounted for a similar problem for our network communication messages by adding CRC to our messages. https://en.wikipedia.org/wiki/Cyclic_redundancy_check https://en.wikipedia.org/wiki/Cyclic_redundancy_check But bit flips inside a microcontroller or memory is beyond our capabilities to detect or simulate. Note I am not one of the embedded engineers they might have some safeguards for this on top of trusting the hardware error-correction that I don't know about.

atoav 3y ago

To me a unaccounted edge case in a tripple redundant system that can be fixed by an update forever feels different than Boeing routinely missing screws and not documenting them. Don't get me wrong, both things should never happen, but the first can even happen to very good teams with stellar safety and testing culture, while the latter is indicative of a broad problem. Missing an edge case nobody knew existed is not the same as not doing a very obvious thing that you should be able to do.

wannacboatmovie 3y ago
Agreed, I was responding to the GP's mention of the recent 787 incident, a bit offtopic but more technically interesting; I don't believe it made the front page.

DanielHB 3y ago

Wow, how do you even test bit flips from the procurer side (Airbus in this case)? I work in industrial manufacturing (although not as mission-critical as a plane) and suppliers have surprisingly amount of problems in their parts and we can't (and shouldn't) need to stress-test every single part we buy from 3rd parties. Just last month we found out one of the bluetooth chips we use is giving kernel panics seemingly randomly once a week. It can only be fixed with a power-cycle. Supplier just shrugged and asked us to run some diagnostics software, we gave up on using bluetooth for this product instead (we had a physical display as an extra addon, but now it is mandatory). Note this is a million-dollar product and we were lucky to have caught this problem before release. We actually accounted for a similar problem for our network communication messages by adding CRC to our messages. https://en.wikipedia.org/wiki/Cyclic_redundancy_check https://en.wikipedia.org/wiki/Cyclic_redundancy_check But bit flips inside a microcontroller or memory is beyond our capabilities to detect or simulate. Note I am not one of the embedded engineers they might have some safeguards for this on top of trusting the hardware error-correction that I don't know about.