56 ms·
This isn't exactly right. The aircraft has two sensors, but the system in question (MCAS) is only ever looking at one of them. That is part of what the software
by ams6110 8y ago
This isn't exactly right. The aircraft has two sensors, but the system in question (MCAS) is only ever looking at one of them. That is part of what the software fix is reportedly going to address.
- JorgeGT 8y agoThat... makes it even worse? Why would someone on Earth disregard an available extra sensor for a system that has unlimited authority to bring the horizontal stabilizer to full deflection?
- ams6110 8y agoYeah I don't disagree. I don't fly, but have sort of been following this out of curiosity. One comment I've seen from some pilots is basically "ehh, runaway trim isn't a new thing, we train for it in the sim, and there's a standard way to deal with it (disengage the automatic system and trim manually)." So perhaps Boeing felt that this didn't really change anything in that regard. They seem to have been proven wrong.
- CaptainZapp 8y agoKey quote is training on a simulator That's a tall order when the manufacturer doesn't inform their customers out of commercial expediency.
- salawat 8y agoNo simulator accounted for a borked AoA sensor being able to nosedive the plane, or the altered aerodynamics from the different engine configuration. So while yes, runaway trim is a thing, the circumstances under which an error could happen are substantially different. For instance, a pilot could check the maintenance log book and see some work or inspection was done on the auto-trim subsystem. This would prime the pilot to be more on the alert for the possibility of trim misbehavior during the flight. Crisis averted, right? But no pilot would look at an entry for an AoA sensor being off or worked on, and would think, "Boy, better look out for that safety system I never knew existed that could cause a trim runaway because my previously airworthyness agnostic AoA sensor is unreliable." Flying and complex system diagnosis in time sensitive conditions requires extensive ahead of time mental model creation ahead of time.
- lutorm 8y agoIt is weird, though, that this was not caught in simulations (not training simulations, but in design simulations.) I would assume that running simulations to see how the design behaves when sensors fail would be an absolutely standard thing to do. I don't work on airplanes, but we routinely simulate sensor failures and their effect on software systems and you'd think this would not be optional for passenger aircraft certification.
- salawat 8y agoWhen all of your testing is in house, and you don't have people who are willing to put the project at jeopardy for the sake of doing it right, you'll be amazed what can get overlooked. If anything, the process of engineering is most difficult in that answering most questions tends to be straightforward (not easy, but straightforward once you know the right methods to apply), while figuring out if you've asked all the right questions is the thing that keeps me awake at night. If you don't ask and dig in until you've answered it fully, it's easy to get blind sided. I have difficulty believing it could have gone so horribly wrong now, but I can't deny that just from the information available, even if MCAS isn't the root cause of the Ethiopian Air crash, there is some egregious failures in sound practice going on with Boeing to have slipped up this badly. That's the thing with sound Engineering, when you do it, it just verks. When you don't...
- inferiorhuman 8y agoTechnically MCAS activation isn't runaway stabilizer trim, and I'm sure that tripped up the Lion Air pilots. Check out the quick reference for a runaway stabilizer[1]. Note the steps: * Control airplane pitch attitude manually with control column and main electric trim as needed * If the runaway stops, stop. Well, electric trim input on the yoke will stop MCAS temporarily. Of course if these guys had any experience on an NG they're already used to the computers trimming the plane in a counterintuitive manner via the speed trim system (STS). So not only is MCAS not a runaway trim situation, but pilots flying the NG will get used to the computer trimming the stabilizer "at random". 1: http://www.737ng.co.uk/737-800%20Quick%20Reference%20Handbook%20(QRH).pdf http://www.737ng.co.uk/737-800%20Quick%20Reference%20Handboo...
- mannykannot 8y agoThis sort of MCAS failure presents itself to the crew as a trim runaway, and the instructions in the document continue: 4 If the runaway continues: STAB TRIM CUTOUT switches (both) . . CUTOUT If the runaway continues: Stabilizer trim wheel . . . . . Grasp and hold - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 5 Stabilizer . . . . . . . . . . . . Trim manually (That bit about grasping and holding the trim wheel was a surprise to me.) This is what the Lion Air crew of the flight before the one that crashed did, in response to the same sort of MCAS failure, and completed the flight without further trouble. While Boeing made a serious error in hiding the differences between this variant and its predecessors, it seems that the prior trim runaway procedure does work for this sort of MCAS failure (unless evidence to the contrary comes out of the Ethiopian Airways investigation.)
- erichocean 8y agoBoeing is "right" in the sense that, if you follow the normal 737 flight manual, you'll be fine, even with MCAS enable or even MCAS misbehaving due to the sensor readings being bad or whatever. I think the problem is that the symptoms are different from other 737s, and if you don't KNOW about the MCAS stuff, that's very surprising. So as humans, even though logically the same procedure applies, instinctively you don't think to do it. And given the very short time frames involved, and the margin of error, it's not too surprising that these events have occurred. Perhaps the need-to-retrain criteria needs to include the following amendment: If the craft exhibits new symptoms, even if the response to those symptoms are covered correctly be the old operations manual, pilots will need to be retrained to account for the new symptoms.
- wahern 8y agoIf you only have two sensors and you're taking input from both sensors, assuming each sensor has the same probability of malfunction then you're doubling the chance of processing bad input. You could choose to select the input that results in the least-worst bad outcome. But that may not bring the total risk below the increased risk of utilizing both sensors. The sensors are doing something purposeful, after all. So it may be logical what they're doing. OTOH, the logic may make sense only in a path dependent sort of way, after a series of bad decisions that put them in a corner with poor options. There probably really should be 3 or more sensors.
- chopin 8y agoYou take input from both sensors only when they don't disagree (by a margin). There may be failure modes where both have the same bad input but I don't think that this doubles your chances of processing bad input.
- wahern 8y agoYou have double the chance that one of the sensors is bad. Which one do you choose? The one that says AoA is normal? Maybe that's the one that is bad. Worse, maybe such a bad input is more likely in a stall scenario. Without any ability to differentiate bad from good input, you may be better off simply relying on a single sensor at any one time and completely ignoring the other, at least as far as controlling MCAS is concerned. Tossing up a warning that says the sensors disagree is another matter.
- chopin 8y agoI meant disabling the system if they disagree. I agree that one can argue that this doubles the chance of system shutdown compared to properly detecting that one sensor fails. That's obviously only ok if the system is expendable to a certain degree.
- inferiorhuman 8y agoBecause there was no available extra sensor. Trevor gets a few other things wrong too. The elevator feel system (EFS) uses multiple sensors, in fact it has its own pitot probes[1] (two of them) on the elevator. It's not clear if it uses both, but I suspect it also uses both alpha vanes up front like MCAS should be doing. Input from the alpha vane (AoA sensor) is new on the MAX, IIRC. His tweets were a bit ambiguous, but the elevator feel system does not change the trim at all (also EFS was non-op on the Lion Air bird). It's almost certainly NOT a sensor problem. Look at the graphs from the black box on the Lion Air flight. The angle-of-attack was almost exactly offset by twenty degrees left to right[2]. That sensor was working just fine, but something else on the path was fucked up (bad wiring? bad ADIRU? both have happened before). The fact that the alpha vane was replaced and that didn't fix the angle-of-attack readings supports this. Also, the 737 NG and 737 MAX use identical alpha vanes. While the angle-of-attack data is less important on an NG, we haven't seen people come out of the woodwork talking about how common alpha vane failures are on an NG. I'm inclined to believe that they're not all that common. The Lion Air flight crew did write up the sensor problems[3]. I saw another one of these "I'm a software engineer so I've got great insight" posts[4] and chuckled. It's a bit of swing and a miss as well. MCAS is hardly silent or subtle. The same trim wheels his Cessna have are present on the 737 and they're noisy[5]. MCAS doesn't quite override the pilot either, and you can see that on the black box graphs from the Lion Air flight[2]. Twenty times the pilot flying entered opposite trim to counteract MCAS and twenty times MCAS paused for five seconds. On the twenty-first time something changed and the plane crashed. It's not quite that simple and that's why there are active investigations. 1: https://4.bp.blogspot.com/-ZsL1asAUBRU/W-pSdWs2dYI/AAAAAAAAFBg/RlymmfQCny8tUpNqZW6deaS-6GXWPL5KQCLcBGAs/s1600/Screen%2BShot%2B2018-11-12%2Bat%2B8.26.08%2BPM.png https://4.bp.blogspot.com/-ZsL1asAUBRU/W-pSdWs2dYI/AAAAAAAAF... 2: https://reports.aviation-safety.net/2018/20181029-0_B38M_PK-LQP_PRELIMINARY.pdf https://reports.aviation-safety.net/2018/20181029-0_B38M_PK-... 3: https://theaircurrent.com/wp-content/uploads/2018/10/jt43-jt610-lionair-mx-log-annotated.jpeg https://theaircurrent.com/wp-content/uploads/2018/10/jt43-jt... 4: https://medium.com/@jpaulreed/the-737max-and-why-software-engineers-should-pay-attention-a041290994bd https://medium.com/@jpaulreed/the-737max-and-why-software-en... 5: https://www.youtube.com/watch?v=3pPRuFHR1co https://www.youtube.com/watch?v=3pPRuFHR1co
- 8y ago
- m463 8y agoserious question: which sensor has failed? you need at least three sensors. (although the "sensors disagree" option you can purchase is better than nothing)
- im_down_w_otp 8y agoFor a system with the responsibilities and impact on the safety-case that MCAS has, not even a 1oo2D design would be sufficient, since there's no clear way to determine which input is correct or incorrect if they diverge, but worse still is that MCAS doesn't seem to leverage even that basic, yet still insufficient, safety design from what I've been able to find on it. At the end of the day the fail-safe is to throw control back to the human. And, then of course pin all the liability onto the human operator when things go wrong. This was the problem I had with the Lion Air conclusion. Sure, it may be the case that a pilot can potentially override this system with situational awareness and training. However, the pilot didn't create this band-aid pile in the first place.
- tzs 8y ago> For a system with the responsibilities and impact on the safety-case that MCAS has, not even a 1oo2D design would be sufficient, since there's no clear way to determine which input is correct or incorrect if they diverge Wouldn't that depend on how often MCAS is actually needed, and how it handles divergent sensors? My understanding as a layman is that there are situations where a 737MAX is more likely to stall than a "regular" 737 would, but I haven't seen anything on how often those occur. Are they expected to routinely occur, so a fully functioning MCAS is important for safe operation of the plane? Or are they supposed to be rare, one in several million events, and the MCAS is there to make handling those events the same as they would be handled in a "regular" 737, to minimize the need for retraining? If the later, then I'd expect two sensors would be fine, or even just one sensor IF there was a reliable way to tell if it had failed, as long as the MCAS response to a sensor problem is to disable itself and let the pilots know they have to stay out of those "more likely to stall" situations for the rest of the flight.
- salawat 8y agoThe problem comes in where the MCAS use should be infrequent, unless the sensor is borked. In computing, we have a principle stated Garbage in, Garbage out. Circuits and electronics don't think. They compute. There is no error checking except that which is specifically designed and implemented into the system. If you're getting data from a biased sensor with +/- 10 degrees AoA, a 10 degree actual AoA (well within the safe operating envelope) suddenly appears to MCAS as a 20 degree AoA (oh shit territory). The system therefore engages doing exactly what it was designed to do. Thus is the crux of the matter. The pilot was flying safely while his AoA sensor was telling a safety system he knew nothing about that he was flying dangerously. The AoA system on earlier models of aircraft that the MAX was based on were a functional luxury/situational awareness aid. The flyability was not impacted by a horked sensor. That changed once they had to add a software driven mechanism to keep the flight characteristics similar enough with the old airframe to be able to release the aircraft to 737 trained pilots, and not have to worry about retraining. Maintenance and pilot alike both needed to be aware that the AoA sensor became a safety critical component due to a failure or miscalibration jeopardizing the controllability of the airframe. If they had gone through a full recert of this airframe, and not an expedited self-certify/grandfathering, these tragedies would have had a much smaller likelihood of occurring due to the increased scrutiny. Props on Brazil for doing their own footwork, and not blindly trusting the FAA's delegation to the manufacturer. Personally, as a software engineer and quality assurance specialist, if I'd seen anything remotely like this come across my desk, I'd be raising hell, even if it meant yanking someone into a VP/C's office and giving them a dressing down for skimping on a cross-cutting safety critical concern, deadlines be damned.
- blattimwind 8y agoAIUI the entire FCC only uses one set of sensors and which set of sensors is used is switched together with the FCC?
- cameldrv 8y agoWe will see. Part of the avionics design is that there is a “Brick Wall” between the two sets of sensors, air data computers, and flight control computers. If the FCC uses both for MCAS, that safety concept doesn’t exist anymore. That is certainly the reason they didn’t compare the AoA sensors in the MCAS logic. I think they could probably use pitot/static as a cross check on AoA, but it has some tricky edge cases.
- clavoie 8y agoSo, a modern aircraft probably have a few thousands sensors (if not way, way more), not strictly redundant but overlapping in some fashion -- I have to believe there's been some modelling trying to check that sensor input is consistent; if only to have some kind of way for the computer to figure out that a sensor has gone crazy (say your compass suddenly detects a significant change in heading, but no sensor has detected even a degree of roll, a change in speed, heading computed by GPS over time is stable... compass might be borked, whaddya say?) Why isn't such modelling used to let sensors be disabled when they're clearly sending bogus input?
- rocqua 8y agoMy guess is that such systems are complex, hard to model for safety assurances, and hard to reason about in the moment. Yes, it will probably make the system safer, but your guarantees become a lot more fuzzy. This is not the field for fuzzy guarantees.
- de_watcher 8y ago> Why isn't such modelling used to let sensors be disabled when they're clearly sending bogus input? You need to crash a lot of planes to train that NN.
- m463 8y agothere should be at least 3 AOA sensors - so you can not only detect a mismatch, but figure out which sensor is most probably incorrect.
- cmurf 8y agoReports say it alternates each flight between them; if that's correct, in the case of one good sensor and one bad, that you wouldn't see back to back flights with anomalous behavior related to a bad sensor. Otherwise, perhaps the problem isn't with the sensor itself, but with some part of how sensor information is communicated to the interpretive computer.