15 ms·

Citing a serious flight test incident, FAA slows Boeing 777X certification

168 points by donohoe 5y ago

trollied 5y ago
> The FAA cited a long litany of concerns, including a serious flight control incident during a test flight on Dec. 8, 2020, when the plane experienced an “uncommanded pitch event” — meaning the nose of the aircraft pitched abruptly up or down without input from the pilots. Crikey. MCAS all over again. Something at Boeing is very, very out of control.
- Aperocky 5y ago
  What could go wrong if we just fire all the programmers that make too much and outsource the software?
  - lawless_c 5y ago
    The programmers are just trying to patch what once again is likely a hardware problem. If it's like the MAX the real issue is the flight characteristics
    - _ph_ 5y ago
      Basically every modern jet aircraft with relatively large engines has some sort of MCAS system. If you raise the nose of such a plane the balance does shift. This isn't a form of an aerodynamic error, it is a consequence of having large engines hanging off your wings (a lot of classic airplanes have the engines in or very close to their central axis). If you want the appropriate feedback in the steering column, you have to compensate for the shift. Airbus does it as well.
      - amelius 5y ago
        What is the advantage of the engines further from the central axis?
        smegcicle 5y ago
        iirc that's another term for larger, more efficient engines hanging lower from the same wings
        dotancohen 5y ago
        Time, money. They don't need to redesign the wings to accommodate the newer, larger engines.
        coredog64 5y ago
        There's only so much room for the landing gear. Larger diameter engines require longer landing gear. Increasing the space given to landing gear requires changes to the wing box. Changes to the wing box is fundamentally a wing redesign.
        Gwypaas 5y ago
        Which is why the A320 is designed to be loaded with conveyors. This means it has no problem accommodating an even larger nacelle than the 737 in a regular position instead of the up and forward the 737 Max resorted to. Even the previous generation 737 was running into this issue and that's why they went with the distinct oval nacelles.
        _ph_ 5y ago
        The main driver is the desire to make engines bigger, thus more fuel-efficient. The fuel savings of the newest engine generations are significant. If you hang engines from your wings (I assume that has practical reasons, it is easier to hang the engine than include it into the wing, it is also easier to maintain), larger engines mean engines further from the axis. That applies to all modern airplanes which share this configuration. A complete redesign of the 737 wouldn't have made this go away either. However the goal of keeping the same type rating made the especial design of MCAS necessary.
      - dundarious 5y ago
        MCAS-like systems exist elsewhere, but MCAS specifically is worthy of criticism, as both an engineering failure and more importantly IMO a management failure. Specifically with regard to the flight characteristics, MCAS is a band-aid on a problem created by cost-minimization -- the desire to avoid a hardware redesign (to properly accommodate the larger engines) and/or a re-certification.
        _ph_ 5y ago
        Of course, the actual implementation of MCAS was a disaster. Especially, that they didn't deactivate it automatically on AOA disagree, that on top of that the screen signal for AOA disagree was not shown due to a separate issue and of course that late in development MCAS was given more control authority than initially planned. My point just was, a complete redesign of the 737 still would have to deal with the same aerodynamic issues as the 737 MAX, as they are rather fundamental to this kind of jets. But indeed, it would have been easier to deal with it with a new design. It is less mentioned, that shortly after the MAX crashes, Airbus mandated that in some planes the last rows were kept empty until some software updates were deployed. So they seem to have discovered some boundary conditions they were not happy with and decided to be rather safe till they augmented the software.
        FabHK 5y ago
        > late in development MCAS was given more control authority than initially planned ... without triggering re-evaluation of the seriousness/consequences of failure. That meant that failures of MCAS were considered unproblematic, which meant that lower levels of redundancy were deemed acceptable. Quite a process/management failure.
        _ph_ 5y ago
        Well, technically the failures of MCAS should be unproblematic, as the stabilizer runaway protocol is well trained for any 737 pilot and would cope with a MCAS misfunction. The problem is: in the two crashes the pilots either completely failed to follow the protocol (Lion Air) or to late and failed to recover (Ethiopian Airlines).
      - wyldfire 5y ago
        The bonkers thing seemed to me to be having the notification and/or override as an opt-in feature you could buy. Southwest didn't want to train pilots and newly visible behavior requires new training. No surprise that airlines elsewhere in the world also skimped on the "optional" features to the doom of the passengers and crew.
        carl_dr 5y ago
        The opt-in feature you mention was for a disagree alert for the plane’s angle of attack sensors. But, MCAS was only ever using a single angle of attack sensor as its source of truth, so if there was a fault in that single sensor, MCAS could activate at normal attitudes. But the real problem was that Boeing hadn’t documented MCAS, to avoid the requirement for pilots to obtain type certification for the MAX (so any existing 737 pilot could to fly it without additional training - and it wasn’t just Southwest objecting to this training), and to avoid a full FAA certification process for the MAX. So on an MCAS activation, even if they had had an AoA alert, pilots would not know how to instinctively deal with it. (I don’t doubt an AOA disagree alert would give invaluable information allowing the pilots to rule out most failures, which when only a few thousand feet high would maybe have saved both planes.) On the previous day to the Lion 610 crash, the incident aircraft suffered an MCAS activation and fortunately a pilot in the jump seat realised what was happening and what was needed to deactivate it. It seems being sat behind the trim wheels (so he could see they were moving) and not wrestling to keep the plane in the sky at the same time was needed for that to happen. It must have been terrifying.
        coredog64 5y ago
        737 trim wheels are very visible and very loud. The noise and motion they create have been a joke within the 737 community for at least 20 years.
      - Gravityloss 5y ago
        Yes, it's the aerodynamic lift from the big engine fans that was the 737 Max issue, according to Aviation Week. Fans have gotten bigger as engines have improved and the bypass ratios and fuel efficiency has gotten better. Think of the engines as a big horizontal stabilizer but it's at the front, in front of the center of gravity. Normally it doesn't do much anything since the angle of attack is zero. But if you increase pitch, it will increase nose-up momentum. For some reasons it's not common to put the engines further back. DC-9 style, on the sides of the rear fuselage might work. Or maybe over the wing and slightly to the rear, like Hondajet.
      - himinlomax 5y ago
        The difference is that all Airbus planes since the 1980's A320 are built around computerized flight commands, so the code paths involved are exercised constantly. MCAS only triggers occasionally, meaning if it has a bug, it only shows up at the worst possible time. Airbus planes also have double redundancy (three inputs and computers) for all those systems, where MCAS had none by default.
    - landemva 5y ago
      Won quoted as, “The technical data required for type certification has not reached a point where it appears the aircraft type design is mature ..." The 737MAX should have gone through a new type certification, but didn't to allow for less pilot retraining. FAA failed to properly serve the public. New type cert finds flaws and saves lives.
    - WalterBright 5y ago
      > If it's like the MAX the real issue is the flight characteristics The real issue was the reliance on one sensor coupled with too much authority give to MCAS and inadequate failure analysis. And pilots who did not read, remember, or follow the 2 step recovery process in the Emergency Airworthiness Directive sent to all MAX pilots.
      - WalterBright 5y ago
        I always get a negative reaction for mentioning the Emergency Airworthiness Directive sent to all MAX pilots. Media accounts never mention it, either. The Boeing Emergency Airworthiness Directive says: "Initially, higher control forces may be needed to overcome any stabilizer nose down trim already applied. Electric stabilizer trim can be used to neutralize control column pitch forces before moving the STAB TRIM CUTOUT switches to CUTOUT. Manual stabilizer trim can be used before and after the STAB TRIM CUTOUT switches are moved to CUTOUT." https://theaircurrent.com/wp-content/uploads/2018/11/B737-MAX-AD-1107.pdf https://theaircurrent.com/wp-content/uploads/2018/11/B737-MA... For those who are skeptical of mainstream media fake news on the topic, here's the official final report on the Lion Air crash: 2018 - 035 - PK-LQP Final Report http://knkt.dephub.go.id/knkt/ntsc_aviation/baru/2018%20-%20035%20-%20PK-LQP%20Final%20Report.pdf http://knkt.dephub.go.id/knkt/ntsc_aviation/baru/2018%20-%20... I also recommend the recent Aviation Disasters episode on it. While shallow, because it has to fit in 40 minutes, I was shocked that it was surprisingly accurate. Kudos to the Smithsonian channel. I wouldn't be surprised if AD waited until that report was released before doing the episode, so it would be using verifiable facts.
        DangitBobby 5y ago
        What are you trying to say, exactly?
        WalterBright 5y ago
        Not sure what you find confusing about it.
  - dundarious 5y ago
    Not sure if you're being serious, and while it hasn't been tried AFAIK for the flight software specifically, but cost-minimization and outsourcing has been credited as the major sources of Boeing's safety problems for the last decade (737 MAX, 787 Dreamliner, 737NG). Doing more of it would be neither a change of course nor an improvement IMO.
    - eganist 5y ago
      > Not sure if you're being serious, and while it hasn't been tried AFAIK for the flight software specifically, but cost-minimization and outsourcing has been credited as the major sources of Boeing's safety problems for the last decade (737 MAX, 787 Dreamliner, 737NG). Doing more of it would be neither a change of course nor an improvement IMO. I hear you, but I suspect aperocky was criticizing that exact approach. I.e it felt like sarcasm.
      - dundarious 5y ago
        Makes sense, I was genuinely 50/50 about it being sarcastic, but I probably should have been more confident of it, and at the very least should have given more credence to that interpretation than "not sure if...".
        Dylan16807 5y ago
        I'm honestly confused that you give it 50/50. Even ignoring the context, the "we" in "what could go wrong if we" only works if it's a rhetorical question. And from there, analyzing it as a rhetorical question, it seems obvious to me that the implied answer isn't "nothing". Do you disagree with either part of my reasoning?
        dundarious 5y ago
        On HN, especially if there's talk of labor (or god forbid, unions), there are often some pretty extreme opinions. And I said I shouldn't have given it just 50/50 odds. I don't see the point in continuing this postmortem.
  - dotancohen 5y ago
    Actually, the responsible bit of software was already outsourced. There was an HN thread on that some time ago.
    - Aperocky 5y ago
      Yes this is what I'm referring to.
  - _ph_ 5y ago
    This is really a crucial point. I am a software developer in an engineering company. What is essential for my job is not only my programming skills, but also a decent understanding of the engineering I deal with. This can only be achieved by working long enough in the job, be it an engineer who picks up programming or a software developer who learns the engineering involved. This is of course much more expensive than picking up any contractor, who might be brilliant at programming, but lacks the domain knowledge that only grows over time, but it is a good way of having more eyes spotting potential spec issues.
    - rbanffy 5y ago
      I really fail to understand the logic that may make someone to think a software engineer, however brilliant, can write software for a domain they don’t understand.
      - slipframe 5y ago
        Wishful thinking.
      - Aperocky 5y ago
        If this brilliance can be translated to understanding in other fields, which usually it can. This requires studying, which firms simply tries to avoid. I am a software engineer now, but my major was mechanical engineering, I can barely remember any of my coursework, but I understand how physics work in general, and a small bonus on how to disect a system. Nobody can know everything in today's world and that's why common engineering/sense training is so important IMO.
        vidanay 5y ago
        Similar. I used to design automation equipment, but I've switched to software development. I'm still closely connected to the mechanical side though. More than once I've had to explain to someone who is a competent programmer that they can't accelerate mass x at rate y.
      - pjmlp 5y ago
  - ricardobayes 5y ago
    What the cookie-cutter management doesn't understand is 100% of the company's value goes home after 5PM.
  - throwaway713 5y ago
    I suddenly became afraid of flying when I started working for one of the companies known for having some of the best software engineers in the world and seeing how many bugs made it into prod. I can't imagine who is programming the flight computers for Boeing and what they're being paid. It's gotten bad enough that I refuse to fly on any plane that was released in the last decade. At least the older planes have some decent trip statistics to back up their safety.
  - WalterBright 5y ago
    The MCAS problem had nothing whatsoever to do with outsourcing. The problem was in the specification of its behavior.
    - metalforever 5y ago
      This does appear to be related to bad engineering or engineers that do not understand how to program safety systems.
      - WalterBright 5y ago
        Of course, but the specifications are normally written by Boeing, and the implementation is outsourced. The reason is the system as a whole has to function, meaning the spec has to be written by the team that has knowledge of the whole system. MCAS interfaces with quite a few other systems on the airplane.
        _ph_ 5y ago
        Even a good team can convince itself, that it is right. It would have added a "second set of eyes", if the implementors of MCAS had their own expertise in aviation. No guarantee, but a chance of spotting the problems with the spec.
  - Guest42 5y ago
    It is my understanding that in aerospace much of the software is outsourced and is then QAed by internal contractors.
- vmception 5y ago
  My understanding is that they are trying to make planes that are too big to actually fly without continual software correction Can they just not do that?
  - VBprogrammer 5y ago
    The 777 is fly by wire so it wouldn't be a good idea.
  - deergomoo 5y ago
    My understanding was that they’re trying to use software to make their new planes fly like their old planes (at least that was the reasoning for MCAS in the 737 MAX). In super basic terms, if the new plane doesn’t fly like the old plane, all the pilots need to be retrained. This is a significant expense that could make a serious dent in sales. I don’t think it’s anything to do with size, there are larger planes than the 777X that are perfectly safe (and of course, the 737 is pretty small as far as airliners go).
    - LatteLazy 5y ago
      Isn't there also some weird requirement that pilots only fly one/two airframes at a time? So training for a 777 means your can't fly 747 or something else anymore?
      - sp332 5y ago
        I don't think so. But it was easier to sell to airlines if it didn't require retraining.
        Dylan16807 5y ago
        It seems like there's a very strong incentive to avoid retraining coming from somewhere, at least. Surely that's not just airlines being stubborn?
        5y ago
xiphias2 5y ago
In a statement Friday, Boeing said it “remains fully focused on safety as our highest priority throughout 777X development.” Now finally that FAA doesn't want to repeat the MAX mistake, I believe Boeing.
xjwm 5y ago
More negative press for Boeing, which they probably don't need after all the 737 MAX issues. From the layman's perspective, I can't tell if Engineers have gotten lazy/complacent in their designs, OR if the FAA has gotten significantly better at screening for potential issues, OR if we're operating so close to the edge of the design envelope that these issues are inevitable. I sincerely hope it is number 2 or 3, but my gut feel is that cost cutting and efficiency are winning over safety and good engineering analysis.
- fzingle 5y ago
  The whole 737 Max debacle made two things clear. First, Boeing doesn't prioritize safety anymore. Profit is the driving factor in their decision making. You can read about the issues they have assembling the 787 in South Carolina. It is so bad some airlines are refusing to take delivery unless the plane is validated by the assembly line in Washington. Second, the FAA was caught being complacent with the 737 Max. It will take some time to fix that, but it is clear they don't want to make the same mistake. Also, the EASA is no longer just rubber stamping approvals following the FAA. Both agencies are combining to improve safety, which is highlighting the management problems at Boeing.
  - bobthebuilders 5y ago
    Just an FYI, it's North Charleston in South Carolina.
    - fzingle 5y ago
      Thanks -- fixed.
  - mjevans 5y ago
    Also worth noting, the South Carolina plant is NON-union, while the Washington (state) plants are unionized. I speculate that focus on safety and quality of work are easier at the unionized plants.
    - 5y ago
jacksonkmarley 5y ago
My (non-expert) interpretation of this situation: After 2 planes down, hundreds of deaths and a massive international backlash and shitstorm, Boeing tries to get a business-as-usual, rubber-stamp, paper-over-the-cracks, fastrack approval, and rightly gets shot down. > Is this just the FAA getting tough because of all the scrutiny? Hopefully this is just the FAA doing its job from now on.
- colechristensen 5y ago
  My interpretation of the situation: Business as usual being propped up as news because the topic gets attention recently. You test to find issues. You expect a certain sort and number of issues to come up in testing. Sometimes an issue comes up outside of expectations so you reset your expectations and test more. The FAA has been doing its job and flying is incredibly safe. There were lapses, those things are being addressed. >Boeing tries to get a business-as-usual, rubber-stamp, paper-over-the-cracks, fastrack approval This is way over the top. It's a fun narrative because building up an enemy is a good passtime and it gets reader attention. When you don't know anything about a topic, be hesitant to share your theories about how everybody is doing it wrong.
  - metalforever 5y ago
    No it's not. They've been doing Mickey Mouse engineering and are rightfully getting slapped the fuck down
  - omegalulw 5y ago
    Except that the testing from regulatory authorities is to verify you are doing what you are supposed to be doing. It's not QA.
  - jacksonkmarley 5y ago
    > When you don't know anything about a topic, be hesitant to share your theories about how everybody is doing it wrong. In the real world people who aren't experts have opinions, and those opinions matter. If the narrative that is everywhere is wrong feel free to correct it, that's a useful undertaking IMO. If you think everyone who knows less than you (in your opinion) should just be a silent spectator to your expertise, I guess we have different views on a how a healthy society behaves.
kerng 5y ago
I wonder how much longer it will take Boeing to realize the best way forward is to cancel this plane, and start from scratch. This project will just keep costing them money, continue eroding trust and probably more fatalities along the way...
- amelius 5y ago
  And the 737MAX too.
plank_time 5y ago
If it’s just pilots and peons in the plane during the test flights, it will only be about saving money. They need to put an executive in every one of the test flights until it is fully certified. I assure you it will be the safest plane you will ever fly.
- yourapostasy 5y ago
  > ...an executive... Even that wouldn't engender change. I could see change if it was selected from member of the board of directors, CEO, CFO, or COO, and random lower-level executive. And all their immediate families. That we would even be discussing this is odd to me. I remember watching various old-timey 1940-1950's era American movies that depicted turn-of-the-century titans of industry proudly and aggressively using what their factories' products, and if it involved putting their lives on the line, so much the better to get more attention, validation, and sales. Don't know whether that happened at all in real life, but it made an impression upon me that the zeitgeist thought it entirely appropriate to depict in popular media. Just as it makes an impression upon me that so few leaders of industry today will make it a point to truly randomly pick out their product and use it with gusto and pride. If they are caught on video doing any similar act, it is a thoroughly massaged "reality show" production carefully "handled" by PR, marketing and sales teams.
alkonaut 5y ago
If Airbus hadn’t made a detour with the A380, it would be almost comical how much better their product portfolio is now compared to Boeing. The A320neo vs the 737MAX and the A350 vs 777 and 787… Boeing needs a win, quickly.
- t0mas88 5y ago
  The A380 was a good example of getting the prediction of demand wrong. It looked good for the gulf carriers, but the US and European carriers moved to more direct flights due to consumer demand, for which the A380 is the wrong choice.
  - alkonaut 5y ago
    It appears to be an excellent plane, but it was the Concorde of its time. Or the F-35. Too large and complex to build and too few customers. The economic downturn and seismic shift in demand from hub and spoke as you point out was the nail in the coffin. I desperately need to go fly one of the “good ones” before they are all converted to 100% economy and then retired…
    - PicassoCTs 5y ago
      The biggest irony of the whole affair is, that the notorious delayed berlin airport was redesigned and rebuilt, to accommodate the A380 and when it was finally done, the plane was no longer relevant.
      - raverbashing 5y ago
        Yeah but the changes needed are not exclusive for the A380, but fit the 777 and 350 as well (I think. At least the 777 and of course the 747 which is more common) Most airports came before the A380, you'd change one/two gates at most.
        alkonaut 5y ago
        777X gets folding wingtips, is that to fit in gates without needing any such changes?
slumdev 5y ago
If it's Boeing, I ain't going! Hard to believe they wouldn't invest and ensure that their software is perfect after the MAX incidents, but here we are.
- nickik 5y ago
  Look at their space division. Constant software issues that totally embraced them on human space flight against SpaceX.
  - killingtime74 5y ago
    Truly the Volkswagen of aerospace
mrtweetyhack 5y ago
"it's nothing" -Boeing
neonate 5y ago
https://archive.is/osff9 https://archive.is/osff9
beprogrammed 5y ago
"Boeing has yet to satisfy the FAA that it has fully understood and corrected what went wrong that day." This appears to be the heart of the problem, how come it's the FAA holding you up on this and not your own engineering department? A solid foundation in design and engineering would require that if an event like this happened you'd rip down the whole structure until you got to the problem, from where you could confidently move forward knowing you know your systems.
404mm 5y ago
“abruptly” ha! Gotta love the news. Given the size of 777X, I doubt it was really abrupt. That thing is huge! On a more serious note, the message didn’t seem to clarify the “abruptness” of the pitch. That word did not need to be added.

168 points by donohoe 5y ago

> The FAA cited a long litany of concerns, including a serious flight control incident during a test flight on Dec. 8, 2020, when the plane experienced an “uncommanded pitch event” — meaning the nose of the aircraft pitched abruptly up or down without input from the pilots. Crikey. MCAS all over again. Something at Boeing is very, very out of control.

Aperocky 5y ago
What could go wrong if we just fire all the programmers that make too much and outsource the software?
- lawless_c 5y ago
  The programmers are just trying to patch what once again is likely a hardware problem. If it's like the MAX the real issue is the flight characteristics
  - _ph_ 5y ago
    Basically every modern jet aircraft with relatively large engines has some sort of MCAS system. If you raise the nose of such a plane the balance does shift. This isn't a form of an aerodynamic error, it is a consequence of having large engines hanging off your wings (a lot of classic airplanes have the engines in or very close to their central axis). If you want the appropriate feedback in the steering column, you have to compensate for the shift. Airbus does it as well.
    - amelius 5y ago
      What is the advantage of the engines further from the central axis?
      - smegcicle 5y ago
        iirc that's another term for larger, more efficient engines hanging lower from the same wings
      - dotancohen 5y ago
        Time, money. They don't need to redesign the wings to accommodate the newer, larger engines.
        coredog64 5y ago
        There's only so much room for the landing gear. Larger diameter engines require longer landing gear. Increasing the space given to landing gear requires changes to the wing box. Changes to the wing box is fundamentally a wing redesign.
        Gwypaas 5y ago
        Which is why the A320 is designed to be loaded with conveyors. This means it has no problem accommodating an even larger nacelle than the 737 in a regular position instead of the up and forward the 737 Max resorted to. Even the previous generation 737 was running into this issue and that's why they went with the distinct oval nacelles.
      - _ph_ 5y ago
        The main driver is the desire to make engines bigger, thus more fuel-efficient. The fuel savings of the newest engine generations are significant. If you hang engines from your wings (I assume that has practical reasons, it is easier to hang the engine than include it into the wing, it is also easier to maintain), larger engines mean engines further from the axis. That applies to all modern airplanes which share this configuration. A complete redesign of the 737 wouldn't have made this go away either. However the goal of keeping the same type rating made the especial design of MCAS necessary.
    - dundarious 5y ago
      MCAS-like systems exist elsewhere, but MCAS specifically is worthy of criticism, as both an engineering failure and more importantly IMO a management failure. Specifically with regard to the flight characteristics, MCAS is a band-aid on a problem created by cost-minimization -- the desire to avoid a hardware redesign (to properly accommodate the larger engines) and/or a re-certification.
      - _ph_ 5y ago
        Of course, the actual implementation of MCAS was a disaster. Especially, that they didn't deactivate it automatically on AOA disagree, that on top of that the screen signal for AOA disagree was not shown due to a separate issue and of course that late in development MCAS was given more control authority than initially planned. My point just was, a complete redesign of the 737 still would have to deal with the same aerodynamic issues as the 737 MAX, as they are rather fundamental to this kind of jets. But indeed, it would have been easier to deal with it with a new design. It is less mentioned, that shortly after the MAX crashes, Airbus mandated that in some planes the last rows were kept empty until some software updates were deployed. So they seem to have discovered some boundary conditions they were not happy with and decided to be rather safe till they augmented the software.
        FabHK 5y ago
        > late in development MCAS was given more control authority than initially planned ... without triggering re-evaluation of the seriousness/consequences of failure. That meant that failures of MCAS were considered unproblematic, which meant that lower levels of redundancy were deemed acceptable. Quite a process/management failure.
        _ph_ 5y ago
        Well, technically the failures of MCAS should be unproblematic, as the stabilizer runaway protocol is well trained for any 737 pilot and would cope with a MCAS misfunction. The problem is: in the two crashes the pilots either completely failed to follow the protocol (Lion Air) or to late and failed to recover (Ethiopian Airlines).
    - wyldfire 5y ago
      The bonkers thing seemed to me to be having the notification and/or override as an opt-in feature you could buy. Southwest didn't want to train pilots and newly visible behavior requires new training. No surprise that airlines elsewhere in the world also skimped on the "optional" features to the doom of the passengers and crew.
      - carl_dr 5y ago
        The opt-in feature you mention was for a disagree alert for the plane’s angle of attack sensors. But, MCAS was only ever using a single angle of attack sensor as its source of truth, so if there was a fault in that single sensor, MCAS could activate at normal attitudes. But the real problem was that Boeing hadn’t documented MCAS, to avoid the requirement for pilots to obtain type certification for the MAX (so any existing 737 pilot could to fly it without additional training - and it wasn’t just Southwest objecting to this training), and to avoid a full FAA certification process for the MAX. So on an MCAS activation, even if they had had an AoA alert, pilots would not know how to instinctively deal with it. (I don’t doubt an AOA disagree alert would give invaluable information allowing the pilots to rule out most failures, which when only a few thousand feet high would maybe have saved both planes.) On the previous day to the Lion 610 crash, the incident aircraft suffered an MCAS activation and fortunately a pilot in the jump seat realised what was happening and what was needed to deactivate it. It seems being sat behind the trim wheels (so he could see they were moving) and not wrestling to keep the plane in the sky at the same time was needed for that to happen. It must have been terrifying.
        coredog64 5y ago
        737 trim wheels are very visible and very loud. The noise and motion they create have been a joke within the 737 community for at least 20 years.
    - Gravityloss 5y ago
      Yes, it's the aerodynamic lift from the big engine fans that was the 737 Max issue, according to Aviation Week. Fans have gotten bigger as engines have improved and the bypass ratios and fuel efficiency has gotten better. Think of the engines as a big horizontal stabilizer but it's at the front, in front of the center of gravity. Normally it doesn't do much anything since the angle of attack is zero. But if you increase pitch, it will increase nose-up momentum. For some reasons it's not common to put the engines further back. DC-9 style, on the sides of the rear fuselage might work. Or maybe over the wing and slightly to the rear, like Hondajet.
    - himinlomax 5y ago
      The difference is that all Airbus planes since the 1980's A320 are built around computerized flight commands, so the code paths involved are exercised constantly. MCAS only triggers occasionally, meaning if it has a bug, it only shows up at the worst possible time. Airbus planes also have double redundancy (three inputs and computers) for all those systems, where MCAS had none by default.
  - landemva 5y ago
    Won quoted as, “The technical data required for type certification has not reached a point where it appears the aircraft type design is mature ..." The 737MAX should have gone through a new type certification, but didn't to allow for less pilot retraining. FAA failed to properly serve the public. New type cert finds flaws and saves lives.
  - WalterBright 5y ago
    > If it's like the MAX the real issue is the flight characteristics The real issue was the reliance on one sensor coupled with too much authority give to MCAS and inadequate failure analysis. And pilots who did not read, remember, or follow the 2 step recovery process in the Emergency Airworthiness Directive sent to all MAX pilots.
    - WalterBright 5y ago
      I always get a negative reaction for mentioning the Emergency Airworthiness Directive sent to all MAX pilots. Media accounts never mention it, either. The Boeing Emergency Airworthiness Directive says: "Initially, higher control forces may be needed to overcome any stabilizer nose down trim already applied. Electric stabilizer trim can be used to neutralize control column pitch forces before moving the STAB TRIM CUTOUT switches to CUTOUT. Manual stabilizer trim can be used before and after the STAB TRIM CUTOUT switches are moved to CUTOUT." https://theaircurrent.com/wp-content/uploads/2018/11/B737-MAX-AD-1107.pdf https://theaircurrent.com/wp-content/uploads/2018/11/B737-MA... For those who are skeptical of mainstream media fake news on the topic, here's the official final report on the Lion Air crash: 2018 - 035 - PK-LQP Final Report http://knkt.dephub.go.id/knkt/ntsc_aviation/baru/2018%20-%20035%20-%20PK-LQP%20Final%20Report.pdf http://knkt.dephub.go.id/knkt/ntsc_aviation/baru/2018%20-%20... I also recommend the recent Aviation Disasters episode on it. While shallow, because it has to fit in 40 minutes, I was shocked that it was surprisingly accurate. Kudos to the Smithsonian channel. I wouldn't be surprised if AD waited until that report was released before doing the episode, so it would be using verifiable facts.
      - DangitBobby 5y ago
        What are you trying to say, exactly?
        WalterBright 5y ago
        Not sure what you find confusing about it.
- dundarious 5y ago
  Not sure if you're being serious, and while it hasn't been tried AFAIK for the flight software specifically, but cost-minimization and outsourcing has been credited as the major sources of Boeing's safety problems for the last decade (737 MAX, 787 Dreamliner, 737NG). Doing more of it would be neither a change of course nor an improvement IMO.
  - eganist 5y ago
    > Not sure if you're being serious, and while it hasn't been tried AFAIK for the flight software specifically, but cost-minimization and outsourcing has been credited as the major sources of Boeing's safety problems for the last decade (737 MAX, 787 Dreamliner, 737NG). Doing more of it would be neither a change of course nor an improvement IMO. I hear you, but I suspect aperocky was criticizing that exact approach. I.e it felt like sarcasm.
    - dundarious 5y ago
      Makes sense, I was genuinely 50/50 about it being sarcastic, but I probably should have been more confident of it, and at the very least should have given more credence to that interpretation than "not sure if...".
      - Dylan16807 5y ago
        I'm honestly confused that you give it 50/50. Even ignoring the context, the "we" in "what could go wrong if we" only works if it's a rhetorical question. And from there, analyzing it as a rhetorical question, it seems obvious to me that the implied answer isn't "nothing". Do you disagree with either part of my reasoning?
        dundarious 5y ago
        On HN, especially if there's talk of labor (or god forbid, unions), there are often some pretty extreme opinions. And I said I shouldn't have given it just 50/50 odds. I don't see the point in continuing this postmortem.
- dotancohen 5y ago
  Actually, the responsible bit of software was already outsourced. There was an HN thread on that some time ago.
  - Aperocky 5y ago
    Yes this is what I'm referring to.
- _ph_ 5y ago
  This is really a crucial point. I am a software developer in an engineering company. What is essential for my job is not only my programming skills, but also a decent understanding of the engineering I deal with. This can only be achieved by working long enough in the job, be it an engineer who picks up programming or a software developer who learns the engineering involved. This is of course much more expensive than picking up any contractor, who might be brilliant at programming, but lacks the domain knowledge that only grows over time, but it is a good way of having more eyes spotting potential spec issues.
  - rbanffy 5y ago
    I really fail to understand the logic that may make someone to think a software engineer, however brilliant, can write software for a domain they don’t understand.
    - slipframe 5y ago
      Wishful thinking.
    - Aperocky 5y ago
      If this brilliance can be translated to understanding in other fields, which usually it can. This requires studying, which firms simply tries to avoid. I am a software engineer now, but my major was mechanical engineering, I can barely remember any of my coursework, but I understand how physics work in general, and a small bonus on how to disect a system. Nobody can know everything in today's world and that's why common engineering/sense training is so important IMO.
      - vidanay 5y ago
        Similar. I used to design automation equipment, but I've switched to software development. I'm still closely connected to the mechanical side though. More than once I've had to explain to someone who is a competent programmer that they can't accelerate mass x at rate y.
    - pjmlp 5y ago
- ricardobayes 5y ago
  What the cookie-cutter management doesn't understand is 100% of the company's value goes home after 5PM.
- throwaway713 5y ago
  I suddenly became afraid of flying when I started working for one of the companies known for having some of the best software engineers in the world and seeing how many bugs made it into prod. I can't imagine who is programming the flight computers for Boeing and what they're being paid. It's gotten bad enough that I refuse to fly on any plane that was released in the last decade. At least the older planes have some decent trip statistics to back up their safety.
- WalterBright 5y ago
  The MCAS problem had nothing whatsoever to do with outsourcing. The problem was in the specification of its behavior.
  - metalforever 5y ago
    This does appear to be related to bad engineering or engineers that do not understand how to program safety systems.
    - WalterBright 5y ago
      Of course, but the specifications are normally written by Boeing, and the implementation is outsourced. The reason is the system as a whole has to function, meaning the spec has to be written by the team that has knowledge of the whole system. MCAS interfaces with quite a few other systems on the airplane.
      - _ph_ 5y ago
        Even a good team can convince itself, that it is right. It would have added a "second set of eyes", if the implementors of MCAS had their own expertise in aviation. No guarantee, but a chance of spotting the problems with the spec.
- Guest42 5y ago
  It is my understanding that in aerospace much of the software is outsourced and is then QAed by internal contractors.
vmception 5y ago
My understanding is that they are trying to make planes that are too big to actually fly without continual software correction Can they just not do that?
- VBprogrammer 5y ago
  The 777 is fly by wire so it wouldn't be a good idea.
- deergomoo 5y ago
  My understanding was that they’re trying to use software to make their new planes fly like their old planes (at least that was the reasoning for MCAS in the 737 MAX). In super basic terms, if the new plane doesn’t fly like the old plane, all the pilots need to be retrained. This is a significant expense that could make a serious dent in sales. I don’t think it’s anything to do with size, there are larger planes than the 777X that are perfectly safe (and of course, the 737 is pretty small as far as airliners go).
  - LatteLazy 5y ago
    Isn't there also some weird requirement that pilots only fly one/two airframes at a time? So training for a 777 means your can't fly 747 or something else anymore?
    - sp332 5y ago
      I don't think so. But it was easier to sell to airlines if it didn't require retraining.
      - Dylan16807 5y ago
        It seems like there's a very strong incentive to avoid retraining coming from somewhere, at least. Surely that's not just airlines being stubborn?
        5y ago

xiphias2 5y ago

In a statement Friday, Boeing said it “remains fully focused on safety as our highest priority throughout 777X development.” Now finally that FAA doesn't want to repeat the MAX mistake, I believe Boeing.

xjwm 5y ago

More negative press for Boeing, which they probably don't need after all the 737 MAX issues. From the layman's perspective, I can't tell if Engineers have gotten lazy/complacent in their designs, OR if the FAA has gotten significantly better at screening for potential issues, OR if we're operating so close to the edge of the design envelope that these issues are inevitable. I sincerely hope it is number 2 or 3, but my gut feel is that cost cutting and efficiency are winning over safety and good engineering analysis.

fzingle 5y ago
The whole 737 Max debacle made two things clear. First, Boeing doesn't prioritize safety anymore. Profit is the driving factor in their decision making. You can read about the issues they have assembling the 787 in South Carolina. It is so bad some airlines are refusing to take delivery unless the plane is validated by the assembly line in Washington. Second, the FAA was caught being complacent with the 737 Max. It will take some time to fix that, but it is clear they don't want to make the same mistake. Also, the EASA is no longer just rubber stamping approvals following the FAA. Both agencies are combining to improve safety, which is highlighting the management problems at Boeing.
- bobthebuilders 5y ago
  Just an FYI, it's North Charleston in South Carolina.
  - fzingle 5y ago
    Thanks -- fixed.
- mjevans 5y ago
  Also worth noting, the South Carolina plant is NON-union, while the Washington (state) plants are unionized. I speculate that focus on safety and quality of work are easier at the unionized plants.
  - 5y ago

jacksonkmarley 5y ago

My (non-expert) interpretation of this situation: After 2 planes down, hundreds of deaths and a massive international backlash and shitstorm, Boeing tries to get a business-as-usual, rubber-stamp, paper-over-the-cracks, fastrack approval, and rightly gets shot down. > Is this just the FAA getting tough because of all the scrutiny? Hopefully this is just the FAA doing its job from now on.

colechristensen 5y ago
My interpretation of the situation: Business as usual being propped up as news because the topic gets attention recently. You test to find issues. You expect a certain sort and number of issues to come up in testing. Sometimes an issue comes up outside of expectations so you reset your expectations and test more. The FAA has been doing its job and flying is incredibly safe. There were lapses, those things are being addressed. >Boeing tries to get a business-as-usual, rubber-stamp, paper-over-the-cracks, fastrack approval This is way over the top. It's a fun narrative because building up an enemy is a good passtime and it gets reader attention. When you don't know anything about a topic, be hesitant to share your theories about how everybody is doing it wrong.
- metalforever 5y ago
  No it's not. They've been doing Mickey Mouse engineering and are rightfully getting slapped the fuck down
- omegalulw 5y ago
  Except that the testing from regulatory authorities is to verify you are doing what you are supposed to be doing. It's not QA.
- jacksonkmarley 5y ago
  > When you don't know anything about a topic, be hesitant to share your theories about how everybody is doing it wrong. In the real world people who aren't experts have opinions, and those opinions matter. If the narrative that is everywhere is wrong feel free to correct it, that's a useful undertaking IMO. If you think everyone who knows less than you (in your opinion) should just be a silent spectator to your expertise, I guess we have different views on a how a healthy society behaves.

kerng 5y ago

I wonder how much longer it will take Boeing to realize the best way forward is to cancel this plane, and start from scratch. This project will just keep costing them money, continue eroding trust and probably more fatalities along the way...

amelius 5y ago
And the 737MAX too.

plank_time 5y ago

If it’s just pilots and peons in the plane during the test flights, it will only be about saving money. They need to put an executive in every one of the test flights until it is fully certified. I assure you it will be the safest plane you will ever fly.

yourapostasy 5y ago
> ...an executive... Even that wouldn't engender change. I could see change if it was selected from member of the board of directors, CEO, CFO, or COO, and random lower-level executive. And all their immediate families. That we would even be discussing this is odd to me. I remember watching various old-timey 1940-1950's era American movies that depicted turn-of-the-century titans of industry proudly and aggressively using what their factories' products, and if it involved putting their lives on the line, so much the better to get more attention, validation, and sales. Don't know whether that happened at all in real life, but it made an impression upon me that the zeitgeist thought it entirely appropriate to depict in popular media. Just as it makes an impression upon me that so few leaders of industry today will make it a point to truly randomly pick out their product and use it with gusto and pride. If they are caught on video doing any similar act, it is a thoroughly massaged "reality show" production carefully "handled" by PR, marketing and sales teams.

alkonaut 5y ago

If Airbus hadn’t made a detour with the A380, it would be almost comical how much better their product portfolio is now compared to Boeing. The A320neo vs the 737MAX and the A350 vs 777 and 787… Boeing needs a win, quickly.

t0mas88 5y ago
The A380 was a good example of getting the prediction of demand wrong. It looked good for the gulf carriers, but the US and European carriers moved to more direct flights due to consumer demand, for which the A380 is the wrong choice.
- alkonaut 5y ago
  It appears to be an excellent plane, but it was the Concorde of its time. Or the F-35. Too large and complex to build and too few customers. The economic downturn and seismic shift in demand from hub and spoke as you point out was the nail in the coffin. I desperately need to go fly one of the “good ones” before they are all converted to 100% economy and then retired…
  - PicassoCTs 5y ago
    The biggest irony of the whole affair is, that the notorious delayed berlin airport was redesigned and rebuilt, to accommodate the A380 and when it was finally done, the plane was no longer relevant.
    - raverbashing 5y ago
      Yeah but the changes needed are not exclusive for the A380, but fit the 777 and 350 as well (I think. At least the 777 and of course the 747 which is more common) Most airports came before the A380, you'd change one/two gates at most.
      - alkonaut 5y ago
        777X gets folding wingtips, is that to fit in gates without needing any such changes?

slumdev 5y ago

If it's Boeing, I ain't going! Hard to believe they wouldn't invest and ensure that their software is perfect after the MAX incidents, but here we are.

nickik 5y ago
Look at their space division. Constant software issues that totally embraced them on human space flight against SpaceX.
- killingtime74 5y ago
  Truly the Volkswagen of aerospace

mrtweetyhack 5y ago

"it's nothing" -Boeing

neonate 5y ago

https://archive.is/osff9 https://archive.is/osff9

beprogrammed 5y ago

"Boeing has yet to satisfy the FAA that it has fully understood and corrected what went wrong that day." This appears to be the heart of the problem, how come it's the FAA holding you up on this and not your own engineering department? A solid foundation in design and engineering would require that if an event like this happened you'd rip down the whole structure until you got to the problem, from where you could confidently move forward knowing you know your systems.

404mm 5y ago

“abruptly” ha! Gotta love the news. Given the size of 777X, I doubt it was really abrupt. That thing is huge! On a more serious note, the message didn’t seem to clarify the “abruptness” of the pitch. That word did not need to be added.