17 ms·

Boeing identifies new software problem on grounded 737 Max

253 points by rafaelm 7y ago

onychomys 7y ago
In their defense, probably every piece of software of any complexity at all has bugs waiting to be found, and it's not super surprising that they found some new ones while doing a rigorous testing regimen.
- totalZero 7y ago
  Shouldn't that rigorous testing regimen have preceded the sale and flight of these large passenger aircraft?
  - onychomys 7y ago
    Sure. But I'd hope that they brought in new people, with a new outlook on things, and started all over from scratch. They're probably running mostly the same tests they were before the crashes, and also some new ones that people have dreamed up.
    - tremon 7y ago
      I think it's more likely that all the managers in the chain of control suddenly feel like their ass is on the line if another 737 MAX drops out of the sky, which is why they're now paying more attention to what their engineers have been telling them for years. Which is probably a good thing, except for the part that it took hundreds of people to die first to get here.
  - kayfox 7y ago
    A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Every other piece of software is tested to levels set by the aeronautical industry and regulators, which may not include all possible scenarios. You will note, some of these scenarios are essentially fuzzing the memory of the flight computer and seeing what happens, ideally most bit flips will either be detected or be minor, but some can end up causing issues. I'm not sure if it would be possible to explore all the branches in the software in any sort of reasonable amount of time. https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72
    - imglorp 7y ago
      That cost is already baked into aviation hardware. It's why fasteners are so expensive, for example, because the cost of certifying, tracking their history, and quantifying their quality is already amortized into each bolt. Flight software development lifecycle--and regs as you point out--still needs to catch up.
      - m4rtink 7y ago
        Actually, for fasteners there is often a material difference - cheaper fasteners are cut to shape while better fasteners are forged into shape, resulting in a much stronger part due to the resulting grain flow.
    - andbberger 7y ago
      What's this about the space shuttle? I'm intrigued. I thought that the paris metro was the posterchild of formal verification methods.
      - salawat 7y ago
        The software that went into the Space Shuttle was coded to CMM level 5; this is a level of Quality Assurance that would make even the most Iron Fisted, detail oriented QA begin to sweat at the pile of paperwork and pressure ahead of them. It's a completely different experience that likely only a handful of modern testers would even contemplate nowadays, but has always been a shining example of something to aspire to be part of one day for me. It isn't at all easy, and basically requires the discipline to understand, document, and justify every single line of code within the context of the overarching system. No business wants to deal with it, and it is no surprise that only publically funded projects tend to get anywhere close. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20110014946.pdf https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/201100... https://developers.slashdot.org/story/00/05/19/050258/space-shuttle-software-not-for-hacks https://developers.slashdot.org/story/00/05/19/050258/space-...
    - tgflynn 7y ago
      > A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Do you have a source for that claim ? I'm skeptical that even a testing process costing billions of dollars could claim to cover all possible inputs and states for a software system of any complexity.
- johannes1234321 7y ago
  As a plane crash costs lives of a few hundred people the criteria should be different from "normal" software. And for most parts and most history software in aeronautics was created with special care and tooling and under research of provability.
  - organsnyder 7y ago
    In the case of MCAS, that's likely true: the software was behaving exactly per spec; the problem was in the earlier requirements-definition phase.
    - salawat 7y ago
      A grievous failure in Systems Engineering and FMEA, I believe is the industry term.
- coldpie 7y ago
  In their defense, the software industry is a complete joke in terms of quality control. I hope that this and the spate of ransomware will wake the industry up to realize we need new processes, languages and tooling to make software provably correct. It's clear we can't use our existing languages and tooling to make high quality software. Realistically though, that's not going to happen. The smart move is to eliminate as much software from your life as possible. It's only going to get worse as decades of laziness catches up to us.
  - m_fayer 7y ago
    Watching Android descend from a place of dorky stability to sleek sealed glitch-city, kinda makes me think it's the institutions, as opposed to the tooling, that's behind the plunge in the quality of mainstream software.
    - peteradio 7y ago
      Feed me features I shit backlogs.
      - salawat 7y ago
        I couldn't help but laugh, because it is so damn true. It's taken me a while to come to the realization that all project management is is a backlog manufacturing and prioritization layer that operates on top of actual software engineers. Most just want to implement things, and don't care what it is they are implementing as long as they are getting paid.
    - shadowgovt 7y ago
      Was Android the stable one? I recall relative to iOS that it was the one likely to be eating its entire battery in the early iterations.
    - jsight 7y ago
      I can't agree. I've been using Android since the Nexus One days, and it is generally much more stable for me than some of those early devices.
- simion314 7y ago
  >and it's not super surprising that they found some new ones while doing a rigorous testing regimen. reply So they did not do enough testing before, just the happy case and never considered sensor failures. I am now wondering if they tested the rest of the software or FAA will just look into MCAS and ignore all the rest.
- anshou- 7y ago
  When some software is being built the constraints are much more rigorous, but it all comes down to the organization. The guidelines established by NASA for their space program are a good example of rigorous requirements. Boeing should not be unfamiliar with this concept and can certainly afford to do this properly. This move was motivated purely by greed.
- skissane 7y ago
  > it's not super surprising that they found some new ones while doing a rigorous testing regimen Why didn’t they find these bugs during the plane’s development and certification? It appears the testing then was less rigorous. They should extend the same rigorous testing to their other aircraft models also.
- Findeton 7y ago
  Actually there are two types of software failures: design failures and implementation failures. Design failures are harder, but we already have ways of completely avoiding implementation failures, with systems that prove mathematically that the software implements the design. NASA for example uses those in many occasions. Boeing doesn't, obviously, and there's no valid excuse.
- nradov 7y ago
  The Space Shuttle flight computer software was effectively bug free. Only a tiny number of defects were found post release, and none impacted safety.
vikramkr 7y ago
So not only are they trying to fix a fundamental hardware issue with a software patch, their inability to do software properly extends beyond just their MCAS system? This is a good reminder that air travel's extraordinary safety record isn't just a given, it's something that takes real work to achieve and when the people responsible for putting in that work (Boeing, regulators) begin taking safety for granted, that's when people die.
- totalZero 7y ago
  I agree. But we should keep in mind that there's no such thing as a bad apple; we can't blame individual executives or regulators. It's a bad barrel: a company that has, on a cultural level, put its business motive above its responsibility to deliver a safe and high-quality product. We have seen documented evidence that employees knew there were dangers and problems, and discussed these issues, but nobody cared enough to slow things down and get the product right.
  - vikramkr 7y ago
    Absolutely. That's why I said Boeing and the FAA failed their responsibility as opposed to a Boeing exec or a particular legislator - there are organizational, structural problems. Sure, some individuals made the decision to ignore reports or set a new culture, but the fact that they succeeded is concerning - why did everyone else enable them? Is there anything we could have done to encourage those engineers to whistleblow their concerns before the planes crashed? Would they have been taken seriously by the FAA or the media or investors who were pushing for growth at all costs? These are deep, structural problems.
    - salawat 7y ago
      Interesting you're getting downvotes. Not sure if the general audience or not. Regardless, the primary motivation to get a worker in a highly consolidated industry to blow the whistle is for them not to feel like they only have one choice. I can see where Boeing having consolidated as much as it has can give the culture extra resilience against employee disruption simply from the fact there isn't anywhere else to go.
kayfox 7y ago
I think the main thing we are seeing here is hundreds of smaller fixes that usually form the steady stream of Airworthiness Directives that an aircraft currently supported by the manufacturer sees turning into a news event every single time one comes out. So far only one "aircraft" has had perfect software, and that was the Space Shuttle, every single other aircraft out there has had software issues that are worked out over the life of the aircraft, just like every piece of software, even that which has very strict testing regimes, has had defects in it.
- blattimwind 7y ago
  > So far only one "aircraft" has had perfect software, and that was the Space Shuttle Actually it had 3+ known bugs.
  - rootbear 7y ago
    And shortly after the code for the Apollo Guidance Computer was put up on Github, someone found a bug!
  - kayfox 7y ago
    Drat, we have no known examples of perfect software.
    - radiorental 7y ago
      10 goto 10
      - glitchc 7y ago
        Great for those cold winter nights! Will probably be optimized out by a modern compiler though. Sad.
- 0xffff2 7y ago
  True enough for modern commercial airliners; I can't help but point out that plenty of aircraft have "perfect" (read: non-existent) software. Those aircraft too generally have issues that are worked out over their lifetime. Software seems to be especially error prone, but maybe that's just because the mechanical engineers have a head start of several hundred years.
sheeshkebab 7y ago
There are two kinds of software: - buggy - the one where bugs were not yet found
- opwieurposiu 7y ago
  - software with off-by-one errors
  - noonespecial 7y ago
    There's a fourth one but its undefined.
    - kibwen 7y ago
      There are actually ten, but the latter six have yet to be documented.
      - dev-ns8 7y ago
        OB1 errror, sorry. Actually 9, one slots null
- rosybox 7y ago
  my hello world has no bugs!
  - mirekrusin 7y ago
    Is it enterprise ready like this one https://gist.github.com/lolzballs/2152bc0f31ee0286b722 https://gist.github.com/lolzballs/2152bc0f31ee0286b722 ? Even that one is alpha quality, no soap support, no saml2.0, xml is not even mentioned once. It needs couple of major versions before it can reach production in express, standard, developer, enterprise and pro distributions.
  - IMTDb 7y ago
    But the kernel and/or the CPU you are running it on does.
Gusen 7y ago
Part II: Impeachment Is A Diversion & Delay - Blocking of the "impeachment" witnesses was collusion planned before the new year. Listen to an FBI agent disclosure from January 1, 2O2O here. President Trump to resign close to election securing it for Biden. See latest updates. Here is the zip file, also made available in the 3Jan2020 update. The file within is VID_20200101_201948.mp3. Turn up the volume and put on headphones. BB10Mp3Footage31Dec1Jan.zip 122.4mb https://drive.google.com/file/d/1IXOOhQhHybwky8Z5pGdr9ZXhWpIUazmi/view?usp=sharing https://drive.google.com/file/d/1IXOOhQhHybwky8Z5pGdr9ZXhWpI... The dialogue about the impeachment starts near the beginning. Having Biden in the White House is as good as Trump or anyone else in their organization. Obviously Schiff and Nadler pledged their allegiance to the organization by raping boys on the record, with their task being to drag out an impeachment designed to obstruct and delay any real efforts to remove the President, thus keeping Trump in power. The witness blocking was to cause an apparent uproar delaying things with legal actions until late Summer. Soon after, the President would resign, leaving any other candidate with not enough time or support to compete with an opportunistic Biden, who is as good as Trump or any other Illuminati friendly politician in the Presidency. 162 page PDF [last updated: Feb|4|2O2O]: https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W2qxVc/view?usp=sharing https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W... Previously reported: \Wag The Dog: first was feigned impeachment hearings meant to obstruct, now an attack on Iranians in Iraq. Here is what they are trying to distract from & cover up to retain power. $100+ billion in bribes to the highest offices in this country. 815+ deaths from child rapes to prove loyalty! See the latest PDF updates: FBI Director Wray, AG Barr, SoD Shanahan, & SoS Pompeo each raped boys and were paid billions in bribes for a Soros & Koch funded child rape org. So did Trump & his "impeachment" team Nadler,Schiff,Mueller.So did media moguls Redstone,Murdoch,Moonves. What are they trying to set up? Who can arrest them since they are all bribed and in on it ? Their strategy to stay in every office and obstruct until forced to leave no matter what. Feigning impeachment: see page 13O. rsvm kl rwe tw twer;l t;wer. \\if;Download the video/audio file, put on headphones and turn up the volume. You will hear these people committing these crimes. Audio was broadcast into my apartment by outdated surveillance equipment illegally embedded within my walls. This very same technology was being used to broadcast me to the internet for five years without my consent. I own this footage. Please use this to prosecute all found within. Note:: I am obliviously speaking throughout the video, and it can be quite loud at times relative to the desired content. The are dozens more links, including these, that can be found in this PDF last updated 4FEB 2O2O: https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W2qxVc/view?usp=sharing https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W... All members of the "Illuminati"; "....an underground organization of homosexuals and child rapists..." (from pg 26: Barack Obama with Jack Dorsey). President Donald Trump: Demands a $4 billion dollar bribe here at 10:18am 4thJan2019: 3JanCh3_900-1100.avi https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77YG0Vr/view?usp=sharing https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77... 3JanCh2_900-1100-avi https://drive.google.com/file/d/1LUmVygl_q0XVs8h2cWr8jZl-24fY0vBB/view?usp=sharing https://drive.google.com/file/d/1LUmVygl_q0XVs8h2cWr8jZl-24f... 3JanCh4_1000-1100.mp3 https://drive.google.com/file/d/1ZpP1pJbJakBgg-y-MWNozTxp3wJojHge/view?usp=sharing https://drive.google.com/file/d/1ZpP1pJbJakBgg-y-MWNozTxp3wJ... President Trump rapes and kills twelve boys, including five boys in a "who can rape five boys to death the fastest" game: 14JanCh3_600.mp3 https://drive.google.com/file/d/1ufPmglde9Mep0m6xYMJ9c4TWTjj258zB/view?usp=sharing https://drive.google.com/file/d/1ufPmglde9Mep0m6xYMJ9c4TWTjj... 14JanCh2_600-700.mp3 https://drive.google.com/file/d/136qLJdEn8eCs9tI4QtIxl4opW_Lw_gCF/view?usp=sharing https://drive.google.com/file/d/136qLJdEn8eCs9tI4QtIxl4opW_L... Speaker of the House Nancy Pelosi: Accepting a $3 billion dollar bribe at 1033 am on the 17th of Jan 2019 to ensure Asian boys can get through the border at "Monterrey" undocumented to be raped: 17JanCh3_949-1100.avi https://drive.google.com/file/d/1eodHu4o5Cm3xEWhDqipSuTj-M1CZUvmE/view?usp=sharing https://drive.google.com/file/d/1eodHu4o5Cm3xEWhDqipSuTj-M1C... 17JanCh4_1017-1100.avi https://drive.google.com/file/d/1y-nWEQbempkVZSz230j9wTyduZNjDmnk/view?usp=sharing https://drive.google.com/file/d/1y-nWEQbempkVZSz230j9wTyduZN... Speaker Nancy Pelosi also "preps" boys with First Lady Melania Trump, defined as in she performs oral sex on the boys’ penis and anus, as a child rapist like Henry Porter would, while trying to remove fecal matter from the boy prior to handing them over to be raped and then subsequently murdered, for Supreme Court Justice Samuel Alito, who apparently decides he would rather just have ten billion dollars instead. US Attorney for Western New York James Kennedy rapes these boys instead: 12JanCh3_1533-1638.mp3 https://drive.google.com/file/d/1AgFkDsbPbI4b5Xd3Wbz2EVNNx25kW2Oz/view?usp=sharing https://drive.google.com/file/d/1AgFkDsbPbI4b5Xd3Wbz2EVNNx25... Attorney General William Barr with FBI Deputy Director Christopher Wray raped and killed boys for billions in bribes in Buffalo, New York, USA on the 17Jan2019 at 7:50am: 18JanCh4_700mp3 https://drive.google.com/file/d/1UIdZkS5ZVksZdHYsnHk2t5losi0N0Bp7/view?usp=sharing https://drive.google.com/file/d/1UIdZkS5ZVksZdHYsnHk2t5losi0... 18JanCh2_700.mp3 https://drive.google.com/file/d/1DFK8IAxm5pQVqZv9L518nfgP7_odB2Um/view?usp=sharing https://drive.google.com/file/d/1DFK8IAxm5pQVqZv9L518nfgP7_o... 18JanCh3_725-.mp3 https://drive.google.com/file/d/1DG5ej59Ic8RT9UhbyMdwT0BDcKIifGGm/view?usp=sharing https://drive.google.com/file/d/1DG5ej59Ic8RT9UhbyMdwT0BDcKI... Secretary of State Michael Pompeo and Secretary of Defense Patrick Shanahan each raped and killed boys on 5thJan'19 at 17:39 for billions in bribes: jkswjnefvr cefe_0 5JanCh3_1600 1800.mp3 https://drive.google.com/file/d/1ghgmNrQvJ8WfJ2TsDVx1ruDU36hrhx6S/view?usp=sharing https://drive.google.com/file/d/1ghgmNrQvJ8WfJ2TsDVx1ruDU36h... 5JanCh2_1721-1818.mp3 https://drive.google.com/file/d/1eSlD4otX4KZqWXboQM92Mu-6J02Mjwrg/view?usp=sharing https://drive.google.com/file/d/1eSlD4otX4KZqWXboQM92Mu-6J02... Leaders of the "impeachment" effort Jerrold Nadler, Robert Mueller, and Adam Schiff all rape and kill boys between 11:20pm and 1:10am: 14JanCh4_2300-0000.mp3 Nadler starts at about twenty mins in- https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU66GuU/view?usp=sharing https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU... 14JanCh2_2304-2359.mp3 https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9WLr1x1/view?usp=sharing https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9W... Mueller at 12:25am, next is Schiff who starts 12:55-ish: 15JanCh2_000-100.mp3 https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62tgCkj/view?usp=sharing https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62... 15JanCh2_100-200.mp3 https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_rGSFH/view?usp=sharing https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_... 15JanCh4_000-100.mp3 cdklndvfr, copemve. https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2OQnwZ/view?usp=sharing https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2... 15JanCh4_100-200.mp3 https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw746LeCL/view?usp=sharing https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw74... Complete Media Protection: Lester Holt, of NBC NightlyNews, apparently a member of the Illuminati since the 1980's, along with ABC Nightly News lead anchor David Muir, stop over to the Porter studio in Buffalo, New York on 14Jan2019 at 5:00 am. They both rape and kill about two dozen boys by 6:00 am. Muir starts around 5.15am, then Holt about 5:38 am. Multi-billionaire Rupert Murdoch, owner of News Corp and also Fox Corporation, takes his turn after Holt. Video links below: 14JanCh3_500-601.avi https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX3o7UL/view?usp=sharing https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX... 14JanCh2_530-600.avi https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1RwidV/view?usp=sharing https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1... Adding to the media protection and reason this is not picked up by the media, CBS and Viacom owner Sumner Redstone and Leslie Moonves rape and kill boys following the President. 14JanCh3_700.avi egmbe;wmgmw, we;rmbgwe. https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXaB4aI0/view?usp=sharing https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXa... 14JanCh2_700-800.avi https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWYKfRDp/view?usp=sharing https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWY... 161p PDF [last updated: Feb|4|2O2O]: https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W2qxVc/view?usp=sharing https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W... \\. Please repost in USA! Post gets censored in US Recently more relevant: From page 49, Senator Mitch McConnell: At 1632 Senator Mitch McConnell checks into the Porter camera system inquiring if he can be part of the "eviction" for $10 million dollars. He is informed by group members that there are enough people for the event already and his participation is not necessary. At 1634 McConnell states "I fucked 15 kids, how am I not getting paid by you?" He is dismissed by Donald Reeves with "I think that will be all Mr. McConnell." 13JanCh3_1600-1700.avi https://drive.google.com/file/d/1L7bqOpvaEWmLiJpMhJNQDrfsQAHtsX8-/view?usp=sharing https://drive.google.com/file/d/1L7bqOpvaEWmLiJpMhJNQDrfsQAH...
platz 7y ago
> designed to warn of a malfunction by a system that helps raise and lower the plane’s nose So, they can't even name the mcas system anymore?
- slumdev 7y ago
  I thought that's what the MCAS was. Unless there's more than one system that overrides the pilot to pitch the nose down?
  - kayfox 7y ago
    Speed trim and the trim system in general.
swiley 7y ago
I really wonder about these large engineering corporations, Toyota seems to have similar problems with software. Part of me feels like many of these companies don’t keep code secret to protect IP, instead they do it because they know it’s a burning train wreck and don’t want people to find out.
- salawat 7y ago
  That's interesting. Toyota would be one of the last companies I'd expect to hear that about. They're notorious in Quality circles for taking Quality seriously; at least as far as their production line is concerned. Do they not apply that same philosophy to in house software?
  - thanatropism 7y ago
    The Toyota / Arthur Deming quality philosophy is really applicable to repeatable process where quality control means detecting abnormal variation amidst normal variation.
  - mark-r 7y ago
    The investigations carried out for unintended acceleration in Toyotas didn't paint a good picture. https://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code https://www.safetyresearch.net/blog/articles/toyota-unintend... https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_slides.pdf https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_...
    - salawat 7y ago
      Jesus... Thanks for posting that. That was a horrifying read. I'm at a loss for words. Looks like I've got some more reading to do...
    - s5ma6n 7y ago
      Damn... - "No configuration management" - "No bug tracking system" - "No formal specifications" - "9,273 – 11,528 global variables" - "Uses recursion, no mitigation for stack overflow. Memory just past stack is OSEK RTOS area" I thought of Toyota as a much better company in terms of safety and reliability. I can't imagine other manufacturers and their code.
stagas 7y ago
Can someone explain how a hugely complex machine with mostly parallel working analog parts fits into the digital computing paradigm? Isn't it predetermined to fail under extreme conditions, like those that are found while flying inbetween clouds and thunderstorms with all that pressure and fluctuations? How does sampling not fail, like, all the time? What kind of tooling is being used to mitigate for all these? Does anyone know?
- starpilot 7y ago
  Yes.
- salawat 7y ago
  https://en.wikipedia.org/wiki/Failure_mode_and_effects_analysis https://en.wikipedia.org/wiki/Failure_mode_and_effects_analy... https://en.wikipedia.org/wiki/Fault_tree_analysis https://en.wikipedia.org/wiki/Fault_tree_analysis Basically, you should be designing every system to gracefully handle the failure of every other system on which it is dependent. So the MCAS routines, if they had been done correctly, and properly classified as to the hazard level, should have taken into account failures of the Flight Computer they were running on, anomaly detection via cross-check with the second AoA vane, etc. That quite clearly did not happen. The same approach applies with any other hardware/software integration. Your sensors will break. You therefore need to determine what you need to do when that happens.
- garbage_88564 7y ago
  I am very naive to commercial aviation but this is my experience with building and crashing model aircraft repeatedly. I fly mostly FPV which puts me in the first person view from the cockpit. Yes, electronics fail in the most weirdest ways due to connector failures, RF interference, software error, sensor failure. When my systems start failing or acting up due to improper stabilization PID gains, etc. I have a big switch for MANUAL mode. I am able to fly this thing as long as the servos, radio, and camera get power. All sensors could be sheared off. I have no idea what my airspeed is ever because I don't use pitot tubes so I use a known engine throttle % whose stall characteristic I understand for level flight in various wind conditions and I don't make sudden maneuvers at throttle below this point. Fixed wing planes have remarkable aerodynamic stability and I don't understand why 737 MAX cannot be piloted in a fly by wire manner with all computer aids disabled, giving the pilots direct control of the servos with a big red switch that mechanically disconnects the flight computers. This requires almost no code to implement.
MDWolinski 7y ago
All software is buggy. The problem with the MCAS system is that pilots were not informed that it was there, nor were they given a way to override it and take full control of the airplane. Also, while the MCAS system relied on two sensors, if either failed, the MCAS system itself failed, so there was no built in back-up for it. Bugs in software happen because situations where they arise are sometimes hard to predict. You can test your software all you want but it's not until it's in the field that you start discovering new issues because people tend to do things in ways developers didn't consider. Tesla's software has over a billion miles of data on it and it still has issues in some basic functionality. And let's not talk about Iowa which in itself was a major failure in software release management.
- linuxftw 7y ago
  > the MCAS system itself failed The MCAS system didn't fail. It operated exactly as it was designed, it received faulty sensor data which in turn flew the plane into the ground. > Bugs in software happen because situations where they arise are sometimes hard to predict. Everything that cause MCAS to drive the plane into the ground was 100% predictable and testable. There are two basic problems. The first, is that the people that thought MCAS was a good idea in the first place are incompetent. The second problem is the people that designed and implemented the MCAS system were even more incompetent. Idiots managing idiots, and if I was the CEO of that company, that system would have never been allowed in the first place.
  - muldvarp 7y ago
    > The MCAS system didn't fail. It operated exactly as it was designed, it received faulty sensor data which in turn flew the plane into the ground. You're technically right, but I'd say that incorrect sensor data is something that should be taken into account. And failing to take that into account is a failure on part of the MCAS.
    - kayfox 7y ago
      It did take it into account, but the incorrect angle reporting was not severe enough to result in the sensor being marked as faulty. From the sound of things it was designed to deal with large bit flips, sensor jamming the the home position or sensor misalignment (which is at fixed angles because of the mounting holes) rather than the sensor jamming in other positions or being horribly miscalibrated. The company that repaired the sensor used in the Lion Air Flight did loose their license to repair any and all certified avionics parts and is probably out of business at this point. Donno about the technician that forged the post-install tests of that sensor.
clSTophEjUdRanu 7y ago
As a former software defense worker, I wish there were 3rd party audits of code and dev ops. If you saw the code that's flying in missiles, aircraft, etc and how they got there youd want to go live in a cave.
- jacquesm 7y ago
  Some whistleblower should one day post an archive of Airbus or Boeing's software archives. That would make for interesting reading.
  - Glawen 7y ago
    It's usually worthless without knowing what is attached to the input/output of the microcontroller. A lot of things are done ecternally on the wiring.
- kayfox 7y ago
  They have missiles for those caves. ;)
  - bradknowles 7y ago
    It doesn't have to be a missile. Any fuel-air explosive will do.
- pnako 7y ago
  A bug in a plane can make it crash in a fireball. But a bug in a missile is something that would make it NOT crash in a fireball. Thus the obvious solution to quality problems is to switch missile software engineers and aircraft software engineers, and encourage them not to care about quality.
  - monocasa 7y ago
    Or make it crash into a fireball literally anywhere except where it was supposed to.
- 7y ago
bsimpson 7y ago
I'm shocked they keep publicly working on this plane. I get that planes cost more money than I can fathom, and that making a whole fleet of impossible amounts of money costs a gazillion dollars. Still, this one seems spent. Nobody is going to knowingly fly on a 737 Max. They ought to have retired the plane last year. They can design a new plane (that because of economics, will probably be very similar to this plane), release it when it's been properly vetted, swap out Maxes for it, retrofit those Maxes, etc. I realize this is naive armchair quarterbacking from someone who has never worked in aviation, but there's a reason that Philip Morris is called Altria now and that Weinstein Co was merged into Spyglass. If the public doesn't trust your brand, no amount of "but we fixed it with this patch we rushed out the door" is going to change that.
- JumpCrisscross 7y ago
  > They ought to have retired the plane last year This would bankrupt the company. (Not necessarily uncalled for. But not something it will do on its own.) > If the public doesn't trust your brand The public has shown one preference, above all else, when it comes to flying: pricing. The 737 MAX will be renamed, re-certified and nobody but an obsessive minority will avoid flying on it. (This is why, btw, we need strong airline regulators. Market pressure is ceteris paribus insufficient.)
  - TeMPOraL 7y ago
    > This would bankrupt the company. Is that even possible? From what I understand, it's a strategic company for the United States, so they have to keep it alive no matter what (I recall reading that it's actually a law) - or at least its military branch.
    - rsynnott 7y ago
      It can certainly be kept alive through bankruptcy. Boeing isn't going anywhere. The board's share value might, tho.
    - nokcha 7y ago
djsumdog 7y ago
I hope these planes get scrapped and never see flight again. Tear them down and recycle the parts, and build something that's modern from the group up instead of recycled, deprecated bullshit.
- V_Terranova_Jr 7y ago
  This just isn't going to happen.
notadoc 7y ago
Throw in the towel on the 737 Max and go back to the drawing board.
benwerd 7y ago
There is a less than zero chance I'll be boarding one of these planes again, ever. Trust is an important idea in any product, but particularly in areas like aviation, and I don't see how they can possibly build it back. I do see an opportunity for software that ensures you are only booking journeys on the aircraft you feel are safe.
- apexalpha 7y ago
  on the other hand, the software for this plane might be the most scrutinized ever when all this is over.
  - oska 7y ago
    The hardware remains fundamentally flawed.
- oska 7y ago
  I agree with you. This plane is fundamentally flawed and the reason such a flawed plane went into production was to reward short-term, sociopathic thinking and earn short-term profits (which ended up blowing up in their faces anyway). Taking a flight on this plane now would, for me, be like going back to an abusive partner after they'd thrown me down a flight of stairs.
bunchOfCucks 7y ago
As long as I can stream TV in flight, who cares.
frandroid 7y ago
> Asked about a likely date for a return to service for the Max, Dickson said it isn’t helpful to talk about timelines. Boeing needs to concentrate on making complete, quality submissions on its fixes for the plane, he said. Ahh, "we'll ship it when it's ready, not on some arbitrary deadline." Music to any engineer/builder's ears.

253 points by rafaelm 7y ago

In their defense, probably every piece of software of any complexity at all has bugs waiting to be found, and it's not super surprising that they found some new ones while doing a rigorous testing regimen.

totalZero 7y ago
Shouldn't that rigorous testing regimen have preceded the sale and flight of these large passenger aircraft?
- onychomys 7y ago
  Sure. But I'd hope that they brought in new people, with a new outlook on things, and started all over from scratch. They're probably running mostly the same tests they were before the crashes, and also some new ones that people have dreamed up.
  - tremon 7y ago
    I think it's more likely that all the managers in the chain of control suddenly feel like their ass is on the line if another 737 MAX drops out of the sky, which is why they're now paying more attention to what their engineers have been telling them for years. Which is probably a good thing, except for the part that it took hundreds of people to die first to get here.
- kayfox 7y ago
  A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Every other piece of software is tested to levels set by the aeronautical industry and regulators, which may not include all possible scenarios. You will note, some of these scenarios are essentially fuzzing the memory of the flight computer and seeing what happens, ideally most bit flips will either be detected or be minor, but some can end up causing issues. I'm not sure if it would be possible to explore all the branches in the software in any sort of reasonable amount of time. https://en.wikipedia.org/wiki/Qantas_Flight_72 https://en.wikipedia.org/wiki/Qantas_Flight_72
  - imglorp 7y ago
    That cost is already baked into aviation hardware. It's why fasteners are so expensive, for example, because the cost of certifying, tracking their history, and quantifying their quality is already amortized into each bolt. Flight software development lifecycle--and regs as you point out--still needs to catch up.
    - m4rtink 7y ago
      Actually, for fasteners there is often a material difference - cheaper fasteners are cut to shape while better fasteners are forged into shape, resulting in a much stronger part due to the resulting grain flow.
  - andbberger 7y ago
    What's this about the space shuttle? I'm intrigued. I thought that the paris metro was the posterchild of formal verification methods.
    - salawat 7y ago
      The software that went into the Space Shuttle was coded to CMM level 5; this is a level of Quality Assurance that would make even the most Iron Fisted, detail oriented QA begin to sweat at the pile of paperwork and pressure ahead of them. It's a completely different experience that likely only a handful of modern testers would even contemplate nowadays, but has always been a shining example of something to aspire to be part of one day for me. It isn't at all easy, and basically requires the discipline to understand, document, and justify every single line of code within the context of the overarching system. No business wants to deal with it, and it is no surprise that only publically funded projects tend to get anywhere close. https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20110014946.pdf https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/201100... https://developers.slashdot.org/story/00/05/19/050258/space-shuttle-software-not-for-hacks https://developers.slashdot.org/story/00/05/19/050258/space-...
  - tgflynn 7y ago
    > A regime which finds all the defects would cost billions on its own, only one such piece of software has been subject to such a regime and that's the Space Shuttle guidance program. Do you have a source for that claim ? I'm skeptical that even a testing process costing billions of dollars could claim to cover all possible inputs and states for a software system of any complexity.
johannes1234321 7y ago
As a plane crash costs lives of a few hundred people the criteria should be different from "normal" software. And for most parts and most history software in aeronautics was created with special care and tooling and under research of provability.
- organsnyder 7y ago
  In the case of MCAS, that's likely true: the software was behaving exactly per spec; the problem was in the earlier requirements-definition phase.
  - salawat 7y ago
    A grievous failure in Systems Engineering and FMEA, I believe is the industry term.
coldpie 7y ago
In their defense, the software industry is a complete joke in terms of quality control. I hope that this and the spate of ransomware will wake the industry up to realize we need new processes, languages and tooling to make software provably correct. It's clear we can't use our existing languages and tooling to make high quality software. Realistically though, that's not going to happen. The smart move is to eliminate as much software from your life as possible. It's only going to get worse as decades of laziness catches up to us.
- m_fayer 7y ago
  Watching Android descend from a place of dorky stability to sleek sealed glitch-city, kinda makes me think it's the institutions, as opposed to the tooling, that's behind the plunge in the quality of mainstream software.
  - peteradio 7y ago
    Feed me features I shit backlogs.
    - salawat 7y ago
      I couldn't help but laugh, because it is so damn true. It's taken me a while to come to the realization that all project management is is a backlog manufacturing and prioritization layer that operates on top of actual software engineers. Most just want to implement things, and don't care what it is they are implementing as long as they are getting paid.
  - shadowgovt 7y ago
    Was Android the stable one? I recall relative to iOS that it was the one likely to be eating its entire battery in the early iterations.
  - jsight 7y ago
    I can't agree. I've been using Android since the Nexus One days, and it is generally much more stable for me than some of those early devices.
simion314 7y ago
>and it's not super surprising that they found some new ones while doing a rigorous testing regimen. reply So they did not do enough testing before, just the happy case and never considered sensor failures. I am now wondering if they tested the rest of the software or FAA will just look into MCAS and ignore all the rest.
anshou- 7y ago
When some software is being built the constraints are much more rigorous, but it all comes down to the organization. The guidelines established by NASA for their space program are a good example of rigorous requirements. Boeing should not be unfamiliar with this concept and can certainly afford to do this properly. This move was motivated purely by greed.
skissane 7y ago
> it's not super surprising that they found some new ones while doing a rigorous testing regimen Why didn’t they find these bugs during the plane’s development and certification? It appears the testing then was less rigorous. They should extend the same rigorous testing to their other aircraft models also.
Findeton 7y ago
Actually there are two types of software failures: design failures and implementation failures. Design failures are harder, but we already have ways of completely avoiding implementation failures, with systems that prove mathematically that the software implements the design. NASA for example uses those in many occasions. Boeing doesn't, obviously, and there's no valid excuse.
nradov 7y ago
The Space Shuttle flight computer software was effectively bug free. Only a tiny number of defects were found post release, and none impacted safety.

vikramkr 7y ago

So not only are they trying to fix a fundamental hardware issue with a software patch, their inability to do software properly extends beyond just their MCAS system? This is a good reminder that air travel's extraordinary safety record isn't just a given, it's something that takes real work to achieve and when the people responsible for putting in that work (Boeing, regulators) begin taking safety for granted, that's when people die.

totalZero 7y ago
I agree. But we should keep in mind that there's no such thing as a bad apple; we can't blame individual executives or regulators. It's a bad barrel: a company that has, on a cultural level, put its business motive above its responsibility to deliver a safe and high-quality product. We have seen documented evidence that employees knew there were dangers and problems, and discussed these issues, but nobody cared enough to slow things down and get the product right.
- vikramkr 7y ago
  Absolutely. That's why I said Boeing and the FAA failed their responsibility as opposed to a Boeing exec or a particular legislator - there are organizational, structural problems. Sure, some individuals made the decision to ignore reports or set a new culture, but the fact that they succeeded is concerning - why did everyone else enable them? Is there anything we could have done to encourage those engineers to whistleblow their concerns before the planes crashed? Would they have been taken seriously by the FAA or the media or investors who were pushing for growth at all costs? These are deep, structural problems.
  - salawat 7y ago
    Interesting you're getting downvotes. Not sure if the general audience or not. Regardless, the primary motivation to get a worker in a highly consolidated industry to blow the whistle is for them not to feel like they only have one choice. I can see where Boeing having consolidated as much as it has can give the culture extra resilience against employee disruption simply from the fact there isn't anywhere else to go.

kayfox 7y ago

I think the main thing we are seeing here is hundreds of smaller fixes that usually form the steady stream of Airworthiness Directives that an aircraft currently supported by the manufacturer sees turning into a news event every single time one comes out. So far only one "aircraft" has had perfect software, and that was the Space Shuttle, every single other aircraft out there has had software issues that are worked out over the life of the aircraft, just like every piece of software, even that which has very strict testing regimes, has had defects in it.

blattimwind 7y ago
> So far only one "aircraft" has had perfect software, and that was the Space Shuttle Actually it had 3+ known bugs.
- rootbear 7y ago
  And shortly after the code for the Apollo Guidance Computer was put up on Github, someone found a bug!
- kayfox 7y ago
  Drat, we have no known examples of perfect software.
  - radiorental 7y ago
    10 goto 10
    - glitchc 7y ago
      Great for those cold winter nights! Will probably be optimized out by a modern compiler though. Sad.
0xffff2 7y ago
True enough for modern commercial airliners; I can't help but point out that plenty of aircraft have "perfect" (read: non-existent) software. Those aircraft too generally have issues that are worked out over their lifetime. Software seems to be especially error prone, but maybe that's just because the mechanical engineers have a head start of several hundred years.

sheeshkebab 7y ago

There are two kinds of software: - buggy - the one where bugs were not yet found

opwieurposiu 7y ago
- software with off-by-one errors
- noonespecial 7y ago
  There's a fourth one but its undefined.
  - kibwen 7y ago
    There are actually ten, but the latter six have yet to be documented.
    - dev-ns8 7y ago
      OB1 errror, sorry. Actually 9, one slots null
rosybox 7y ago
my hello world has no bugs!
- mirekrusin 7y ago
  Is it enterprise ready like this one https://gist.github.com/lolzballs/2152bc0f31ee0286b722 https://gist.github.com/lolzballs/2152bc0f31ee0286b722 ? Even that one is alpha quality, no soap support, no saml2.0, xml is not even mentioned once. It needs couple of major versions before it can reach production in express, standard, developer, enterprise and pro distributions.
- IMTDb 7y ago
  But the kernel and/or the CPU you are running it on does.

Gusen 7y ago

Part II: Impeachment Is A Diversion & Delay - Blocking of the "impeachment" witnesses was collusion planned before the new year. Listen to an FBI agent disclosure from January 1, 2O2O here. President Trump to resign close to election securing it for Biden. See latest updates. Here is the zip file, also made available in the 3Jan2020 update. The file within is VID_20200101_201948.mp3. Turn up the volume and put on headphones. BB10Mp3Footage31Dec1Jan.zip 122.4mb https://drive.google.com/file/d/1IXOOhQhHybwky8Z5pGdr9ZXhWpIUazmi/view?usp=sharing https://drive.google.com/file/d/1IXOOhQhHybwky8Z5pGdr9ZXhWpI... The dialogue about the impeachment starts near the beginning. Having Biden in the White House is as good as Trump or anyone else in their organization. Obviously Schiff and Nadler pledged their allegiance to the organization by raping boys on the record, with their task being to drag out an impeachment designed to obstruct and delay any real efforts to remove the President, thus keeping Trump in power. The witness blocking was to cause an apparent uproar delaying things with legal actions until late Summer. Soon after, the President would resign, leaving any other candidate with not enough time or support to compete with an opportunistic Biden, who is as good as Trump or any other Illuminati friendly politician in the Presidency. 162 page PDF [last updated: Feb|4|2O2O]: https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W2qxVc/view?usp=sharing https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W... Previously reported: \Wag The Dog: first was feigned impeachment hearings meant to obstruct, now an attack on Iranians in Iraq. Here is what they are trying to distract from & cover up to retain power. $100+ billion in bribes to the highest offices in this country. 815+ deaths from child rapes to prove loyalty! See the latest PDF updates: FBI Director Wray, AG Barr, SoD Shanahan, & SoS Pompeo each raped boys and were paid billions in bribes for a Soros & Koch funded child rape org. So did Trump & his "impeachment" team Nadler,Schiff,Mueller.So did media moguls Redstone,Murdoch,Moonves. What are they trying to set up? Who can arrest them since they are all bribed and in on it ? Their strategy to stay in every office and obstruct until forced to leave no matter what. Feigning impeachment: see page 13O. rsvm kl rwe tw twer;l t;wer. \\if;Download the video/audio file, put on headphones and turn up the volume. You will hear these people committing these crimes. Audio was broadcast into my apartment by outdated surveillance equipment illegally embedded within my walls. This very same technology was being used to broadcast me to the internet for five years without my consent. I own this footage. Please use this to prosecute all found within. Note:: I am obliviously speaking throughout the video, and it can be quite loud at times relative to the desired content. The are dozens more links, including these, that can be found in this PDF last updated 4FEB 2O2O: https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W2qxVc/view?usp=sharing https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W... All members of the "Illuminati"; "....an underground organization of homosexuals and child rapists..." (from pg 26: Barack Obama with Jack Dorsey). President Donald Trump: Demands a $4 billion dollar bribe here at 10:18am 4thJan2019: 3JanCh3_900-1100.avi https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77YG0Vr/view?usp=sharing https://drive.google.com/file/d/1Grdr8xF2psKNsuYlEnl9dIRV-77... 3JanCh2_900-1100-avi https://drive.google.com/file/d/1LUmVygl_q0XVs8h2cWr8jZl-24fY0vBB/view?usp=sharing https://drive.google.com/file/d/1LUmVygl_q0XVs8h2cWr8jZl-24f... 3JanCh4_1000-1100.mp3 https://drive.google.com/file/d/1ZpP1pJbJakBgg-y-MWNozTxp3wJojHge/view?usp=sharing https://drive.google.com/file/d/1ZpP1pJbJakBgg-y-MWNozTxp3wJ... President Trump rapes and kills twelve boys, including five boys in a "who can rape five boys to death the fastest" game: 14JanCh3_600.mp3 https://drive.google.com/file/d/1ufPmglde9Mep0m6xYMJ9c4TWTjj258zB/view?usp=sharing https://drive.google.com/file/d/1ufPmglde9Mep0m6xYMJ9c4TWTjj... 14JanCh2_600-700.mp3 https://drive.google.com/file/d/136qLJdEn8eCs9tI4QtIxl4opW_Lw_gCF/view?usp=sharing https://drive.google.com/file/d/136qLJdEn8eCs9tI4QtIxl4opW_L... Speaker of the House Nancy Pelosi: Accepting a $3 billion dollar bribe at 1033 am on the 17th of Jan 2019 to ensure Asian boys can get through the border at "Monterrey" undocumented to be raped: 17JanCh3_949-1100.avi https://drive.google.com/file/d/1eodHu4o5Cm3xEWhDqipSuTj-M1CZUvmE/view?usp=sharing https://drive.google.com/file/d/1eodHu4o5Cm3xEWhDqipSuTj-M1C... 17JanCh4_1017-1100.avi https://drive.google.com/file/d/1y-nWEQbempkVZSz230j9wTyduZNjDmnk/view?usp=sharing https://drive.google.com/file/d/1y-nWEQbempkVZSz230j9wTyduZN... Speaker Nancy Pelosi also "preps" boys with First Lady Melania Trump, defined as in she performs oral sex on the boys’ penis and anus, as a child rapist like Henry Porter would, while trying to remove fecal matter from the boy prior to handing them over to be raped and then subsequently murdered, for Supreme Court Justice Samuel Alito, who apparently decides he would rather just have ten billion dollars instead. US Attorney for Western New York James Kennedy rapes these boys instead: 12JanCh3_1533-1638.mp3 https://drive.google.com/file/d/1AgFkDsbPbI4b5Xd3Wbz2EVNNx25kW2Oz/view?usp=sharing https://drive.google.com/file/d/1AgFkDsbPbI4b5Xd3Wbz2EVNNx25... Attorney General William Barr with FBI Deputy Director Christopher Wray raped and killed boys for billions in bribes in Buffalo, New York, USA on the 17Jan2019 at 7:50am: 18JanCh4_700mp3 https://drive.google.com/file/d/1UIdZkS5ZVksZdHYsnHk2t5losi0N0Bp7/view?usp=sharing https://drive.google.com/file/d/1UIdZkS5ZVksZdHYsnHk2t5losi0... 18JanCh2_700.mp3 https://drive.google.com/file/d/1DFK8IAxm5pQVqZv9L518nfgP7_odB2Um/view?usp=sharing https://drive.google.com/file/d/1DFK8IAxm5pQVqZv9L518nfgP7_o... 18JanCh3_725-.mp3 https://drive.google.com/file/d/1DG5ej59Ic8RT9UhbyMdwT0BDcKIifGGm/view?usp=sharing https://drive.google.com/file/d/1DG5ej59Ic8RT9UhbyMdwT0BDcKI... Secretary of State Michael Pompeo and Secretary of Defense Patrick Shanahan each raped and killed boys on 5thJan'19 at 17:39 for billions in bribes: jkswjnefvr cefe_0 5JanCh3_1600 1800.mp3 https://drive.google.com/file/d/1ghgmNrQvJ8WfJ2TsDVx1ruDU36hrhx6S/view?usp=sharing https://drive.google.com/file/d/1ghgmNrQvJ8WfJ2TsDVx1ruDU36h... 5JanCh2_1721-1818.mp3 https://drive.google.com/file/d/1eSlD4otX4KZqWXboQM92Mu-6J02Mjwrg/view?usp=sharing https://drive.google.com/file/d/1eSlD4otX4KZqWXboQM92Mu-6J02... Leaders of the "impeachment" effort Jerrold Nadler, Robert Mueller, and Adam Schiff all rape and kill boys between 11:20pm and 1:10am: 14JanCh4_2300-0000.mp3 Nadler starts at about twenty mins in- https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU66GuU/view?usp=sharing https://drive.google.com/file/d/1Kuvv2Zmbw5Jw7onbRI2hCZ0M8FU... 14JanCh2_2304-2359.mp3 https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9WLr1x1/view?usp=sharing https://drive.google.com/file/d/1nofp5xF-aXXcCSgQVwj30KlzE9W... Mueller at 12:25am, next is Schiff who starts 12:55-ish: 15JanCh2_000-100.mp3 https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62tgCkj/view?usp=sharing https://drive.google.com/file/d/1EsmHfguwBuo2PbavJ1WYyhiML62... 15JanCh2_100-200.mp3 https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_rGSFH/view?usp=sharing https://drive.google.com/file/d/1NZnWRnBryalNQu2yJmfJUdS2pA_... 15JanCh4_000-100.mp3 cdklndvfr, copemve. https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2OQnwZ/view?usp=sharing https://drive.google.com/file/d/1ZEDJR6jb6ARpcNnWJTokBUKb2J2... 15JanCh4_100-200.mp3 https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw746LeCL/view?usp=sharing https://drive.google.com/file/d/173aYWvWHH4VGht1h_2nM0IMdw74... Complete Media Protection: Lester Holt, of NBC NightlyNews, apparently a member of the Illuminati since the 1980's, along with ABC Nightly News lead anchor David Muir, stop over to the Porter studio in Buffalo, New York on 14Jan2019 at 5:00 am. They both rape and kill about two dozen boys by 6:00 am. Muir starts around 5.15am, then Holt about 5:38 am. Multi-billionaire Rupert Murdoch, owner of News Corp and also Fox Corporation, takes his turn after Holt. Video links below: 14JanCh3_500-601.avi https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX3o7UL/view?usp=sharing https://drive.google.com/file/d/1i7NKepeyG_FfdQRrM7KsnFOZOOX... 14JanCh2_530-600.avi https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1RwidV/view?usp=sharing https://drive.google.com/file/d/1NZzgN5ilI7ToroU5cfqMaL4o2u1... Adding to the media protection and reason this is not picked up by the media, CBS and Viacom owner Sumner Redstone and Leslie Moonves rape and kill boys following the President. 14JanCh3_700.avi egmbe;wmgmw, we;rmbgwe. https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXaB4aI0/view?usp=sharing https://drive.google.com/file/d/10XDw6x3ldnnQiq7oIjpdYVENyXa... 14JanCh2_700-800.avi https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWYKfRDp/view?usp=sharing https://drive.google.com/file/d/1NS_e6AzEZ05wnfljkGMETGU5CWY... 161p PDF [last updated: Feb|4|2O2O]: https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W2qxVc/view?usp=sharing https://drive.google.com/file/d/1S7T_kDv48E40eHzus6CTXHxcm0W... \\. Please repost in USA! Post gets censored in US Recently more relevant: From page 49, Senator Mitch McConnell: At 1632 Senator Mitch McConnell checks into the Porter camera system inquiring if he can be part of the "eviction" for $10 million dollars. He is informed by group members that there are enough people for the event already and his participation is not necessary. At 1634 McConnell states "I fucked 15 kids, how am I not getting paid by you?" He is dismissed by Donald Reeves with "I think that will be all Mr. McConnell." 13JanCh3_1600-1700.avi https://drive.google.com/file/d/1L7bqOpvaEWmLiJpMhJNQDrfsQAHtsX8-/view?usp=sharing https://drive.google.com/file/d/1L7bqOpvaEWmLiJpMhJNQDrfsQAH...

platz 7y ago

> designed to warn of a malfunction by a system that helps raise and lower the plane’s nose So, they can't even name the mcas system anymore?

slumdev 7y ago
I thought that's what the MCAS was. Unless there's more than one system that overrides the pilot to pitch the nose down?
- kayfox 7y ago
  Speed trim and the trim system in general.

swiley 7y ago

I really wonder about these large engineering corporations, Toyota seems to have similar problems with software. Part of me feels like many of these companies don’t keep code secret to protect IP, instead they do it because they know it’s a burning train wreck and don’t want people to find out.

salawat 7y ago
That's interesting. Toyota would be one of the last companies I'd expect to hear that about. They're notorious in Quality circles for taking Quality seriously; at least as far as their production line is concerned. Do they not apply that same philosophy to in house software?
- thanatropism 7y ago
  The Toyota / Arthur Deming quality philosophy is really applicable to repeatable process where quality control means detecting abnormal variation amidst normal variation.
- mark-r 7y ago
  The investigations carried out for unintended acceleration in Toyotas didn't paint a good picture. https://www.safetyresearch.net/blog/articles/toyota-unintended-acceleration-and-big-bowl-%E2%80%9Cspaghetti%E2%80%9D-code https://www.safetyresearch.net/blog/articles/toyota-unintend... https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_slides.pdf https://users.ece.cmu.edu/~koopman/pubs/koopman14_toyota_ua_...
  - salawat 7y ago
    Jesus... Thanks for posting that. That was a horrifying read. I'm at a loss for words. Looks like I've got some more reading to do...
  - s5ma6n 7y ago
    Damn... - "No configuration management" - "No bug tracking system" - "No formal specifications" - "9,273 – 11,528 global variables" - "Uses recursion, no mitigation for stack overflow. Memory just past stack is OSEK RTOS area" I thought of Toyota as a much better company in terms of safety and reliability. I can't imagine other manufacturers and their code.

stagas 7y ago

Can someone explain how a hugely complex machine with mostly parallel working analog parts fits into the digital computing paradigm? Isn't it predetermined to fail under extreme conditions, like those that are found while flying inbetween clouds and thunderstorms with all that pressure and fluctuations? How does sampling not fail, like, all the time? What kind of tooling is being used to mitigate for all these? Does anyone know?

starpilot 7y ago
Yes.
salawat 7y ago
https://en.wikipedia.org/wiki/Failure_mode_and_effects_analysis https://en.wikipedia.org/wiki/Failure_mode_and_effects_analy... https://en.wikipedia.org/wiki/Fault_tree_analysis https://en.wikipedia.org/wiki/Fault_tree_analysis Basically, you should be designing every system to gracefully handle the failure of every other system on which it is dependent. So the MCAS routines, if they had been done correctly, and properly classified as to the hazard level, should have taken into account failures of the Flight Computer they were running on, anomaly detection via cross-check with the second AoA vane, etc. That quite clearly did not happen. The same approach applies with any other hardware/software integration. Your sensors will break. You therefore need to determine what you need to do when that happens.
garbage_88564 7y ago
I am very naive to commercial aviation but this is my experience with building and crashing model aircraft repeatedly. I fly mostly FPV which puts me in the first person view from the cockpit. Yes, electronics fail in the most weirdest ways due to connector failures, RF interference, software error, sensor failure. When my systems start failing or acting up due to improper stabilization PID gains, etc. I have a big switch for MANUAL mode. I am able to fly this thing as long as the servos, radio, and camera get power. All sensors could be sheared off. I have no idea what my airspeed is ever because I don't use pitot tubes so I use a known engine throttle % whose stall characteristic I understand for level flight in various wind conditions and I don't make sudden maneuvers at throttle below this point. Fixed wing planes have remarkable aerodynamic stability and I don't understand why 737 MAX cannot be piloted in a fly by wire manner with all computer aids disabled, giving the pilots direct control of the servos with a big red switch that mechanically disconnects the flight computers. This requires almost no code to implement.

MDWolinski 7y ago

All software is buggy. The problem with the MCAS system is that pilots were not informed that it was there, nor were they given a way to override it and take full control of the airplane. Also, while the MCAS system relied on two sensors, if either failed, the MCAS system itself failed, so there was no built in back-up for it. Bugs in software happen because situations where they arise are sometimes hard to predict. You can test your software all you want but it's not until it's in the field that you start discovering new issues because people tend to do things in ways developers didn't consider. Tesla's software has over a billion miles of data on it and it still has issues in some basic functionality. And let's not talk about Iowa which in itself was a major failure in software release management.

linuxftw 7y ago
> the MCAS system itself failed The MCAS system didn't fail. It operated exactly as it was designed, it received faulty sensor data which in turn flew the plane into the ground. > Bugs in software happen because situations where they arise are sometimes hard to predict. Everything that cause MCAS to drive the plane into the ground was 100% predictable and testable. There are two basic problems. The first, is that the people that thought MCAS was a good idea in the first place are incompetent. The second problem is the people that designed and implemented the MCAS system were even more incompetent. Idiots managing idiots, and if I was the CEO of that company, that system would have never been allowed in the first place.
- muldvarp 7y ago
  > The MCAS system didn't fail. It operated exactly as it was designed, it received faulty sensor data which in turn flew the plane into the ground. You're technically right, but I'd say that incorrect sensor data is something that should be taken into account. And failing to take that into account is a failure on part of the MCAS.
  - kayfox 7y ago
    It did take it into account, but the incorrect angle reporting was not severe enough to result in the sensor being marked as faulty. From the sound of things it was designed to deal with large bit flips, sensor jamming the the home position or sensor misalignment (which is at fixed angles because of the mounting holes) rather than the sensor jamming in other positions or being horribly miscalibrated. The company that repaired the sensor used in the Lion Air Flight did loose their license to repair any and all certified avionics parts and is probably out of business at this point. Donno about the technician that forged the post-install tests of that sensor.

clSTophEjUdRanu 7y ago

As a former software defense worker, I wish there were 3rd party audits of code and dev ops. If you saw the code that's flying in missiles, aircraft, etc and how they got there youd want to go live in a cave.

jacquesm 7y ago
Some whistleblower should one day post an archive of Airbus or Boeing's software archives. That would make for interesting reading.
- Glawen 7y ago
  It's usually worthless without knowing what is attached to the input/output of the microcontroller. A lot of things are done ecternally on the wiring.
kayfox 7y ago
They have missiles for those caves. ;)
- bradknowles 7y ago
  It doesn't have to be a missile. Any fuel-air explosive will do.
pnako 7y ago
A bug in a plane can make it crash in a fireball. But a bug in a missile is something that would make it NOT crash in a fireball. Thus the obvious solution to quality problems is to switch missile software engineers and aircraft software engineers, and encourage them not to care about quality.
- monocasa 7y ago
  Or make it crash into a fireball literally anywhere except where it was supposed to.
7y ago

bsimpson 7y ago

I'm shocked they keep publicly working on this plane. I get that planes cost more money than I can fathom, and that making a whole fleet of impossible amounts of money costs a gazillion dollars. Still, this one seems spent. Nobody is going to knowingly fly on a 737 Max. They ought to have retired the plane last year. They can design a new plane (that because of economics, will probably be very similar to this plane), release it when it's been properly vetted, swap out Maxes for it, retrofit those Maxes, etc. I realize this is naive armchair quarterbacking from someone who has never worked in aviation, but there's a reason that Philip Morris is called Altria now and that Weinstein Co was merged into Spyglass. If the public doesn't trust your brand, no amount of "but we fixed it with this patch we rushed out the door" is going to change that.

JumpCrisscross 7y ago
> They ought to have retired the plane last year This would bankrupt the company. (Not necessarily uncalled for. But not something it will do on its own.) > If the public doesn't trust your brand The public has shown one preference, above all else, when it comes to flying: pricing. The 737 MAX will be renamed, re-certified and nobody but an obsessive minority will avoid flying on it. (This is why, btw, we need strong airline regulators. Market pressure is ceteris paribus insufficient.)
- TeMPOraL 7y ago
  > This would bankrupt the company. Is that even possible? From what I understand, it's a strategic company for the United States, so they have to keep it alive no matter what (I recall reading that it's actually a law) - or at least its military branch.
  - rsynnott 7y ago
    It can certainly be kept alive through bankruptcy. Boeing isn't going anywhere. The board's share value might, tho.
  - nokcha 7y ago

djsumdog 7y ago

I hope these planes get scrapped and never see flight again. Tear them down and recycle the parts, and build something that's modern from the group up instead of recycled, deprecated bullshit.

V_Terranova_Jr 7y ago
This just isn't going to happen.

notadoc 7y ago

Throw in the towel on the 737 Max and go back to the drawing board.

benwerd 7y ago

There is a less than zero chance I'll be boarding one of these planes again, ever. Trust is an important idea in any product, but particularly in areas like aviation, and I don't see how they can possibly build it back. I do see an opportunity for software that ensures you are only booking journeys on the aircraft you feel are safe.

apexalpha 7y ago
on the other hand, the software for this plane might be the most scrutinized ever when all this is over.
- oska 7y ago
  The hardware remains fundamentally flawed.
oska 7y ago
I agree with you. This plane is fundamentally flawed and the reason such a flawed plane went into production was to reward short-term, sociopathic thinking and earn short-term profits (which ended up blowing up in their faces anyway). Taking a flight on this plane now would, for me, be like going back to an abusive partner after they'd thrown me down a flight of stairs.

bunchOfCucks 7y ago

As long as I can stream TV in flight, who cares.

frandroid 7y ago

> Asked about a likely date for a return to service for the Max, Dickson said it isn’t helpful to talk about timelines. Boeing needs to concentrate on making complete, quality submissions on its fixes for the plane, he said. Ahh, "we'll ship it when it's ready, not on some arbitrary deadline." Music to any engineer/builder's ears.