8 ms·

The can of worms has been officially opened, and this may turn out to be a case study in how bad software and outsourcing your centers of excellence can bankrup

• dhimes 7y ago
  I hope they aren't expecting people to fly on these planes.
• zaroth 7y ago
  As a tangent, there have been times when HN is ahead of the curve in understanding where something is headed before the investing public. The benchmarks and excitement around AMDs offerings are a great example. When you see something that presents a paradigm shift, often the market will not have fully digested it. AMD is one of those cases, and I believe BA is as well. Their stock is essentially unchanged from where it was before the Lion Air crash in October 2018. This company is worth $200B and we’re talking about 5,000 units of a $100mm plane (that’s $500B in revenue right? I’m not adding a zero there?)
  • Xixi 7y ago
    $100mm is the list price. Actual prices are decided by the market. I've read about $50mm for the 737-8, the baseline 737MAX model. That was of course before the MCAS disaster. My layman understanding is that actual prices would be around 40% off list price for models that sell really well, with little competition (for instance A321). Much steeper discount above 60% for models that have a hard time selling (A330NEO) or for end of line to close a production gap (777-300ER is very cheap to buy while Boeing is working on the 777-9).
• xvf22 7y ago
  A split brain in a plane is a bit of a nightmare. I'm not sure how they can work around it with just two computers.
  • imglorp 7y ago
    Big "disagree" alarm for the pilots and let them decide to keep either computer or go direct. Airbus has the concept of flight control laws, surfaced and changeable by the wet backups. http://www.airbusdriver.net/airbus_fltlaws.htm http://www.airbusdriver.net/airbus_fltlaws.htm
    • linuxftw 7y ago
      Which computer calculates that there's a disagreement, and how can we ensure it's not indicating a false negative?
      • imglorp 7y ago
        Either. The alarm must be an OR.
        • linuxftw 7y ago
          Then, IMO, it should be two independent indicator lights, not 1.
    • TomVDB 7y ago
      And Boeing will make that disagree alarm an optional extra?
      • cmurf 7y ago
        Am optical extra that isn't actually properly functioning as intended in numerous delivered products? Almost as if it wasn't sanity tested in one production unit, let alone some random sample of units?
• patcon 7y ago
  Hey mods, pls be attentive with the stats on voting patterns for this kinda thing Not claiming intent on part of OP, but a perfect storm of ego-stroking and groupthink could be possible, and this is exactly what foreign misinformation campaigns (with anti-USA interests) would be negligent not to notice the potential of EDIT: To those voting me down, this is how the world we built can work now, right? I'm not saying this is 100% what's happening this time. But let's stay on our toes, no?
• chkaloon 7y ago
  That, or this article missed the boat somehow. I find it hard to believe that a total redesign is happening and that they expect to be done by the end of the year. Either this article is wrong, or something is terribly wrong at Boeing.
  • dboreham 7y ago
    I'm a huge MAX skeptic, but this article just has to be wrong. Someone who had dropped out of a CS degree would know not to do what the article says they're doing.
    • SQueeeeeL 7y ago
      You would be surprised, if managament is broken to it's core, the situation is "Do this unethical thing or we'll find someone who will!" Computer Scientists aren't usually taught engineering ethics and definitely get fired all the time for arbitrary reasons.
      • uxp100 7y ago
        Is that true that CS students don't take an ethics course? I have a 10 year old SE degree and I took 3, only 1 of which was engineering specific. (I took a philosophy ethics course, which really isn't that relevant to issues like these anyway, but also a course that was specific to ethical issues in computing.) OTOH, I'm skeptical those courses change behavior, and in this particular case others in comments are saying this is a common architecture in the space, so how could anyone solve this ethical conundrum if getting to the base level of whether this is an ethical conundrum to begin with is difficult?
        • SQueeeeeL 7y ago
          In my opinion courses wouldn't change behavior, modern computer science is designed to infantalize coders and make them interchangeable. Think of all the effort startups put into marketing themselves as "COOL TO WORK FOR"/"WE HAVE BEER ON FRIDAYS". These reinforce the idea that you are small and need management to tell you exactly what to do everyday, because you're a dumb idiot who spends all day watching anime and playing video games. In that environment, serious ethical concerns could easily be brushed aside as "coders just being coders". When Google got found out for making drones, the employees of a company staged a 5 minute walkout.... that was it, I'm sure some people quit, but not enough to make any impact
• Merrill 7y ago
  >Landing contracts for key systems on the Max cemented a relationship with Boeing that Collins had long cultivated, in part by creating what its chief executive in 2002 called an “enormous sea change in a very long legacy business." For years, the company had been “a technology leader, an innovator,” Clayton Jones, then the chief executive, told Fortune magazine. “Unfortunately, along the way they forgot to hone their financial skills.” In 1998, Boeing executives summoned Mr. Jones to Seattle, he later recalled in a speech, and made clear that, to get more of Boeing’s business, Collins would have to cut prices dramatically. In response, Collins introduced what it called “lean electronics,” its take on a belt-tightening philosophy popularized by Toyota. >Collins reorganized business units and retrained managers, with an eye toward efficiency and speed. It pushed its suppliers to do the same, and established partnerships with companies such as HCL Technologies, which provides outsourced, lower-cost engineering services from India. >Along the way, Collins unseated Honeywell as the provider of flight control computers on a predecessor of the Max, the 737 NG, and supplied numerous systems for the Boeing 787, which went into operation in 2011. >When Collins secured the contract for the Max displays in 2012, it credited the belt-tightening. “There were a lot of cost-saving measures — a lot of tough decisions — that had to be made,” one manager said at the time in a company publication. https://www.nytimes.com/2019/11/07/business/boeing-737-max-collins.html https://www.nytimes.com/2019/11/07/business/boeing-737-max-c... If there was a problem with the master-slave operation on the MAX, not necessarily connected with MCAS, might not the same problem exist on the 737 NGs?
  • dk- 7y ago
    not likely the NG didn't add new systems. not saying other cost-cutting won't be exposed but nothing as egregious as MCAS. the MAX added new engines, which required forward placement on the wings, which changed COG, which necessitated MCAS.
    • Merrill 7y ago
      They changed flight control computers from Honeywell to Collins when going from the 737 to the 737 NG. Seems like a new system.
• zbrozek 7y ago
  "Dual-dual" is a thing in this space. There are safety-intent microprocessors which have a couple of cores running a couple of cycles out of sync with a delay and comparison on registers. If they disagree after that delay, the processor concludes it's insane and you can take action based on that hardware-level signal. It's not my favorite solution for every application, but it's a good tool for a handful of situations. And it's a good way to avoid having to think about picking who to trust with only two sockets. There are other approaches, of course. For example, if your system can afford more than a couple clock cycles of latency (e.g., if you don't believe you can crash a plane with a couple milliseconds of control error) you can do the same thing with well-designed software and a bit of extra hardware. It gives you more freedom since there's more hardware to buy, at the expense of engineering and test effort to prove that your solution does what you want. Doing this over large physical distances also adds some challenges. There are debates around dedicated signal wires direct between control centers vs data networks. Every situation is a little different, so there aren't really easy answers that you can swoop in and prescribe as best-practices. In general I find that people are more-skeptical of packet-switched solutions (e.g., Ethernet) than I would be. But most folks are also better at reasoning about the failure modes there than they are at thinking about the lower-level issues that Ethernet solves for you. And of course there are tons of different signaling systems - not just Ethernet - to consider. Consider just a few different possible applications and the constraints that they impose. A car has less severe failure modes (many crashes are nonfatal) but less freedom in time. Your trajectory is simply more cluttered. A passenger plane has dramatically more energy, so failures are typically more severe, but you have a lot more time to deal with them because your trajectory is empty (much of the time, anyway). And if you're sane, you've designed it to be passively aerodynamically stable because twitchy maneuverability isn't critical when you're not a fighter jet. A rocket is a different animal altogether. Those are often rigged with self-destruct explosives, and you don't really get much time to change your mind if you erroneously choose to trigger them. Plus they're typically less stable (performance is more dear, gravity isn't helping you, and purely aerodynamic solutions don't take you outside the atmosphere) and are thus more dependent on their control systems.
  • MFLoon 7y ago
    I'm confused as to what Boeing is changing then - your description of 'dual-dual' systems sounds like what is happening with the master/slave failover system already in place. It sounds like what they're changing is to have the two systems cross-check each other in real time, rather than have the in-band delayed core be the sanity check for each system, which seems like a big difference because it means that there are now disagreement scenarios whereas an in-band sanity check always is just checking one system against one set of invariants and thus can't disagree with it's own observations. Or am I misunderstanding?
• drawkbox 7y ago
  Classic case of hitting the date but crunched out with technical debt and buggy software, a tell that management/finance/marketing is driving the decision over the engineer/designer/product people. People forget a project was late if it succeeds, they never forget a project on time that failed because it was rushed, buggy and not polished production. MBA/business and marketing need to go back to letting the product people and engineers create value, then they can extract the value and sell it. It is much easier to sell value of a good product, it basically sells itself. For some reason American business has completely forgotten the rule this millennia, product then marketing and selling. You see this same thing happen at game companies when they put a date out there and it is driven by management and marketing over reality of engineering/product/design that makes a fun and solid game. Boeing was greatly diminished in engineering/product excellence by the McDonnell Douglas management control takeover, everything newly produced since has the tell of this.
  • nikanj 7y ago
    > People forget a project was late if it succeeds, they never forget a project on time that failed because it was rushed, buggy and not polished production. #notallMBAs, but the surest way for your product to fail on the market is for your company to go bankrupt before you’ve polished it enough. In this specific case it was about greed, but sometimes MBAs push for release, because according to the engineering teams the product is never truly perfect
    • kelnos 7y ago
      "Perfect" is of course unattainable, but I'd at least hope that engineering could be able to push for -- and be granted -- the time to build a system that meets what should be some pretty obvious safety requirements around redundancy and failure detection.
    • drawkbox 7y ago
      A crunch type culture and pushing can also create the issues seen in these problem projects. In fact I personally believe this is why more software and products are bad today, the product professionals are being overridden by marketing/finance and exact dates to meet some marketing goal not a good product goal. Product people and engineers know how to manage time probably better than business because they are doing the work. If something fails because it doesn't meet the market there are numerous reasons for that besides not hitting the date. A good product, that hits the date, can still fail due to market timing or market movement and many other things. Whether a project is launched at a certain date has little bearing on the success of that project long term, short term it might meet marketing and financial quarterly goals, but long term success is always better when it is a good product, and for it to be a good product the people that make products and create value should determine or at least have some say when it is complete and ready.
• heisenbit 7y ago
  If what we read is true this is then the second time that Boeing is trying to fix hardware shortcoming with software in a hair raising manner.
• tus88 7y ago
  > bad software The software was fine (or at least not to blame). Unstable aeronautic design and lack of redundant sensors was the real problem.
  • carlmr 7y ago
    Looking at the current article the software is also low quality.
• nobodyandproud 7y ago
  tl;dr : The Kruger-Dunning effect is most visible in the upper-management circles, where politics and charismatic talents are necessary skills. Former engineering powerhouses like HP, Boeing, and IBM slowly get hollowed-out into incompetence when it’s assumed that Engineering and technical skills don’t require a considerable depth of knowledge and experience.
  • bdavis__ 7y ago
    profound! could it also be the normal generational rotation of people through the organization? Everyone grew at a tremendous rate since the 1960's. Many, many more engineers. And then they retired.
    • nobodyandproud 7y ago
      I don’t think it’s generational. It’s just unlearned lessons of history. The most effective leadership for a technical company is leadership with actual technical creds to back it up. The technical side is a must because you need someone who actually understands the business and the innovations that must takes place. But you also need someone who can provide the people-leadership skills and the business wheeling&dealing. These are all crucial skills and take up a persons time, so it’s not uncommon to see this come in the form of a duo (Apple and Steves, or Microsoft with Bill+Steve).
• deleted 7y ago
  [deleted]
• beamatronic 7y ago
  I didn’t feel this way at first, but now I feel like I will never fly on this plane or allow my family members to fly on this plane.
• speg 7y ago
  What is TFA?