7 ms·

> Swampy believed relying on mechanics to self-inspect their work was not only insane but illegal This sounds like the changes that have taken place in the sof

by proc0 3y ago

> Swampy believed relying on mechanics to self-inspect their work was not only insane but illegal This sounds like the changes that have taken place in the software industry in the past 10-20 years. Engineers are meant to do much more than engineering, including testing their own software, managing project timelines, etc., however with software nobody dies, you just get crappy software that constantly breaks and needs an update every other day. There's an overall theme here of underestimating how hard engineering is, and as a result expecting a lot more from engineers which then of course causes bad engineering. Not surprisingly this is caused by non-technical people with power. Perhaps the fix is a cultural shift and a renewed respect for people who want to spend all their time specializing in technical skills.

necheffa 3y ago
> however with software nobody dies Well...no actually. This may be the case for _some_ software. I happen to work on (software) products where public safety is a concern.
- seb1204 3y ago
  Well in the story is the point of an software error leading to the plane crash right...
margalabargala 3y ago
> however with software nobody dies The MCAS issue which crashed two Boeing planes was a software hack.
- SahAssar 3y ago
  Well, yes, but also no. It was a hardware design change that necessitated a software hack (to escape mandatory retraining of pilots) that relied on unreliable hardware, right? Sure, software played a big part in it, but I think it seems like it was more a management and communication failure. If it was just software it'd probably be much easier to diagnose and fix.
  - margalabargala 3y ago
    I disagree. The hardware design of the cockpit is intended to be such that any computer inputs also move the pilot's controls, so that the pilots can countermand computer inputs if necessary. In this case, software was written such that this was not possible. The software that operates MCAS operates on a garbage-in-garbage-out model, like most software. There was no software written to determine if the incoming data was garbage, thus the software decided to crash two planes. Here's an article that goes into detail on the software: https://spectrum.ieee.org/how-the-boeing-737-max-disaster-looks-to-a-software-developer https://spectrum.ieee.org/how-the-boeing-737-max-disaster-lo... > When the flight computer trims the airplane to descend, because the MCAS system thinks it’s about to stall, a set of motors and jacks push the pilot’s control columns forward. It turns out that the Elevator Feel Computer can put a lot of force into that column—indeed, so much force that a human pilot can quickly become exhausted trying to pull the column back, trying to tell the computer that this really, really should not be happening. > Indeed, not letting the pilot regain control by pulling back on the column was an explicit design decision. Because if the pilots could pull up the nose when MCAS said it should go down, why have MCAS at all? > MCAS is implemented in the flight management computer, even at times when the autopilot is turned off, when the pilots think they are flying the plane. In a fight between the flight management computer and human pilots over who is in charge, the computer will bite humans until they give up and (literally) die. At the end of the day, if the person who wrote that software had written it differently, then those planes would not have crashed and hundreds of people would not have died.
    - error503 3y ago
      > At the end of the day, if the person who wrote that software had written it differently, then those planes would not have crashed and hundreds of people would not have died. You can't really blame the software engineers. This was all thought out and tightly specified by Boeing to their avionics subcontractor (Collins, IIRC). This is how it was designed and engineered to work at a systems level - it is a design hack. As far as I know there weren't any software bugs or 'hacks' involved, and the avionics operated as designed (aside from the AoA DISAGREE alert, which was due to a requirements miscommunication, not a bug). It was broken by design, which happened long before implementation, at Boeing. > When the flight computer trims the airplane to descend, because the MCAS system thinks it’s about to stall, a set of motors and jacks push the pilot’s control columns forward. It turns out that the Elevator Feel Computer can put a lot of force into that column—indeed, so much force that a human pilot can quickly become exhausted trying to pull the column back, trying to tell the computer that this really, really should not be happening. The Elevator Feel Computer is a part of the 737NG as well, and would behave the same way in those airplanes when receiving such erroneous AoA data; it's nothing new in the MAX. It certainly did not help the crews during the fatal MAX incidents, and is clearly not an ideal design, but it's also barely a footnote in the root cause analysis, along with the stick shaker and stall warnings blaring at them constantly. The pilots would easily be able to overcome it long enough to get safely on the ground. What was a bigger problem for those crews was that the MCAS has enough trim authority to make it impossible, with any amount of elevator input, to restore level flight - limiting its trim authority was part of the 'fixes' required to get them airborne again. I don't think it's reasonable to blame the implementation of MCAS for the accidents, its existence is to blame, and really highlights how nothing about the 737 platform has been designed holistically - it is a patchwork of hacks on hacks dating from the 1970s, which is difficult to reason about as a whole, and has dark corners. To truly 'fix' MCAS, you need to consider AoA as critical air data (which the 737 does not), and you need to integrate it holistically with the rest of the flight controls (which the 737 cannot, since it is not fly-by-wire), and you need to consider it critical equipment (it's an 'augmentation' and not considered critical on the 737, 'justifying' the lack of redundancy). Once you've done those things, you've basically got the bones of a proper envelope protection system in place, and you've obviated the need for MCAS in the first place. Of course the 737 team couldn't do this, because the business decided that it was more important to avoid (and hide) any differences than to bring the aircraft in line with modern standards. Realistically, this should have been trapped by the safety analysis of the flawed design, which should have considered its effect on the whole flight control system when evaluating it, but Boeing again only considered MCAS to be an 'augmentation' and it got an abbreviated safety review as a result. Some engineers did express concern about some of these factors, but given the environment outlined in TFA, those concerns did not go anywhere, because they would have basically scuttled the idea and sent everyone back to the drawing board, which Boeing was desperate to avoid having already been caught flat-footed with the launch of the A320neo. The 737 airframe needs to be put to rest, it is simply not safe or sane to keep stacking more hacks onto it. But there's no indication Boeing's working on a successor so it's probably going to be on the market for another 20+ years. Hard to imagine folks will probably be flying on an airframe with a 100 year old design (2024 + 20 years before a new revision + 20 years life span = 2064, around 100 years from the 737 launch before they start retiring)!
- proc0 3y ago
  I meant software only products like websites, apps, games, etc., but it should be clarified, good point.
- idontpost 3y ago
  [dead]
rjbwork 3y ago
Great idea but how will the bean counters and schmoozers get their multi-million dollar bonuses if they can't force engineers to do 5 jobs while paying them for 1? Will never work.
- pfortuny 3y ago
  quality control is most of the times unmeasurable in the short term…
Terr_ 3y ago
The difference is whether organizations are willing--or forced--to invest in it. That's not automatically a wrong though, since different objects or processes merit different levels of quality.
mattgreenrocks 3y ago
The real fix is popularizing the notion that management is just as commoditizatable than those they seek to commoditize. Note that in the recent dialogue about AI eating jobs, there's zero mention of whether it could come for management positions. Nothing. Curious, isn't it? Why wouldn't an LLM be good enough at this? I mean, it's really data-driven, right?
- passwordoops 3y ago
  Frankly I think ChatGPT 3.5 was already good enough to replace the majority of CxO positions
  - lrem 3y ago
    Why not a Markov chain generator?
    - paradox460 3y ago
      Or an 8 ball
- joe_the_user 3y ago
  The real fix is popularizing the notion that management is just as commoditizatable than those they seek to commoditize. What's described here is exactly what happens when you have "generic" management. Generic management finds unneeded expenses and eliminates them. The only way a senior expert isn't a cost to be eliminated is if you have managers focused on and understanding the enterprise they are managing (and no promises with that, however).
  - seb1204 3y ago
    When the AI has QA/QC as part of the over all requirements should this not solve it? A question of ensuring all requirements are internalized.
    - 3y ago
lozenge 3y ago
> however with software nobody dies Eh, not a good hard-and-fast rule. Fujitsu's Horizon software drove some of its users to suicide.
- magnetowasright 3y ago
  Australia's robodebt scheme also caused suicides, but unlike the Fujitsu horizon shenanigans, the suffering was largely the point of robodebt
JohnFen 3y ago
> however with software nobody dies Unless that software is running a life-critical or potentially life-threatening piece of equipment. People have died from software bugs in such things.
ccakes 3y ago
> however with software nobody dies Software that runs systems which can directly kill people does tend to get a lot of scrutiny, but there is also a lot of software which can indirectly kill someone that doesn’t get the same level of attention https://www.technologydecisions.com.au/content/convergence/article/tpg-fined-4-for-failures-1393667141 https://www.technologydecisions.com.au/content/convergence/a...
oglop 3y ago
100% agree. Had a manager that does this try and tell the Challenger story once. He had the lessons of the report exactly backwards. Instead of the managers creating an environment of risk by not understanding the engineering and overriding the engineers, he claimed it was engineers not listening to or informing managers properly. It was wild to me to sit there and hear this manager say the exact wrong lesson of that tragedy.
runeb 3y ago
> however with software nobody dies https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25

by proc0 3y ago

necheffa 3y ago

> however with software nobody dies Well...no actually. This may be the case for _some_ software. I happen to work on (software) products where public safety is a concern.

seb1204 3y ago
Well in the story is the point of an software error leading to the plane crash right...

margalabargala 3y ago

> however with software nobody dies The MCAS issue which crashed two Boeing planes was a software hack.

SahAssar 3y ago
Well, yes, but also no. It was a hardware design change that necessitated a software hack (to escape mandatory retraining of pilots) that relied on unreliable hardware, right? Sure, software played a big part in it, but I think it seems like it was more a management and communication failure. If it was just software it'd probably be much easier to diagnose and fix.
- margalabargala 3y ago
  I disagree. The hardware design of the cockpit is intended to be such that any computer inputs also move the pilot's controls, so that the pilots can countermand computer inputs if necessary. In this case, software was written such that this was not possible. The software that operates MCAS operates on a garbage-in-garbage-out model, like most software. There was no software written to determine if the incoming data was garbage, thus the software decided to crash two planes. Here's an article that goes into detail on the software: https://spectrum.ieee.org/how-the-boeing-737-max-disaster-looks-to-a-software-developer https://spectrum.ieee.org/how-the-boeing-737-max-disaster-lo... > When the flight computer trims the airplane to descend, because the MCAS system thinks it’s about to stall, a set of motors and jacks push the pilot’s control columns forward. It turns out that the Elevator Feel Computer can put a lot of force into that column—indeed, so much force that a human pilot can quickly become exhausted trying to pull the column back, trying to tell the computer that this really, really should not be happening. > Indeed, not letting the pilot regain control by pulling back on the column was an explicit design decision. Because if the pilots could pull up the nose when MCAS said it should go down, why have MCAS at all? > MCAS is implemented in the flight management computer, even at times when the autopilot is turned off, when the pilots think they are flying the plane. In a fight between the flight management computer and human pilots over who is in charge, the computer will bite humans until they give up and (literally) die. At the end of the day, if the person who wrote that software had written it differently, then those planes would not have crashed and hundreds of people would not have died.
  - error503 3y ago
    > At the end of the day, if the person who wrote that software had written it differently, then those planes would not have crashed and hundreds of people would not have died. You can't really blame the software engineers. This was all thought out and tightly specified by Boeing to their avionics subcontractor (Collins, IIRC). This is how it was designed and engineered to work at a systems level - it is a design hack. As far as I know there weren't any software bugs or 'hacks' involved, and the avionics operated as designed (aside from the AoA DISAGREE alert, which was due to a requirements miscommunication, not a bug). It was broken by design, which happened long before implementation, at Boeing. > When the flight computer trims the airplane to descend, because the MCAS system thinks it’s about to stall, a set of motors and jacks push the pilot’s control columns forward. It turns out that the Elevator Feel Computer can put a lot of force into that column—indeed, so much force that a human pilot can quickly become exhausted trying to pull the column back, trying to tell the computer that this really, really should not be happening. The Elevator Feel Computer is a part of the 737NG as well, and would behave the same way in those airplanes when receiving such erroneous AoA data; it's nothing new in the MAX. It certainly did not help the crews during the fatal MAX incidents, and is clearly not an ideal design, but it's also barely a footnote in the root cause analysis, along with the stick shaker and stall warnings blaring at them constantly. The pilots would easily be able to overcome it long enough to get safely on the ground. What was a bigger problem for those crews was that the MCAS has enough trim authority to make it impossible, with any amount of elevator input, to restore level flight - limiting its trim authority was part of the 'fixes' required to get them airborne again. I don't think it's reasonable to blame the implementation of MCAS for the accidents, its existence is to blame, and really highlights how nothing about the 737 platform has been designed holistically - it is a patchwork of hacks on hacks dating from the 1970s, which is difficult to reason about as a whole, and has dark corners. To truly 'fix' MCAS, you need to consider AoA as critical air data (which the 737 does not), and you need to integrate it holistically with the rest of the flight controls (which the 737 cannot, since it is not fly-by-wire), and you need to consider it critical equipment (it's an 'augmentation' and not considered critical on the 737, 'justifying' the lack of redundancy). Once you've done those things, you've basically got the bones of a proper envelope protection system in place, and you've obviated the need for MCAS in the first place. Of course the 737 team couldn't do this, because the business decided that it was more important to avoid (and hide) any differences than to bring the aircraft in line with modern standards. Realistically, this should have been trapped by the safety analysis of the flawed design, which should have considered its effect on the whole flight control system when evaluating it, but Boeing again only considered MCAS to be an 'augmentation' and it got an abbreviated safety review as a result. Some engineers did express concern about some of these factors, but given the environment outlined in TFA, those concerns did not go anywhere, because they would have basically scuttled the idea and sent everyone back to the drawing board, which Boeing was desperate to avoid having already been caught flat-footed with the launch of the A320neo. The 737 airframe needs to be put to rest, it is simply not safe or sane to keep stacking more hacks onto it. But there's no indication Boeing's working on a successor so it's probably going to be on the market for another 20+ years. Hard to imagine folks will probably be flying on an airframe with a 100 year old design (2024 + 20 years before a new revision + 20 years life span = 2064, around 100 years from the 737 launch before they start retiring)!
proc0 3y ago
I meant software only products like websites, apps, games, etc., but it should be clarified, good point.
idontpost 3y ago
[dead]

rjbwork 3y ago

Great idea but how will the bean counters and schmoozers get their multi-million dollar bonuses if they can't force engineers to do 5 jobs while paying them for 1? Will never work.

pfortuny 3y ago
quality control is most of the times unmeasurable in the short term…

Terr_ 3y ago

The difference is whether organizations are willing--or forced--to invest in it. That's not automatically a wrong though, since different objects or processes merit different levels of quality.

mattgreenrocks 3y ago

The real fix is popularizing the notion that management is just as commoditizatable than those they seek to commoditize. Note that in the recent dialogue about AI eating jobs, there's zero mention of whether it could come for management positions. Nothing. Curious, isn't it? Why wouldn't an LLM be good enough at this? I mean, it's really data-driven, right?

passwordoops 3y ago
Frankly I think ChatGPT 3.5 was already good enough to replace the majority of CxO positions
- lrem 3y ago
  Why not a Markov chain generator?
  - paradox460 3y ago
    Or an 8 ball
joe_the_user 3y ago
The real fix is popularizing the notion that management is just as commoditizatable than those they seek to commoditize. What's described here is exactly what happens when you have "generic" management. Generic management finds unneeded expenses and eliminates them. The only way a senior expert isn't a cost to be eliminated is if you have managers focused on and understanding the enterprise they are managing (and no promises with that, however).
- seb1204 3y ago
  When the AI has QA/QC as part of the over all requirements should this not solve it? A question of ensuring all requirements are internalized.
  - 3y ago

lozenge 3y ago

> however with software nobody dies Eh, not a good hard-and-fast rule. Fujitsu's Horizon software drove some of its users to suicide.

magnetowasright 3y ago
Australia's robodebt scheme also caused suicides, but unlike the Fujitsu horizon shenanigans, the suffering was largely the point of robodebt

JohnFen 3y ago

> however with software nobody dies Unless that software is running a life-critical or potentially life-threatening piece of equipment. People have died from software bugs in such things.

ccakes 3y ago

> however with software nobody dies Software that runs systems which can directly kill people does tend to get a lot of scrutiny, but there is also a lot of software which can indirectly kill someone that doesn’t get the same level of attention https://www.technologydecisions.com.au/content/convergence/article/tpg-fined-4-for-failures-1393667141 https://www.technologydecisions.com.au/content/convergence/a...

oglop 3y ago

100% agree. Had a manager that does this try and tell the Challenger story once. He had the lessons of the report exactly backwards. Instead of the managers creating an environment of risk by not understanding the engineering and overriding the engineers, he claimed it was engineers not listening to or informing managers properly. It was wild to me to sit there and hear this manager say the exact wrong lesson of that tragedy.

runeb 3y ago

> however with software nobody dies https://en.wikipedia.org/wiki/Therac-25 https://en.wikipedia.org/wiki/Therac-25