7 ms·
Outsourcing Cost Boeing Billions (2019)
- jonplackett 2y agoShould have ‘2019’ in title
- agomez314 2y agoDone!
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- selfmodruntime 2y agoThe more often capitalism strives to cost reduction and profit-maximizing at the cost of quality -- and sometimes lives --, the more often I think that companies should rely on non-voting shares far more often, with a promise to not blindly follow profits over all else. I currently have very limited options if I want to invest my money into an (engineering) quality-first company.
- belter 2y ago> The more often capitalism strives to cost reduction and profit-maximizing at the cost of quality Dumb capitalism only? > Very limited options if I want to invest my money into an (engineering) quality-first company. NVIDIA, Porsche, Tektronix...
- uytersdo 2y ago> Dumb capitalism only? Ah yes, we all love that famous tenet of ‘smart capitalism’: “the more you share the more your bowl will be plentiful”.
- FroshKiller 2y agoNVIDIA? With the melting power connectors on their high-end cards?
- selfmodruntime 2y ago> Dumb capitalism only? More slightly-ethical, hopefully sustainable capitalism. I liked my products pre-enshittification, when they lasted 20 years and were built like tanks. Hark on them all you want, but I‘ve been using my 2016 MacBook for 8 years without problems.
- bananapub 2y ago> I think that companies should rely on non-voting shares far more often, with a promise to not blindly follow profits over all else. sure, but how is that relevant? it's not like the previous CEO of the company was sacked by shareholders because they demanded a 0.05% increase in net revenue.
- selfmodruntime 2y agoIt’s more like CEO‘s are focused on short term gains in order to get shareholders to grant them large pay packages.
- greenthrow 2y agoThe 2024 version of this is "over reliance on Copilot et al. costs various industries billions."
- Cthulhu_ 2y agoBut also, lack of oversight, reviews, inadequate testing, etc. I may not be $9 / hour but I still wouldn't trust myself to write aviation software. Nobody should have that kind of hubris, yet there's developers like that in all income tiers. Trust no code, least of all your own.
- Ekaros 2y agoHere I would question the whole process. Do these developers have even any chance to stop this process or raise up issues? Is anyone listening to them? Or will they end up being removed if they raise up something?
- zeroc8 2y agoWhen writing aviation software, there is a process you need to follow, DO-178C. So even if you produce bugs, they should be caught in later stages. Everything needs to go through a risk assessment phase. The fact that they were determining that one AOA sensor would be enough for such a critical system is what needs to be examined. And people involved with this decision need to be put into jail.
- plimko 2y ago[flagged]
- api 2y ago[flagged]
- bananapub 2y agoI wish people wouldn't spread this nonsense. Public companies can care about whatever they want until their shareholders sack the CEO, which is an extremely high bar. It seems extremely improbable to me that there would be a shareholder revolt over "hi, we'll be increasing COG by a tiny amount to improve quality of the METAL BIRDS PEOPLE PAY TO BE PUT IN". your blithe blackpilled acceptance of sociopathy is wrong and unhelpful.
- Sl1mb0 2y agoAlright guy, I'll bite. Why do _you_ think that Boeing off-loaded manufacturing burden to 3rd party companies? Their executives have effectively gutted the company over the last 20 years. Now we're left with a bailed-out husk the government literally _cannot_ afford to let go out of business.
- orwin 2y agoThe shareholders choose the CEO to improve returns, not to improve safety, until safety safety impact returns (which it did for Boeing). Which is fair, we live in a capitalist world, capital is thus by definition deciding power and wanting to have more of it is a decent want.
- exe34 2y ago> your blithe blackpilled acceptance of sociopathy is wrong and unhelpful. if only shareholders and boards of directors held your innocent rose tinted views, the world would be a lovely place indeed.
- andrewstuart 2y agoAre you saying the CEO really wants to pay for high quality local software engineers but due to external factors behind their control they really can’t.
- 2y ago
- beeboobaa3 2y agoThese "software engineers" also killed 189 people. I don't think Boeing losing billions is all that important in the grand scheme of things.
- janice1999 2y agoManagement at Boeing killed those 346 passengers.
- Ekaros 2y agoShareholders of Boeing killed them in the end. They wanted these profits and stock prices, but sadly they are the group that is protected.
- ahoka 2y agoSo at the end of the day it was you and me killing those poor souls by investing our pension money in those companies?
- Ekaros 2y agoExactly. Whole current financial system is build on this.
- paulryanrogers 2y agoDon't your investors oversee cutting of corners like recycling old designs to the breaking point? Or hiding critical new workarounds in the shallowest possible training?
- _cenw 2y ago"the sole fiduciary duty is to deliver profits" is a reductive meme at this point. The problem is putting short-term over long-term, and that's a choice Boeing executives made. Of course the system incentivizes this, especially if you get away with it. But they didn't.
- 2y ago
- deathtrader666 2y agoThe original article on Bloomberg with the same title - https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-...
- Sytten 2y agoAlmost all software outsourcing I have seen end up costing more in reputation damage, drop in quality, bug pilling up, incompetent programmers, etc. World would be a better place if MBA managers were not running it. Anyway this is why professional engineering were created, to protect the public against those exact bad practices. If we were serious about fixing the issue, we have the solution.
- gavinhoward 2y agoAnd I already have a plan for doing so: https://gavinhoward.com/2024/06/a-plan-for-professionalism/ https://gavinhoward.com/2024/06/a-plan-for-professionalism/ .
- CM30 2y agoThe problem with any professionalism setup is that software engineering is a very broad field whose use cases range from "it doesn't matter, it's mostly for fun" and "this software runs the international space station". On the one hand, there should probably be some sort of certification or liability if you're writing code for NASA or Boeing or Airbus or maybe the New York Stock Exchange or what not, where a mistake could cost billions of dollars and put lives at risk. At the same time though, there's also stuff like the video games industry, people developing programs for fun, blogs and fan sites and things that are free and mostly static, video game mods and hacks and about 99% of small business and organisation websites where frankly it doesn't matter that much how well they're coded or how stable they are or what not. If my bank screws up and gives away all my money, that's a problem. If my copy of Tears of the Kingdom breaks, that's not a big deal (and might even be a benefit). If there's any sort of professionalism or certification or requirement, it should probably be limited to industries and fields where the consequences of screwing up are life changing for a high number of people.
- gavinhoward 2y agoYes, which is why https://gavinhoward.com/2023/11/how-to-fund-foss-save-it-from-the-cra-and-improve-cybersecurity/#software-certification https://gavinhoward.com/2023/11/how-to-fund-foss-save-it-fro... specifically calls that out.
- rhelz 2y ago$9 an hour is shameful. But what's really scary is that my software has bugs in it too.
- manuelmoreale 2y agoWhy do you think $9 an hour is shameful? Doesn’t that entirely depend on the cost of living? $9 could be 3x the average wage in some parts of the world.
- myworkinisgood 2y agoIt's shameful for the quality of work asked for.
- manuelmoreale 2y agoSo you're not saying it's shameful in general, just in this specific context because of the type of work involved?
- rhelz 2y agoIf you take the article at face value--if its claims are true that $9-an-hour programmers cost Boeing billions of dollars--it is indeed a shame that, in this particular instance, Boeing was not willing to pay more for better programmers. Are those claims true? Who knows. I'm sure that whoever hired those $9-an-hour programmers thought that if they could manage those programmers soooo well that they could get them to program as well as $100 an hour programmers. YMMV, but in my experience, you can't manage programmers to be better programmers. No matter how much check-in approval bureaucracy, or how much QA, you will not get better programs. If you want better programs, you need to hire better programmers.
- manuelmoreale 2y agoI’d agree on that wholeheartedly. Especially the last part.
- andrewstuart 2y agoI don’t have much sympathy for companies who go this path and get the results.
- _cenw 2y agoI seriously doubt this is the mistake of $9/hr workers. Those workers are unlikely to be familiar with the whole picture and would probably just have requirements delivered to them. This is not a case of badly written code, the code didn't crash. It just didn't fit the requirements, likely because those requirements weren't known, because someone I'd wager working directly at Boeing getting a industry-typical salary didn't deliver them.
- gnfargbl 2y agoPart of good engineering is having a solid understanding of your business domain and being able to spot things that just don't smell right. An experienced professional software engineer on an industry standard salary, having been provided with nonsensical or missing requirements, should know when to impute and when to raise an exception. I'm not at all confident that a $9/hr engineer will do that.
- _cenw 2y agoWell, we don't know if they did. We do know that Boeing employees (including one of the whistleblowers) have attempted to raise such issues internally for a long time, but mostly went ignored, moved to a department where they can't complain, or fired. So purely on the power differential, a $9 engineer wouldn't make a stink. Says nothing about their work ethic or ability to see disaster (at least upon integrating code, probably not literal disaster) coming. And also, how would they have known they're just using one AOA input, and a component above them isn't already delivering a value that's known good - these are not the kind of engineers that'd know that.
- smeej 2y agoI'm not at all confident that a $9/hr engineer can do that. You don't hire somebody for a price like that because you intend to listen to their feedback and treat them like a valued member of the team. You hire somebody for a price like that because your LLM hasn't quite reached the level you want yet...or frankly costs more to run.
- igortg 2y ago
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- xiaodai 2y agomore racism and hate basically.
- cpursley 2y agoCan you elaborate?
- kapad 2y ago> I seriously doubt this is the mistake of $9/hr workers. Those workers are unlikely to be familiar with the whole picture and would probably just have requirements delivered to them. This is not a case of badly written code, the code didn't crash. It just didn't fit the requirements, likely because those requirements weren't known, because someone I'd wager working directly at Boeing getting a industry-typical salary didn't deliver them. source: https://news.ycombinator.com/item?id=41328272 https://news.ycombinator.com/item?id=41328272
- deleted 2y ago[deleted]
- kopirgan 2y ago[dead]
- ChumpGPT 2y agoWithout mentioning any names.... The largest company in it's industry that uses many mission critical applications systems every second of the day, is in the process of offshoring all its software development to you know where. When a company brings in an Indian CIO, it's only a matter of time.
- kjrfghslkdjfl 2y ago[dead]
- dghughes 2y agoI don't see how $9 is relevant. If they are based in India $9USD/hour is 755 Indian rupees. That $9 is a really good wage. A tech worker in India supposedly makes 32,000 rupees per month. So that's 200 rupees per hour or $2.38USD. It's relative to the region and its economy. I'd love to have my Canadian wage paid in Kuwaiti Dinars one for one.
- TrackerFF 2y agoSo let's assume the following: You outsource your software development to some country with quite low average monthly salary. Say some country in eastern Europe, where average salary is $500 / month. Then you only hire from the top CS programs in the country and pay them, say, 3 times the national average - so $1500 / month. Given 150 work hours in a month, that comes out to $10/hour gross pay. You get good engineers, pay less, and they earn more than they'd get from pretty much any domestic employer. If the average software dev in the US makes, dunno, $70k / year, then that would be the equivalent of getting hired on a $210k / salary. Of course, it's not all smooth sailing - but I think the important part here is to keep in mind that pay is relative. What could be a pitiful salary in rich western countries, could be a very good salary other places.
- deleted 2y ago[deleted]
- ffgjgf1 2y ago> Say some country in eastern Europe, where average salary is $500 / month. Then you only hire from the top CS programs in the country and pay them, say, 3 times the national average - so $1500 / month. This is 10-15 years out of date. And even then it was hardly ever as straightforward as that. And only ever applied to junior and maybe mid level developers or those who couldn’t effectively communicate in English. High skilled one were significantly more mobile and were basically competing with a much more global pool of developers. They could relatively easily move (if not to the US then at least Western Europe so your potential savings were usually limited to the difference in CoL + some premium). This led to pretty high inequality based on skill/experience e.g. to top CS graduates could expect their income to increase by 3x if not more over the next 5 years after graduation.
- typewithrhythm 2y agoThis assumes that A) The institutions in the country make it possible to identify good from bad local candidates. B) There are enough experienced people in the region to have a healthy functional workplace. C) The outcome is at least proportional to the cost. In my experience none of these are true.
- 2y ago
- YouWhy 2y agoI think that describing the MCAS disaster using predominantly SW Engineering language is misleading. The substantial change that might have averted the catastrophe is having qualified engineering oversight integrated into the MCAS project management structure. MCAS is a flight control application; its defining discipline is control theory. Thus the hypothetical engineer who could have averted the catastrophe would have had to be a controls, rather than a SW person. I have read much about MCAS, but no detailed narrative ever mentioned a SW bug, which implies that MCAS SW has apparently functioned according to given specifications. Thus, while $9/hr SW engineers is a choice correlated with an inadequate safety culture, I fail to see the casual link between that and the tragic outcomes; I cannot condone asserting such causation.
- agomez314 2y agoApparently 1/3 of all software vulnerabilities represent design weaknesses which were introduced in the requirements phase. The MCAS flaw seems to belong to this category which you describe. source: https://insights.sei.cmu.edu/blog/a-tool-to-address-cybersecurity-vulnerabilities-through-design/ https://insights.sei.cmu.edu/blog/a-tool-to-address-cybersec...
- jvanderbot 2y agoSeparating controls from SW is a strange thing to do. The issue here is a culture of shortcuts and awful money-driven decisions, not any particular discipline, though delayed software fixes did cause more accidents, and it was poorly designed overall. Ultimately, the MCAS relied on a single sensor, one which was known to fail, and only displayed the redundant sensor data - get this - if they bought the additional option to show when the sensor failed. > However, whereas MCAS was activated automatically, without pilot action, the cockpit crew would have to notice and act on an AOA DISAGREE alert. Further, the AOA indicator and disagree alert were not standard equipment on the 737 MAX, although the AOA indicator had been on earlier models. Boeing offered them as “add ons” at additional cost Without that "add on" you'd never know the MCAS was acting on faulty data. And all the while management worked overtime to mislead regulators on the potential impact of MCAS to avoid additional scrutiny and training requirements. So nobody knew the safety of their aircraft depended on that non-standard equipment package. Yeah - total business and design mismanagement driven by greed. --- https://mitsloan.mit.edu/teaching-resources-library/boeings-737-max-8-disasters https://mitsloan.mit.edu/teaching-resources-library/boeings-... > When I say I changed the culture of Boeing, that was the intent, so it’s run like a business rather than a great engineering firm. It is a great engineering firm, but people invest in a company because they want to make money. --Harry Stonecipher, 2004 I think that outsourcing was a symptom of this disease. Not MBA cancer, not poor software, not anything other than a deliberate cultural shift which led to all those other things.
- deleted 2y ago[deleted]
- chimp_brain 2y agoFeels like someone completely ignores incompetence of those who wrote the requirements
- WirelessGigabit 2y agoBut if you present your badly written requirements to a person who has the culture to ask questions, you at least have that safeguard. After all, it's a collaboration. And when you work with an offshore team, quite often as contractors, you have the issue that you don't get top quality. The in-between company will do its very best to hire low to maximize profit. The good quality developers work either on local grown software, or leave the country for better opportunity. And you are now working with a group of developers whose livelihood depends on saying 'yes' or not saying 'no. And because of this you will not know whether there is an issue in the requirements until x time later and the plane goes down.
- random_ind_dude 2y agoI remember reading that development of the MCAS system was not done by offshore companies, which was confirmed by Boeing. So saying that $9/hour employees caused the 737 MAX issues is incorrect. https://www.industryweek.com/supply-chain/article/22027840/boeings-737-max-software-outsourced-to-9-an-hour-engineers https://www.industryweek.com/supply-chain/article/22027840/b... Excerpts from the the above article: >Boeing said the company did not rely on engineers from HCL and Cyient for the Maneuvering Characteristics Augmentation System, which has been linked to the Lion Air crash last October and the Ethiopian Airlines disaster in March. The Chicago-based planemaker also said it didn’t rely on either firm for another software issue disclosed after the crashes: a cockpit warning light that wasn’t working for most buyers. >In a statement, HCL said it “has a strong and long-standing business relationship with The Boeing Company, and we take pride in the work we do for all our customers. However, HCL does not comment on specific work we do for our customers. HCL is not associated with any ongoing issues with 737 Max.” >Based on resumes posted on social media, HCL engineers helped develop and test the Max’s flight-display software, while employees from another Indian company, Cyient Ltd., handled software for flight-test equipment.
- jvanderbot 2y agoCheck out this: https://mitsloan.mit.edu/teaching-resources-library/boeings-737-max-8-disasters https://mitsloan.mit.edu/teaching-resources-library/boeings-... It's a super fun read. And you're absolutely right: Boeing caused the problem itself. Shifting blame to poor software "accidentally" produced by 9/hr engineers is not legitimate. There was a bug. But it was only a problem because all the other redundancies and safeguards were shortcut and regulators and pilots were mislead about the severity of MCAS system's "corrections" and not given proper data to know when to override it.
- freitzkriesler2 2y ago[dead]
- gxd 2y agoAs they say, "I'm too poor to afford cheap things". In the business world, that corresponds to "we can't afford to hire inexpensive, unqualified labor".
- blitzar 2y agoStaff blame non staff, passing blame to someone else that is not them and remaining employed. In other news; dog fights cat, more on that bizarre story after the break.
- aussieguy1234 2y agoIn some countries, they literally hire people off the street for coding who aren't even programmers, to sell them to suckers in rich countries overseas who will buy their services. Any decent engineer worth their salt gets a visa to work in the said rich countries where they can earn 10x more. They don't stay in the local lower paying roles as they can't compete on salary. You get what you pay for.
- serverlessmania 2y agoThis article is a click bait.
- hintymad 2y agoI forgot which book mentioned that big government contractors like Boeing had to set up their factories and give business to vendors in many, if not all, the states no matter how inefficient or expensive that is because senators wouldn't approve key contracts unless the contractors brought jobs to the senators' states. I'm guessing the cost of the outsourcing is also part of this compromise to the senators' demands. A trajectory question: our government also hands out billions to NGOs and contractors to "solve" social problems, yet making such problems worse. For instance, multiple articles reported that how so many organizations charged ridiculous amount of money to SF government to help the homeless, yet the city's situation only got worse over the years. Wouldn't this be a worrisome sign of a society. An empire didn't fall because of external enemies but because the empire had so many entangled interests and the society simply declined and then collapsed.
- OutOfHere 2y agoThe author himself notes paying at least a $100K global salary as of 2019, which implies he does engage in outsourcing. As such, the article is self-contradicting. Maybe the author should go back to writing about Javascript since javscript-scene is in the URL. You can get bad engineers anywhere, including in the US. It's up to management to enforce a sane engineering process, although being the parasites that they are, they never will.