10 ms·

Boeing’s 737 Max Software Outsourced to $9-an-Hour Engineers (2019)

213 points by treyfitty 5y ago

323 5y ago
Makes sense, software is not their core competency - obtaining contracts. Everything non-core, like engineering or software should be outsourced to the lowest bidder.
pinewurst 5y ago
"Boeing has also expanded a design center in Moscow. At a meeting with a chief 787 engineer in 2008, one staffer complained about sending drawings back to a team in Russia 18 times before they understood that the smoke detectors needed to be connected to the electrical system, said Cynthia Cole, a former Boeing engineer who headed the engineers’ union from 2006 to 2010."
- bozhark 5y ago
  "ours aren't"
  - froh 5y ago
    ?
    - selimthegrim 5y ago
      Meaning the Russian ones
- throw0101a 5y ago
  Working smoke detectors would have been useful given the battery fires that the 787 experienced: * https://en.wikipedia.org/wiki/2013_Boeing_787_Dreamliner_grounding https://en.wikipedia.org/wiki/2013_Boeing_787_Dreamliner_gro...
- vasco 5y ago
  3 times would say something about the Russian team, 18 says something about both sides.
  - bcrl 5y ago
    I have worked on a couple of projects where we had to use an offshore team for QA. The biggest problem was written instruction comprehension: the offshore team simply could not follow instructions written in perfectly coherent english in an email. The only way we made progress was to set up a call where we walked the offshore team through the steps in the email - as in literally telling them to "run this command, then run that command". It was hell as we kept getting lambasted for not meeting our deliverables because the offshore team couldn't follow simple instructions to run a few scripts and record their output.
    - ClumsyPilot 5y ago
      That raises numerous questions, and all of them indicate bad management. Who is in charge of that team? Is there no bilingual, western trained person that can mange these problems? Are they not understanding what needs to be done, or are they missing that many skills? Do they not recieve training? If training is not helping, why were these candidates selected in the first place? If these were the best candidates avaliable, why was that offshore office created to begin with?
      - bcrl 5y ago
        We had no control over the other team. They were already responsible for QA of most of the system, but most of that just consisted of running existing QA test cases and reporting when they failed. As we were implementing new network protocols that they did not have existing test cases for, the work QA was responsible for was rather more complex. The issues were typically of the form "run this test script and tell us what happened for you" => "it failed" and then provided no useful information. Each step was a 24 rtt nightmare until we pushed for conference calls at 11pm local time for us. At the time we also heard of a horror tales from the offshore team. In one case a new hire had gone through the interview process, accepted the job offer, showed up for the first day, just left and never came back after a better offer came in. Suffice it to say that the offshore QA team was not exactly high quality without the ability to retain employees. It was not like the QA teams I've encountered at other jobs where QA writes a lot of their own testing infrastructure and are on an equal footing with the developers. Sadly they were more like a glorified checklist processing system, and were probably in over their heads.
kjuulh 5y ago
I just cannot see why outsourcing this way is sustainable, and it probably isn't. Some are great, and some are absolutely abysmal, an internal joke in the company was that a senior engineer was the same a junior engineer or worse. I had an a senior engineer who couldn't change a string to another string in a code base. I don't even exaggerate I had to prove to my manager that he had no skills at all.
- kitten_mittens_ 5y ago
  As an American, I think that arguments about outsourcing that boil down to ability being all over the map don't feel right. Plenty of American engineers are all over the map. The biggest difference is distance, the time zone split, and cultural differences. I say this as someone that thinks limiting insourcing raises wages in the short term.
  - kjuulh 5y ago
    I agree, I've had plenty of colleagues whom really didn't excel in the field. I think for the most part with outsourcing as you say is the distance, but for me/us (team) it was cultural, we simply couldn't mix with the foreign part of the team. I've left the country out, because I don't want to point fingers, but it was simply about face, and no asking questions at all. To be honest, there were mistakes on both fronts, I feel like we as the engineering team in the country where the company was located was, felt entitled to our way of working, while the other party had to fit in.
  - cpeterso 5y ago
    Plus companies are outsourcing to reduce development costs. They get what they pay for, regardless of country.
- thelittleone 5y ago
  Not all execs care about "sustainable". They often have quarterly targets. To get some wins an exec could outsource, complete a bunch of tasks, then move to another company on a higher paid job, leaving former company to mop up. See this a lot with sales also. Transactional client relationships promising the world to get the deal and hit commission multipliers. They then move to another company with "success stories" leaving implementation teams to reset client expectations.
newuser94303 5y ago
There are bad engineers in the US as well as India. If you are paying $9 per hour, you are not getting the good Indian engineers. It is like going to a local McDonalds and hiring programmers. You deserve what you get.
- ddjsn111 5y ago
- dboreham 5y ago
  Where I live in the middle of the usa McDonald's is paying $19/hr
- austincheney 5y ago
  That’s becoming hard to fathom when you can now get hired as a React developer with about 3 years experience for $170k without any understanding of how things work under the hood.
  - woodruffw 5y ago
    Why would a React developer (and hopefully SME, if they've been at it for 3 years) need to know anything "under the hood" to earn their value? I work on compilers for a living, but I couldn't build an IC if you put me in front of an EUV machine. Does that mean that I don't have an understanding of how things work "under the hood"?
    - austincheney 5y ago
      “Under the hood” is fancy talk for doing more than put text on screen. Whether or not that’s needed depends on the employer. BTW, nobody is an expert within just 3 years. That is a key indicator of a Dunning-Kruger moment that becomes clear by drilling down just a bit into key performance indicators from research and experience.
      - woodruffw 5y ago
faangiq 5y ago
“Yep, just fungible code monkeys.” -every company, even FAANG
- gruez 5y ago
  >even FAANG Why are they paying american programmers 6 figure salaries then?
  - deleted 5y ago
    [deleted]
  - MattGaiser 5y ago
    We are valuable commodities. Something being expensive doesn't mean that it is valued on an individual basis. An ounce of gold is expensive, but extremely fungible.
  - Sebguer 5y ago
    Because they extract more value than that from them. Doesn't make them less fungible.
    - gruez 5y ago
      Even if they can afford it, it still defies rational behavior to pay 6 figures for an american programmer when they're "fungible" with indian programmers making $9/hr. I'm sure a FAANG programmer "extracts more value" from a $20 cup of coffee than it costs. That doesn't mean it's not highly unusual/irrational behavior, considering that other coffee shops charge a few bucks for a cup.
  - ushakov 5y ago
    because they drive millions of $ in profits and because all the great engineers are already employed somewhere else
  - Nicksil 5y ago
    >Why are they paying american programmers 6 figure salaries then? Because those companies are making an outsized profit from the relatively new and unregulated online advertising/marketing industry.
DantesKite 5y ago
That seems like a bad decision from a security perspective too. I can imagine cyberattacks meticulously crashing airplanes during a time of war.
cletus 5y ago
This is what happens when Boeing is run by accountants and MBAs instead of engineers (as it once was). Short term financial "wins" like outsourcing eventually make Boeing unrecognizable from the industrial behemoth that cranked out 747s in Everett. What these finance and business people fail to understand (or simply don't care about) is that every point of outsourcing becomes a point of friction. You now need to negotiate specs, deliverables, timetables and payments. There are often perverse incentives that reward change orders. You often get cost blowouts from "cost plus" pricing.
- Victerius 5y ago
  If I started my own aerospace company I would try to put in a legally binding rule that the majority of management and the executive team has to be engineers. But I don't know if an executive team can tie its successors hands like that. Unless the majority of voting shares remain in the hands of the family of the founder and successive generations commit to keep the company run by professional engineers. That, or a federal law.
  - dboreham 5y ago
    The medical business works this way, fwiw.
    - Sebguer 5y ago
      Has that actually made things better?
    - mbreese 5y ago
      What medical business is like this? You don't need to be a doctor to own or run a hospital or pharma company. The legal business is like this however. I'm fairly certain that, offices must be owned by lawyers. The thought being that if a law office was owned by a non-lawyer you'd have a conflict of interest if your boss told you to do something illegal or unethical.
  - pinewurst 5y ago
    How would that make a difference? Both the odious Harry Stonecipher at Boeing and the even more odious Jack Welch started as technical people.
MattGaiser 5y ago
> making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace -- notably India. I have always wondered about the consequences of vast numbers of devs working in fields they barely understand (myself included) and the relatively rapid churn of those devs. That being said, in most places I have worked, developers don't get a lot of influence in big picture decisions. If Boeing is like that, it would not have mattered.
okl 5y ago
There's good/bad engineers everywhere. I have worked with students that earned less than $9/h and produced better quality work results than highly payed senior engineers. Nationality is also not a reason for high quality/sloppy work. However, I can accept that expecting to get the same quality work for a fraction of the cost should be viewed skeptically.
- Victerius 5y ago
  > Nationality is also not a reason of high quality/sloppy work. In some ways it can be, though. Poor countries tend to be poor for a myriad of reasons, but among these reasons should be the lack of a quality education system. Wealthy countries tend to have excellent school systems and universities, and a better trained student body, than poor countries. If poor countries had the education systems of wealthy countries, they wouldn't be poor, but the reason that can't sustain a system like that is precisely because they are poor.
  - Sebguer 5y ago
    I mean, India was "poor" because it had 90 years of having everything of value stolen by the British Empire.
    - Victerius 5y ago
      That's one point of view, yes. Like I said, there are a lot of reasons why poor countries became poor, or failed to industrialize. Personally I don't think India would be America 2.0 (or China 2.0) had the Brits never colonized it. The partition of India happened 75 years ago, however. The British Empire is no longer a valid excuse.
      - yunohn 5y ago
        > The partition of India happened 75 years ago, however. The British Empire is no longer a valid excuse. This particular argument is usually made without understanding history. India, the country, did not exist before British enslavement & colonisation. India’s diversity and size are comparable to the entire EU, with double the population. I don’t see why 75 years is /too long/. It's quite tiring to hear such naïve takes. I would really recommend reading some books…
jka 5y ago
Perhaps a dumb question, but is there a definitive connection between the outsourcing mentioned here and the MCAS[1] system that has caused the 737-MAX tragedies? (I have searched the web to try to confirm a link; there are some articles[2][3][4] that mention both MCAS and the outsourcing, but I haven't yet seen anything to indicate that outsourcing was used for development of MCAS-related software) [1] - https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Augmentation_System https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Au... [2] - https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-... [3] - https://pulse.icdm.com.my/resource/boeing-a-plane-wreck/ https://pulse.icdm.com.my/resource/boeing-a-plane-wreck/ [4] - https://simpat.tech/did-boeings-software-development-outsourcing-contribute-to-its-737-max-8-failures/ https://simpat.tech/did-boeings-software-development-outsour...
- imglorp 5y ago
  The software was not truly to blame for the deaths. Management was fully aware that a redundant AOA sensor was desireable: they charged extra for it. Management also demanded a product that did not require pilot training. Then there was the whole certification malfeasance. The software was one piece of an impossible configuration.
- AnimalMuppet 5y ago
  $9/hour devs in India are less likely to say "Only listening to one angle-of-attack sensor is stupid; you need to redesign this." They are especially less likely to keep pushing until they get management to listen. I mean, sure, the designers should have known that, too. But by outsourcing the software development, they missed a chance to have the bad design questioned and corrected.
  - bdavis__ 5y ago
    in general, a software dev anywhere might not realize the flaw. but an systems engineer or other aircraft and flight controls engineer would notice it. and they should have, when the requirements were written. and again when each requirement was verified prior to flight.
VWWHFSfQ 5y ago
> the iconic American planemaker and its subcontractors have relied on temporary workers making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace -- notably India This is bullshit. At least, for anything important. > Bloomberg | Quint is a multiplatform, Indian business and financial news company. So this is defintely bullshit. Edit: I don't even know what the implication they're trying to make is. Are they saying that the airplanes crashed because some incompetent Indians wrote the software?
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
- dralley 5y ago
  >I don't even know what the implication they're trying to make is. Are they saying that the airplanes crashed because some incompetent Indians wrote the software? The implication is that Boeing cares so little about the quality of the product that they are willing to pay $9 an hour for it. Anyway, regardless of the merits of the offshore team, Boeing is still responsible for any flaws in their work product. You can't just point fingers at your subcontractors when you're the one who hired them and then signed off on their work.
harry8 5y ago
As a CEO, if your team sets up a company to succeed, the payoff is likely not for the next decade. You're setting up some other executive's payday. If you burn the co's reputation to get your payday, probably nobody really notices and is likely enough for there to be no real consequences to management until after you leave. The problem you paid yourself with is someone else's to solve. Anyone suggesting there's an easy answer to that is likely full of it. How expert, competent and interested are the board of directors? How do you know? Some people look and sound plausible and convincing but aren't. Credentials are not much guide, all the board candidates have them including the total self-serving Muppets. Even noting who was great 10 years ago might not care much anymore. Boeing stock price is up more than 1% on the last day of trading. Not as much as Lockheed and Raytheon.
nonamechicken 5y ago
[I may be wrong. Hence deleting.]
- okl 5y ago
  ITAR pertains to export/import. I'm not aware of person-specific clearances. In any case it would depend on the specific component or task being worked on.
  - deleted 5y ago
    [deleted]
  - bdavis__ 5y ago
    there are not person-specific clearances, but there are person-specific non-clearances; e.g. there are lists of people and companies that you cannot do business with because of them being non-compliant with ITAR.
- kevin_thibedeau 5y ago
  ITAR doesn't have a clearance process with extensive background investigations. It is the lowest level of control for sensitive development operations. Non-citizens can work on it but there are paperwork hurdles that some employers don't want to bother with. It does also need to be conducted in US territory.
user_7832 5y ago
Something I don't think is mentioned here is that initially the Boeing designers thought that both sensors were being used for the data but the code written only took it off a single sensor. Whether it was not properly communicated to the coders or what, it is not clear, but someone at Boeing had wanted multiple sensors. (I don't remember the exact source, I'd written a report of the Max-8 crashes and this was part of what I found. If you're interested in a generally excellent source for the accident/design flaws, the (latter) US Department of Transport report is excellent. I can also share my work if someone is curious, though its quality and depth is of course much lesser.)
- zarmin 5y ago
  Someone should invent a way to check whether software is working correctly. Call it "writing checks" or "check driven development" or something.
  - radu_floricica 5y ago
    If they don't get the requirements right, they'll just write the tests wrong as well. You have the same problems, just with a bunch of extra green oks.
    - deleted 5y ago
      [deleted]
fredgrott 5y ago
Keep in mind this is the tip of the iceberg as the FAA had outsourced safety to Boeing via Boeing employees also being FAA employees. Very hard to complain against your financial master unless you fly on that plane make daily as a passenger,
imglorp 5y ago
Even if they were a stellar team, the problem with offshoring anything is always culture. The offshore team is isolated from the company, all of its storied century of knowledge (in this case), about engineering, materials, machining, quality, certification, aerospace, systems, it goes on and on. It's inconceivable to spawn any human safety article outside of that culture into a vacuum behind a contract written by lay business people and PM's.
- rjzzleep 5y ago
  That’s not really the culture problem here. The problem is that if they outsourced it to a team at $9 an hour someone at Boeing thought they made a killing. While the offshore team gets $9 an hour, the engineer itself gets a fraction of that. And those engineers are very well aware that you didn’t choose them because you think they’re amazing. They know you chose them because you don’t value engineering. So regardless of whether they are good or not they’ll do the minimum work possible. I remember a bigger automation softwares CTO telling me about a bigger enterprise that offshored their database change management process. The process was defined as putting a word file put into a jira issue. At the end of the approval process some person in India had to paste it into the database shell. If word auto correct messed it up the whole process gets marked as failed and it started from 0. That person pasting into the database has no interest whatsoever on making any recommendations even if they could.
  - qchris 5y ago
    Not to take the easy pun here, but by making this decision and ones like it, someone at Boeing did make a killing. Or 346 killings, more precisely. The same culture and internal processes that allowed this to occur are the exact same ones that let the MCAS design flaws get through, and I will forever be furious that Boeing as an organization was not ripped into parts for that failure.
  - laurencerowe 5y ago
    I wonder if that cultural problem stemmed from Boeing’s 2001 relocation of their headquarters to Chicago, a decision made explicitly to separate their financial infrastructure from their existing operations.
HL33tibCe7 5y ago
Pay peanuts, get monkeys
victor106 5y ago
About 20-15 years ago we found offshoring teams had terrible communication and software skills. In the past 7-5 years the communication and techskills for a little better but it’s still no where near close to an onshore resource. However occasionally i find an offshore resource who is exceptional.
- Consultant32452 5y ago
  The last mega-corp I worked for had this program where they primarily hired offshore workers, and they offered the really good ones the chance to move to the US on work visas. I made friends with one of them and he told me there is rampant fraud on the offshore teams. There's 2-3 people doing all the work, and they let other people claim it, because that's just the culture somehow. The off-shore managers are in on it and sometimes are one of the few doing the work. He claimed this was the norm at multiple big-name contracting firms he'd worked for. I have no way of confirm any of this obviously, but it did offer an explanation for some of the experiences I had.
  - deleted 5y ago
    [deleted]
  - mariuolo 5y ago
    > There's 2-3 people doing all the work, and they let other people claim it, because that's just the culture somehow. Could it be a question of caste hierarchy if in India?
- Karupan 5y ago
  > offshore resource In these times when “master” is renamed to “main”, I still find it strange that calling human beings “resources” is tolerated in the industry.
  - parineum 5y ago
    That could easily be a reference to a team. Honestly, when outsourcing, you really don't know who or how many people are doing the work (and that's kinda the point).
blunte 5y ago
This is just modern capitalism. If a few dozen lives are lost, it's not significant compared to the quarterly earnings per share numbers.
Simplicitas 5y ago
The https://www.rottentomatoes.com/m/downfall_the_case_against_boeing https://www.rottentomatoes.com/m/downfall_the_case_against_b... documentary is a good take on this
csours 5y ago
Who did the design, architecture and tests? The tests seem more important to me.
- sofixa 5y ago
  What tests? They hid the MCAS from regulators, so maybe they took the same approach internally - out of sight, out of mind. Why is still there nobody in prison over this ?
known 5y ago
nunez 5y ago
> Still, for the 787, HCL gave Boeing a remarkable price – free, according to Sam Swaro, an associate vice president who pitched HCL’s services at a San Diego conference sponsored by Avionics International magazine in June. He said the company took no up-front payments on the 787 and only started collecting payments based on sales years later, an “innovative business model” he offered to extend to others in the industry. Lol at the industry thinking that deferred bundles and services discounts are revolutionary
testemailfordg2 5y ago
The last line should be the title of the article. “Senior company leadership,” the statement added, “was not involved in the review.”
atoav 5y ago
Maybe that is a little it harsh of an judgement, but IMO the person responsible for this decision should be in jail.
deleted 5y ago
[deleted]
svilen_dobrev 5y ago
mmh. Not all $9-per-hour engineers are the same. Been there done that. And some of these $9 may buy them more of their-own bread-or-house than the $$RICH ones can with their 6-digit $s. Thus being happier. Envy? Competition? And of course it's always them outside teams that are to blame, while the actual elephant isn't. On the other side, when you outsource everything, .. you're what, just a web-site? oh wait, not even that.. https://berthub.eu/articles/posts/how-tech-loses-out/ https://berthub.eu/articles/posts/how-tech-loses-out/
itsdrewmiller 5y ago
>Boeing said the company did not rely on engineers from HCL and Cyient for the Maneuvering Characteristics Augmentation System, which has been linked to the Lion Air crash last October and the Ethiopian Airlines disaster in March. Feels like most of the commenters missed this from the article.
sceean 5y ago
$9 is probably a rate for a software tester. Even in the US, you can get a software tester for not much more than $9 an hour, at least that was the case in 2019. Probably, not true any more with the current rampant inflation. There is a vast salary difference between software engineer and software tester. Let's not sensationalize things unnecessarily.
mbrodersen 5y ago
The problem with profit optimising companies like Boing is that they don’t know when to stop. They will keep squeezing every last drop of blood out of their products, and their employees, until the business fail, or a competitor gets competitive enough. Especially when they hire a “General Manager” who’s only expertise is to force a 10% cost reduction on the business every year. Until the business is a dead corpse. But by then of course the CEO has already skipped ship and started the cycle again with another company.
weare138 5y ago
We seriously need to start charging the people who make these decisions at companies like Boeing for the crimes they committed like we would anyone else. Colorado sentenced a truck driver for a multiple fatality accident that killed 4 to over 100 years in prison because the driver was deemed negligent. But if you're a corporation like Boeing who's gross negligence kills hundreds the company just gets an inconsequential fine. Why do we have to play by the rules but corporations don't? Every Boeing executive who was involved in this should be in prison right now.

213 points by treyfitty 5y ago

323 5y ago

Makes sense, software is not their core competency - obtaining contracts. Everything non-core, like engineering or software should be outsourced to the lowest bidder.

pinewurst 5y ago

"Boeing has also expanded a design center in Moscow. At a meeting with a chief 787 engineer in 2008, one staffer complained about sending drawings back to a team in Russia 18 times before they understood that the smoke detectors needed to be connected to the electrical system, said Cynthia Cole, a former Boeing engineer who headed the engineers’ union from 2006 to 2010."

bozhark 5y ago
"ours aren't"
- froh 5y ago
  ?
  - selimthegrim 5y ago
    Meaning the Russian ones
throw0101a 5y ago
Working smoke detectors would have been useful given the battery fires that the 787 experienced: * https://en.wikipedia.org/wiki/2013_Boeing_787_Dreamliner_grounding https://en.wikipedia.org/wiki/2013_Boeing_787_Dreamliner_gro...
vasco 5y ago
3 times would say something about the Russian team, 18 says something about both sides.
- bcrl 5y ago
  I have worked on a couple of projects where we had to use an offshore team for QA. The biggest problem was written instruction comprehension: the offshore team simply could not follow instructions written in perfectly coherent english in an email. The only way we made progress was to set up a call where we walked the offshore team through the steps in the email - as in literally telling them to "run this command, then run that command". It was hell as we kept getting lambasted for not meeting our deliverables because the offshore team couldn't follow simple instructions to run a few scripts and record their output.
  - ClumsyPilot 5y ago
    That raises numerous questions, and all of them indicate bad management. Who is in charge of that team? Is there no bilingual, western trained person that can mange these problems? Are they not understanding what needs to be done, or are they missing that many skills? Do they not recieve training? If training is not helping, why were these candidates selected in the first place? If these were the best candidates avaliable, why was that offshore office created to begin with?
    - bcrl 5y ago
      We had no control over the other team. They were already responsible for QA of most of the system, but most of that just consisted of running existing QA test cases and reporting when they failed. As we were implementing new network protocols that they did not have existing test cases for, the work QA was responsible for was rather more complex. The issues were typically of the form "run this test script and tell us what happened for you" => "it failed" and then provided no useful information. Each step was a 24 rtt nightmare until we pushed for conference calls at 11pm local time for us. At the time we also heard of a horror tales from the offshore team. In one case a new hire had gone through the interview process, accepted the job offer, showed up for the first day, just left and never came back after a better offer came in. Suffice it to say that the offshore QA team was not exactly high quality without the ability to retain employees. It was not like the QA teams I've encountered at other jobs where QA writes a lot of their own testing infrastructure and are on an equal footing with the developers. Sadly they were more like a glorified checklist processing system, and were probably in over their heads.

kjuulh 5y ago

I just cannot see why outsourcing this way is sustainable, and it probably isn't. Some are great, and some are absolutely abysmal, an internal joke in the company was that a senior engineer was the same a junior engineer or worse. I had an a senior engineer who couldn't change a string to another string in a code base. I don't even exaggerate I had to prove to my manager that he had no skills at all.

kitten_mittens_ 5y ago
As an American, I think that arguments about outsourcing that boil down to ability being all over the map don't feel right. Plenty of American engineers are all over the map. The biggest difference is distance, the time zone split, and cultural differences. I say this as someone that thinks limiting insourcing raises wages in the short term.
- kjuulh 5y ago
  I agree, I've had plenty of colleagues whom really didn't excel in the field. I think for the most part with outsourcing as you say is the distance, but for me/us (team) it was cultural, we simply couldn't mix with the foreign part of the team. I've left the country out, because I don't want to point fingers, but it was simply about face, and no asking questions at all. To be honest, there were mistakes on both fronts, I feel like we as the engineering team in the country where the company was located was, felt entitled to our way of working, while the other party had to fit in.
- cpeterso 5y ago
  Plus companies are outsourcing to reduce development costs. They get what they pay for, regardless of country.
thelittleone 5y ago
Not all execs care about "sustainable". They often have quarterly targets. To get some wins an exec could outsource, complete a bunch of tasks, then move to another company on a higher paid job, leaving former company to mop up. See this a lot with sales also. Transactional client relationships promising the world to get the deal and hit commission multipliers. They then move to another company with "success stories" leaving implementation teams to reset client expectations.

newuser94303 5y ago

There are bad engineers in the US as well as India. If you are paying $9 per hour, you are not getting the good Indian engineers. It is like going to a local McDonalds and hiring programmers. You deserve what you get.

ddjsn111 5y ago
dboreham 5y ago
Where I live in the middle of the usa McDonald's is paying $19/hr
austincheney 5y ago
That’s becoming hard to fathom when you can now get hired as a React developer with about 3 years experience for $170k without any understanding of how things work under the hood.
- woodruffw 5y ago
  Why would a React developer (and hopefully SME, if they've been at it for 3 years) need to know anything "under the hood" to earn their value? I work on compilers for a living, but I couldn't build an IC if you put me in front of an EUV machine. Does that mean that I don't have an understanding of how things work "under the hood"?
  - austincheney 5y ago
    “Under the hood” is fancy talk for doing more than put text on screen. Whether or not that’s needed depends on the employer. BTW, nobody is an expert within just 3 years. That is a key indicator of a Dunning-Kruger moment that becomes clear by drilling down just a bit into key performance indicators from research and experience.
    - woodruffw 5y ago

faangiq 5y ago

“Yep, just fungible code monkeys.” -every company, even FAANG

gruez 5y ago
>even FAANG Why are they paying american programmers 6 figure salaries then?
- deleted 5y ago
  [deleted]
- MattGaiser 5y ago
  We are valuable commodities. Something being expensive doesn't mean that it is valued on an individual basis. An ounce of gold is expensive, but extremely fungible.
- Sebguer 5y ago
  Because they extract more value than that from them. Doesn't make them less fungible.
  - gruez 5y ago
    Even if they can afford it, it still defies rational behavior to pay 6 figures for an american programmer when they're "fungible" with indian programmers making $9/hr. I'm sure a FAANG programmer "extracts more value" from a $20 cup of coffee than it costs. That doesn't mean it's not highly unusual/irrational behavior, considering that other coffee shops charge a few bucks for a cup.
- ushakov 5y ago
  because they drive millions of $ in profits and because all the great engineers are already employed somewhere else
- Nicksil 5y ago
  >Why are they paying american programmers 6 figure salaries then? Because those companies are making an outsized profit from the relatively new and unregulated online advertising/marketing industry.

DantesKite 5y ago

That seems like a bad decision from a security perspective too. I can imagine cyberattacks meticulously crashing airplanes during a time of war.

cletus 5y ago

This is what happens when Boeing is run by accountants and MBAs instead of engineers (as it once was). Short term financial "wins" like outsourcing eventually make Boeing unrecognizable from the industrial behemoth that cranked out 747s in Everett. What these finance and business people fail to understand (or simply don't care about) is that every point of outsourcing becomes a point of friction. You now need to negotiate specs, deliverables, timetables and payments. There are often perverse incentives that reward change orders. You often get cost blowouts from "cost plus" pricing.

Victerius 5y ago
If I started my own aerospace company I would try to put in a legally binding rule that the majority of management and the executive team has to be engineers. But I don't know if an executive team can tie its successors hands like that. Unless the majority of voting shares remain in the hands of the family of the founder and successive generations commit to keep the company run by professional engineers. That, or a federal law.
- dboreham 5y ago
  The medical business works this way, fwiw.
  - Sebguer 5y ago
    Has that actually made things better?
  - mbreese 5y ago
    What medical business is like this? You don't need to be a doctor to own or run a hospital or pharma company. The legal business is like this however. I'm fairly certain that, offices must be owned by lawyers. The thought being that if a law office was owned by a non-lawyer you'd have a conflict of interest if your boss told you to do something illegal or unethical.
- pinewurst 5y ago
  How would that make a difference? Both the odious Harry Stonecipher at Boeing and the even more odious Jack Welch started as technical people.

MattGaiser 5y ago

> making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace -- notably India. I have always wondered about the consequences of vast numbers of devs working in fields they barely understand (myself included) and the relatively rapid churn of those devs. That being said, in most places I have worked, developers don't get a lot of influence in big picture decisions. If Boeing is like that, it would not have mattered.

okl 5y ago

There's good/bad engineers everywhere. I have worked with students that earned less than $9/h and produced better quality work results than highly payed senior engineers. Nationality is also not a reason for high quality/sloppy work. However, I can accept that expecting to get the same quality work for a fraction of the cost should be viewed skeptically.

Victerius 5y ago
> Nationality is also not a reason of high quality/sloppy work. In some ways it can be, though. Poor countries tend to be poor for a myriad of reasons, but among these reasons should be the lack of a quality education system. Wealthy countries tend to have excellent school systems and universities, and a better trained student body, than poor countries. If poor countries had the education systems of wealthy countries, they wouldn't be poor, but the reason that can't sustain a system like that is precisely because they are poor.
- Sebguer 5y ago
  I mean, India was "poor" because it had 90 years of having everything of value stolen by the British Empire.
  - Victerius 5y ago
    That's one point of view, yes. Like I said, there are a lot of reasons why poor countries became poor, or failed to industrialize. Personally I don't think India would be America 2.0 (or China 2.0) had the Brits never colonized it. The partition of India happened 75 years ago, however. The British Empire is no longer a valid excuse.
    - yunohn 5y ago
      > The partition of India happened 75 years ago, however. The British Empire is no longer a valid excuse. This particular argument is usually made without understanding history. India, the country, did not exist before British enslavement & colonisation. India’s diversity and size are comparable to the entire EU, with double the population. I don’t see why 75 years is /too long/. It's quite tiring to hear such naïve takes. I would really recommend reading some books…

jka 5y ago

Perhaps a dumb question, but is there a definitive connection between the outsourcing mentioned here and the MCAS[1] system that has caused the 737-MAX tragedies? (I have searched the web to try to confirm a link; there are some articles[2][3][4] that mention both MCAS and the outsourcing, but I haven't yet seen anything to indicate that outsourcing was used for development of MCAS-related software) [1] - https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Augmentation_System https://en.wikipedia.org/wiki/Maneuvering_Characteristics_Au... [2] - https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-... [3] - https://pulse.icdm.com.my/resource/boeing-a-plane-wreck/ https://pulse.icdm.com.my/resource/boeing-a-plane-wreck/ [4] - https://simpat.tech/did-boeings-software-development-outsourcing-contribute-to-its-737-max-8-failures/ https://simpat.tech/did-boeings-software-development-outsour...

imglorp 5y ago
The software was not truly to blame for the deaths. Management was fully aware that a redundant AOA sensor was desireable: they charged extra for it. Management also demanded a product that did not require pilot training. Then there was the whole certification malfeasance. The software was one piece of an impossible configuration.
AnimalMuppet 5y ago
$9/hour devs in India are less likely to say "Only listening to one angle-of-attack sensor is stupid; you need to redesign this." They are especially less likely to keep pushing until they get management to listen. I mean, sure, the designers should have known that, too. But by outsourcing the software development, they missed a chance to have the bad design questioned and corrected.
- bdavis__ 5y ago
  in general, a software dev anywhere might not realize the flaw. but an systems engineer or other aircraft and flight controls engineer would notice it. and they should have, when the requirements were written. and again when each requirement was verified prior to flight.

VWWHFSfQ 5y ago

> the iconic American planemaker and its subcontractors have relied on temporary workers making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace -- notably India This is bullshit. At least, for anything important. > Bloomberg | Quint is a multiplatform, Indian business and financial news company. So this is defintely bullshit. Edit: I don't even know what the implication they're trying to make is. Are they saying that the airplanes crashed because some incompetent Indians wrote the software?

deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
dralley 5y ago
>I don't even know what the implication they're trying to make is. Are they saying that the airplanes crashed because some incompetent Indians wrote the software? The implication is that Boeing cares so little about the quality of the product that they are willing to pay $9 an hour for it. Anyway, regardless of the merits of the offshore team, Boeing is still responsible for any flaws in their work product. You can't just point fingers at your subcontractors when you're the one who hired them and then signed off on their work.

harry8 5y ago

As a CEO, if your team sets up a company to succeed, the payoff is likely not for the next decade. You're setting up some other executive's payday. If you burn the co's reputation to get your payday, probably nobody really notices and is likely enough for there to be no real consequences to management until after you leave. The problem you paid yourself with is someone else's to solve. Anyone suggesting there's an easy answer to that is likely full of it. How expert, competent and interested are the board of directors? How do you know? Some people look and sound plausible and convincing but aren't. Credentials are not much guide, all the board candidates have them including the total self-serving Muppets. Even noting who was great 10 years ago might not care much anymore. Boeing stock price is up more than 1% on the last day of trading. Not as much as Lockheed and Raytheon.

nonamechicken 5y ago

[I may be wrong. Hence deleting.]

okl 5y ago
ITAR pertains to export/import. I'm not aware of person-specific clearances. In any case it would depend on the specific component or task being worked on.
- deleted 5y ago
  [deleted]
- bdavis__ 5y ago
  there are not person-specific clearances, but there are person-specific non-clearances; e.g. there are lists of people and companies that you cannot do business with because of them being non-compliant with ITAR.
kevin_thibedeau 5y ago
ITAR doesn't have a clearance process with extensive background investigations. It is the lowest level of control for sensitive development operations. Non-citizens can work on it but there are paperwork hurdles that some employers don't want to bother with. It does also need to be conducted in US territory.

user_7832 5y ago

Something I don't think is mentioned here is that initially the Boeing designers thought that both sensors were being used for the data but the code written only took it off a single sensor. Whether it was not properly communicated to the coders or what, it is not clear, but someone at Boeing had wanted multiple sensors. (I don't remember the exact source, I'd written a report of the Max-8 crashes and this was part of what I found. If you're interested in a generally excellent source for the accident/design flaws, the (latter) US Department of Transport report is excellent. I can also share my work if someone is curious, though its quality and depth is of course much lesser.)

zarmin 5y ago
Someone should invent a way to check whether software is working correctly. Call it "writing checks" or "check driven development" or something.
- radu_floricica 5y ago
  If they don't get the requirements right, they'll just write the tests wrong as well. You have the same problems, just with a bunch of extra green oks.
  - deleted 5y ago
    [deleted]

fredgrott 5y ago

Keep in mind this is the tip of the iceberg as the FAA had outsourced safety to Boeing via Boeing employees also being FAA employees. Very hard to complain against your financial master unless you fly on that plane make daily as a passenger,

imglorp 5y ago

Even if they were a stellar team, the problem with offshoring anything is always culture. The offshore team is isolated from the company, all of its storied century of knowledge (in this case), about engineering, materials, machining, quality, certification, aerospace, systems, it goes on and on. It's inconceivable to spawn any human safety article outside of that culture into a vacuum behind a contract written by lay business people and PM's.

rjzzleep 5y ago
That’s not really the culture problem here. The problem is that if they outsourced it to a team at $9 an hour someone at Boeing thought they made a killing. While the offshore team gets $9 an hour, the engineer itself gets a fraction of that. And those engineers are very well aware that you didn’t choose them because you think they’re amazing. They know you chose them because you don’t value engineering. So regardless of whether they are good or not they’ll do the minimum work possible. I remember a bigger automation softwares CTO telling me about a bigger enterprise that offshored their database change management process. The process was defined as putting a word file put into a jira issue. At the end of the approval process some person in India had to paste it into the database shell. If word auto correct messed it up the whole process gets marked as failed and it started from 0. That person pasting into the database has no interest whatsoever on making any recommendations even if they could.
- qchris 5y ago
  Not to take the easy pun here, but by making this decision and ones like it, someone at Boeing did make a killing. Or 346 killings, more precisely. The same culture and internal processes that allowed this to occur are the exact same ones that let the MCAS design flaws get through, and I will forever be furious that Boeing as an organization was not ripped into parts for that failure.
- laurencerowe 5y ago
  I wonder if that cultural problem stemmed from Boeing’s 2001 relocation of their headquarters to Chicago, a decision made explicitly to separate their financial infrastructure from their existing operations.

HL33tibCe7 5y ago

Pay peanuts, get monkeys

victor106 5y ago

About 20-15 years ago we found offshoring teams had terrible communication and software skills. In the past 7-5 years the communication and techskills for a little better but it’s still no where near close to an onshore resource. However occasionally i find an offshore resource who is exceptional.

Consultant32452 5y ago
The last mega-corp I worked for had this program where they primarily hired offshore workers, and they offered the really good ones the chance to move to the US on work visas. I made friends with one of them and he told me there is rampant fraud on the offshore teams. There's 2-3 people doing all the work, and they let other people claim it, because that's just the culture somehow. The off-shore managers are in on it and sometimes are one of the few doing the work. He claimed this was the norm at multiple big-name contracting firms he'd worked for. I have no way of confirm any of this obviously, but it did offer an explanation for some of the experiences I had.
- deleted 5y ago
  [deleted]
- mariuolo 5y ago
  > There's 2-3 people doing all the work, and they let other people claim it, because that's just the culture somehow. Could it be a question of caste hierarchy if in India?
Karupan 5y ago
> offshore resource In these times when “master” is renamed to “main”, I still find it strange that calling human beings “resources” is tolerated in the industry.
- parineum 5y ago
  That could easily be a reference to a team. Honestly, when outsourcing, you really don't know who or how many people are doing the work (and that's kinda the point).

blunte 5y ago

This is just modern capitalism. If a few dozen lives are lost, it's not significant compared to the quarterly earnings per share numbers.

Simplicitas 5y ago

The https://www.rottentomatoes.com/m/downfall_the_case_against_boeing https://www.rottentomatoes.com/m/downfall_the_case_against_b... documentary is a good take on this

csours 5y ago

Who did the design, architecture and tests? The tests seem more important to me.

sofixa 5y ago
What tests? They hid the MCAS from regulators, so maybe they took the same approach internally - out of sight, out of mind. Why is still there nobody in prison over this ?

known 5y ago

nunez 5y ago

> Still, for the 787, HCL gave Boeing a remarkable price – free, according to Sam Swaro, an associate vice president who pitched HCL’s services at a San Diego conference sponsored by Avionics International magazine in June. He said the company took no up-front payments on the 787 and only started collecting payments based on sales years later, an “innovative business model” he offered to extend to others in the industry. Lol at the industry thinking that deferred bundles and services discounts are revolutionary

testemailfordg2 5y ago

The last line should be the title of the article. “Senior company leadership,” the statement added, “was not involved in the review.”

atoav 5y ago

Maybe that is a little it harsh of an judgement, but IMO the person responsible for this decision should be in jail.

deleted 5y ago

[deleted]

svilen_dobrev 5y ago

mmh. Not all $9-per-hour engineers are the same. Been there done that. And some of these $9 may buy them more of their-own bread-or-house than the $$RICH ones can with their 6-digit $s. Thus being happier. Envy? Competition? And of course it's always them outside teams that are to blame, while the actual elephant isn't. On the other side, when you outsource everything, .. you're what, just a web-site? oh wait, not even that.. https://berthub.eu/articles/posts/how-tech-loses-out/ https://berthub.eu/articles/posts/how-tech-loses-out/

itsdrewmiller 5y ago

>Boeing said the company did not rely on engineers from HCL and Cyient for the Maneuvering Characteristics Augmentation System, which has been linked to the Lion Air crash last October and the Ethiopian Airlines disaster in March. Feels like most of the commenters missed this from the article.

sceean 5y ago

$9 is probably a rate for a software tester. Even in the US, you can get a software tester for not much more than $9 an hour, at least that was the case in 2019. Probably, not true any more with the current rampant inflation. There is a vast salary difference between software engineer and software tester. Let's not sensationalize things unnecessarily.

mbrodersen 5y ago

The problem with profit optimising companies like Boing is that they don’t know when to stop. They will keep squeezing every last drop of blood out of their products, and their employees, until the business fail, or a competitor gets competitive enough. Especially when they hire a “General Manager” who’s only expertise is to force a 10% cost reduction on the business every year. Until the business is a dead corpse. But by then of course the CEO has already skipped ship and started the cycle again with another company.

weare138 5y ago

We seriously need to start charging the people who make these decisions at companies like Boeing for the crimes they committed like we would anyone else. Colorado sentenced a truck driver for a multiple fatality accident that killed 4 to over 100 years in prison because the driver was deemed negligent. But if you're a corporation like Boeing who's gross negligence kills hundreds the company just gets an inconsequential fine. Why do we have to play by the rules but corporations don't? Every Boeing executive who was involved in this should be in prison right now.