6 ms·
Blame for 'switch from hell' falls heavily on one GM engineer
- seren 12y agoWhile interesting, I find the narrative pinpointing the issue on a single engineer behavior a bit too convenient to be believable.
- fleitz 12y agoWhy? Have you never fixed a bug in your own code, or not followed every single one of your companies' procedures?
- watwut 12y agoOf course people do mistakes and fail to follow procedures. That is why the company should have processes to catch dangerous mistakes and omissions in place. If one low level guy can cause such huge problems, then there is something very wrong with the way whole process works.
- seren 12y agoI am a SW engineer currently working in a regulated industry, and there is no procedure in place for me to fix something quickly behind everybody's back. The code should be at least reviewed, integrated and tested by 3 independent people. And the issue and the resolution action would have been reviewed/signed off at the system level as well. So it would be very unlikely that I could hide something for years without suspicion. This is in a way cumbersome and annoying but it is understandable why it is put in place this way. I used to work in consumer electronics, where I could commit whatever I wanted without any kind of supervision or checks, but this is generally not something you can do when it is a matter of life and death.
- TheSpiceIsLife 12y agoThat being the case, what is your opinion of what is going on here?
- seren 12y agoIt is possible a single engineer was negligent or even malicious, but I would rather believe the whole organization was at the very least encouraging a culture of "do whatever to be ready at the end of the quarter and I don't want any detail" (orally of course) You can't be reckless in a place where everyone is paying really close attention to safety and regulations. On the other hand if you are in environment where everyone is pretty lenient, it is much easier to slip between the checks. But, from a journalistic point of view, even if the story is not fabricated, it is a better story to have a villain, than trying to convey to the readership how complex organizations (mis)behave.
- calinet6 12y agoIn the situation you describe above, it would be nearly impossible for the defect to appear in the first place. You wouldn't need to hide anything for years, because it would never need to be fixed. Furthermore, when trust is placed in the system and blame not given to the individual, then the momentum is to improve the final quality, not to hide a defect. The problem would have been fixed as soon as it was realized without a need to hide, fear, or cover up. No one would be blamed and no one held responsible, and the problem may never have escalated. The system was poor, therefore the quality was poor. This was not an individual's fault. Quality is a systems problem, plain and simple.
- dccoolgai 12y agoYeah, my thoughts exactly... I wonder if the project managers who pressured him to cut corners to save time are also getting fired...
- forgottenpass 12y agoAs an engineer that works on safety critical stuff I see it as GM finding a fall guy, and/or having laughable-if-it-wasn't-so-tragic controls in place. Edit: The bbc report says 15 were fired, half were senior legal and engineering executives. To me the issue stops being DeGiorgio's fuckup, but how so many other folks were incentivized to roll with the fuckup. The fuckup is bad, but the the environment that permitted it and later didn't correct it is the real danger. But don't worry, GM assures us that the Valukas report doesn't find a conspiracy or cover-up.
- jusben1369 12y agoI think it can be both. It can be a single guy who is responsible for the switch being deliberately misleading through acts and omissions. Then it can be the chain of command above and around him as well. To have GM yesterday fire 15 people - including many senior execs - and then today accuse them of copping out and finding a fall guy and planting the media story doesn't make a lot of sense. It's possible one of the execs who was fired leaked this if it was in fact leaked/planted.
- TheSpiceIsLife 12y agoAll of the above, always. I highly doubt the legal / regulatory staff and the HR and PR staff in large corporations are stupid, you can bet they've covered every angle. It's the scapegoat (the single engineer), it's the staff around him, it's the intentionally leaked email, it's the exec who's a plant in the fired group, who will gave a job next week with some other auto manufacturer. Is there a term 'Public Relations Theatre'? If not, I coin it now. Everyone's got everyone else's back, and hands in each others pockets. Maybe someone will appear to take the fall, but behind the scenes that person will be looked after. Call me cynical, or maybe realistic?
- jusben1369 12y agoInteresting study on cynical people and dementia FYI: http://www.telegraph.co.uk/science/science-news/10861276/Cynics-three-times-more-likely-to-suffer-from-dementia.html http://www.telegraph.co.uk/science/science-news/10861276/Cyn...
- leoedin 12y agoDoes anyone with experience in the automotive industry have any thoughts on this? What sort of culture would lead to the selection of an inappropriate part in the first place? Would it be cost driven? Does this idea that an individual engineer's component selection decision would play such a big part ring true with other commentators experiences?
- walshemj 12y agoWell when I did my mech eng college course one lecturer mentioned that a UK car manufacturer used the wrong type of component to fix the steering wheel to the shaft. Resulting in it being possible for the steering wheel to detach! whilst driving.
- serf 12y agoIt's likely just an error in the design process that turns into a problem that is cost driven, but from the perspective that the problem was found after tooling had been acquired and production started and would require a lengthy/costly recall procedure.
- fleitz 12y agoNot an automotive engineer, however, it's the same culture that produces the plethora of bugs in software. It's a human factors issue. It's no different than an engineer selection of a single line of code, if it's in the wrong place, crashy crashy. Here's a much better article: http://www.seattlepi.com/news/us/article/Engineer-s-switch-from-hell-began-GM-recall-woes-5533018.php http://www.seattlepi.com/news/us/article/Engineer-s-switch-f... Perhaps the engineer was distracted by an underground boxing ring.
- leoedin 12y agoFor reference, I work in the space industry, where piles of paperwork are produced to justify any component selection decision. I can't imagine a situation where anything, even down to individual capacitors, would be selected without at least 2 people having some input to that selection. Our qualification processes are absolutely massive and take decades though - not particularly suitable for the automotive industry. I don't think that this can be compared directly to the kind of adhoc software development that is routine in many places. Engineering processes have developed over decades to eliminate human error and very, very rarely allow single unjustified decisions made by a single person to propegate through to a finished product. I'm surprised that in a world where individual car platforms cost billions of dollars to develop there was never a point at which the switch selection was challenged. I haven't read the full report though - perhaps it's not quite as the news article suggests. It would be interesting to hear the point of view of someone in the automotive industry (if any of them are HN readers?)
- deleted 12y ago[deleted]
- pjmorris 12y ago"A loss of X dollars is always the responsibility of an executive whose financial responsibility exceeds X dollars." - Gerald Weinberg's 'First Principle of Financial Management' and 'Second Rule of Failure Prevention' [1] [1] 'First-Order Measurement', Quality Software Management, Volume 2, Gerald Weinberg, Dorset House Publishing, 1993
- puppetmaster3 12y agoBlame the engineer. Hmm. Lets see. Will I ever buy Government Motors in my life? That would be a no.
- jusben1369 12y agoKind of disappointing to see "Government Motors" on Hacker News.
- shiven 12y agoGiant Monoliths are (understandably) not looked upon kindly on HN. Move fast & Break things is the mantra here. ;-)
- TheSpiceIsLife 12y agoWell they've got the 'break things' part covered hey ;)
- jusben1369 12y agoI hear you. But "Government Motors" is a clever marketing phrase right along with "Death Panels" that became a catch cry for a political movement high on rhetoric and low on intellectual rigor. Thus my disappointment that someone on HN might also inhabit that camp.
- shiven 12y agoThus my disappointment that someone on HN might also inhabit that camp. Yeah, I hear your disappointment. But, HN is not (and doubt ever was) a homogenous community. It would be pretty boring and even more of an echo chamber if it were. Roses and Dandelions, one more than the other ;-), all form the HN crowd! Sure, dissing redditization is a part of being a responsible HN citizen, but let's not throw the baby out with the bath water! (now get off my nice lawn all you pesky whippersnappers) :P
- SilkRoadie 12y agoIt doesn't matter what the engineer did. The fact that the car company had so few controls that a single engineer could approve something faulty then change the specs of it in future models without anyone noticing is worrying to say the least. It is a bit like banking and these "rogue traders" who spend a billion dollars. They are at fault certainly BUT more so it is the banks fault for having such lax measures that they had access to this much money.
- oldmanjay 12y agoit definitely matters what the engineerr did. your point of view is equivalent to blaming the police for crime.
- keyme 12y agoMore like blaming lack of policing for crime...
- oldmanjay 12y agoyou're incorrect in this case. according to the article, this blameless paragon of virtue deliberately hid his actions.
- jusben1369 12y agoRight. They're called checks and balances for a reason. It's more than fair for GM to assume that it's paid employees follow procedures to ensure that this type of thing doesn't happen. So to say it doesn't matter what the engineer did is a poor choice of words. Given the stakes though it's also incumbent upon GM to ensure that there are other procedures and processes in place should an employee not do the right thing. It sounds like multiple people over the course of time (failure at a media launch?!) were suspicious about this switch but no one escalated the issue.
- spindritf 12y agoThey had checks in place for errors, not for being deliberately misled by one of their own employees. If they had checks against that, we'd be reading a blog about procedures and corporate culture in GM suffocating engineering.
- funinobu 12y agoThis news story talks about the proximate cause of the failure and how the cover-up started. But it doesn't say how the cover-up propagated. 14 other people were fired. But we don't know why or whether that was the extent of either covering up or incompetently failing to to uncover either the problem or the cover-up.
- dankohn1 12y agoI think modern testing culture and source repositories can help avoid this kind of negligence in software projects today. I'm hardly Panglossian enough to suggest that it's common, but if you have an engineering culture where specifications and especially regressions are encoded in tests, it starts becoming clear when your software project is or is not satisfying the requirements. Decisions like removing a test because it fails intermittently are documented by source control, and can be reviewed by peers and management.
- xyzzy123 12y agoI have to wonder how much the focus on the switch is papering over other problems. The fact that a non-essential mechanical component can cause accidents like this seems fragile. When the key pops out, the engine stalls, power steering goes out and the airbags do not inflate. Where is the "defense in depth"? A robust system should really require two things to go wrong. Is it that "key in ignition" is a critical part of the vehicle's state machine?
- hga 12y agoPretty much, I think. Not being able to decisively turn off the engine has also killed people, remember the Toyota issue not long ago? We think that in general the drivers panicked and floored the accelerator thinking they were pressing the brakes, but the complexity/difference of turning off the engine with a button---as I recall it had to be held down for something more than a second---vs. turning a key all the way counterclockwise and ultimately pulling it out could have resolved the user errors short of fatal crashes. Ditto the theory that some or many of these were caused by the accelerator getting stuck on a floor mat. Or an embedded computer and/or components attached to it failing in a bad way. Compare to the big red buttons on some computers and hopefully in all big machine rooms: there are times, when lives are threatened, that you want an easy way to immediately turn everything off. Even if you add a molly-guard in front: https://en.wikipedia.org/wiki/Big_red_button#Molly-guard https://en.wikipedia.org/wiki/Big_red_button#Molly-guard
- deserted 12y agoIf you're driving down the freeway, putting the gear selection lever to neutral would be a much better choice than killing the engine and taking out the key.
- hga 12y agoIf you're so panicked you're flooring the accelerator thinking it's the brake, you might not think of that, and since you've not actually depressed the brake pedal, it wouldn't shift anyway, would it? If you're truly standing on the brake, then it ought to work, but you might not think of it. I'll bet a lot of people don't usually think of any gears but park, normal forward and reverse.
- dmritard96 12y agoIts interesting how much attention the general public has given to this yet for only a handful of deaths. For some perspective, 34,000 people died in the US during 2012 due to motor vehicles[1]. The big automakers are ultimately causing way more deaths by not adopting autonomous vehicle technology sooner if you ask me. And they will continue to do so because they realize that autonomous fleets make more sense than every consumer buying a car or two hence autonomous cars mean their business is in jeopardy/going to shrink. We might also look into deaths associated with air quality/pollution and the number of deaths resulting from it. Its always going to be hard to pin down what number of deaths correlates to how many cars driving some number of miles, but I would put serious money on lower emission vehicles significantly reducing deaths resulting from air pollution. Detroit's reluctance to adopt cleaner technology and its obvious love affair with oil is killing many more than their incompetent switch DREs. Ultimately, even the switch design is a similar phenomena. While many automakers started to get away from keyed ignition switches altogether, GM has continued to use a super cheap part that's obviously a design from the past. You can't tell me the radio on my grandmothers key ring cost versus their switch is worth 13 lives. Certainly, don't mean to trivialize those that have been killed by this, just want to point out that Detroit is so slow to change that ultimately its costing us way more lives than just those killed by this switch problem. 1 http://en.wikipedia.org/wiki/List_of_motor_vehicle_deaths_in_U.S._by_year http://en.wikipedia.org/wiki/List_of_motor_vehicle_deaths_in...
- chris_wot 12y agoI disagree. This is particularly interesting because there was an active coverup of the issue. This could have been resolved with almost NO deaths, had the actions of this one engineer not been taken.
- dmritard96 12y agoI don't mean to troll. I agree its interesting because there is a cover up and an engineer who could have saved 13 lives and didn't. My main point is that big automakers are doing this same think, cover up and all on a grand scale but because you have to look at it from 10,000 feet to see it, it doesn't get the same type of criticism. you said you disagree, there were like six points I tried to make, was there a particular set you disagreed with?