14 ms·

How a lawyer, mechanic, and engineer blew open an auto scandal

269 points by waffle_ss 12y ago

latch 12y ago
Seems to be a lot of things at play, but what surprised me was the ability to issue a modified part without a change to the model number. Seems like something that could be enforced automatically? Like a version control system? Are we running out of numbers?
- bagels 12y ago
  Well, it's either a way to perform a coverup, or a cost savings by not needing to make new tooling with the new part number on it.
  - mirkules 12y ago
    A new part number doesn't necessarily mean new tooling. However, you are correct in saying that it is a way to cut cost by not issuing a new part number. A new part number means: - Entirely new release/approval cycle. That means going through design, release and QA stages (QA is extremely expensive in automotive industry). Remember auto industry is not the software industry, there is no "agile" - waterfall is king because once the product is released, it is infinitely more expensive to change it - Recalling old part numbers and shipping new ones - Even the cost of printing new labels and making new packing materials can be non-insignificant That said, a change in part equals a new part number, period. So, this was very likely a coverup AND an improper cost-reduction exercise.
    - bagels 12y ago
      I agree with you. I did happen to notice from pictures that these particular parts have the part number cast on to them, so new tooling may be required.
      - mirkules 12y ago
        Ah yeah, good observation.
  - frankydp 12y ago
    Doesnt this mean that the part manufacturer had to be in on the process and that it probably wasnt the first time?
- Gargoyle888 12y ago
  I was assume that this was part of a structured bill of materials. Changing the rev level should be sufficient to identify the change and would not entail changing the BOM. I'm going to make a strong guess that the rev level was incremented (the article doesn't mention rev levels). If so, I don't see anything wrong with using the same P/N. It's not like that part number exists elsewhere in the vehicle.
scotty79 12y ago
Absolutely horrified. > The Cobalt’s engine shut off and the lights inside and outside the car dimmed. Melton hit the brakes, but no power from the engine meant no anti-lock brakes and no power steering. I wasn't aware that anti-lock brakes (is that ABS?) is not a purely mechanical system. Also, was power steering hydraulic or electric? Shouldn't it be run of battery for short period of time? > While Brooke’s lap belt glued her waist to the seat, her shoulder harness went slack the instant the engine shut off. Now that's just criminally stupid. EDIT: I guess I need to read through to the end to the article but that's a hard read for me for some reason.
- mirkules 12y ago
  ABS is indeed the Anti-lock Braking System and I think it depends on sensors to determine the skidding amount and on software to determine the interval at which the brakes are applied and let go. So, the ABS is run off electricity and not a purely mechanical system. However, brakes themselves (without the "anti-lock" part) are purely mechanical/hydraulic, but since the brakes locked up, it didn't matter. I'm not entirely sure about electric power steering systems and how they work. ABS: https://en.wikipedia.org/wiki/Anti-lock_braking_system https://en.wikipedia.org/wiki/Anti-lock_braking_system EPS: http://www.caranddriver.com/features/electric-vs-hydraulic-steering-a-comprehensive-comparison-test-feature http://www.caranddriver.com/features/electric-vs-hydraulic-s... https://en.wikipedia.org/wiki/Electric_power_steering https://en.wikipedia.org/wiki/Electric_power_steering
  - toomuchtodo 12y ago
    ABS depends on power not only for wheel speed measurement, but also for power to drive the brake piston.
  - bonestamp2 12y ago
    > I'm not entirely sure about electric power steering systems and how they work. It comes in a variety of forms, but it's usually an electric motor that adds torque as you turn the steering wheel to make it easier to turn.
concernedhner 12y ago
At some level, I am horrified at what was done with prior knowledge by this institution. At another, this incident makes me wonder how this ties into the diffused nature of responsibility within corporations. Was the diffusion of responsibility within these corporations was so great that they could perpetuate these acts without any one individual bearing the moral responsibility of what happened? If so, then these corporations inadvertently have created an entity that is independent of the individual stakeholders and their moral pangs. That is a very frightening prospect...
- csours 12y ago
  It is not only responsibility but also knowledge. The ignition switch engineer may not be aware of what exactly it turns on and off; even if that engineer is, other engineers are not. (See page 1 of the GM Valukas report) Analogy: does your DBA know about flushing to disk?
  - Too 12y ago
    Very good point. In huge systems such as a car everybody can not know everything. But in this case, if the ignition switch team does not know how their component affects everything about the car or how important it is the responsibility should move up to whoever writes the specs on the ignition switch. First of all for designing a system with a single point of failure in the first place, secondly for not loudly notifying ignition switch team that their component is vitally important and a single point of failure. As i also said in a post above, way too much attention is given to the faulty switch when this is actually a much larger system design error.
PhantomGremlin 12y ago
Along with the other posters here, I am also "horrified". But isn't this more-or-less the same at most/all other companies? They become old and sclerotic and hidebound and resistant to change. Aren't most old companies guilty of the equivalent of the "GM nod"? a staple of GM managers, nodding in agreement at steps that should be taken then doing nothing Maybe companies like GM eventually run their course, and should be allowed to die. Maybe we were wrong to bail them out in 2009? Would the country be better off if a new company could take GM's place? But, of course, GM couldn't be replaced "as is". It could only be replaced by something more in tune with the times. It's only a {relatively speaking} tiny company, but Elon Musk and Telsa could be "the next GM". Until Tesla eventually runs its course!
- djloche 12y ago
  Not one company should be bailed out. If a business fails, those who think they can run that sort of company better should step in and buy the assets and give it a go. In the many or most cases where companies have built what has become core infrastructure, regulations already exist for orderly shutdown and sale of assets without disruption of service. If a company becomes 'too big to fail', this means regulatory capture has already occurred and you have far greater problems to resolve than mere bankruptcy.
  - kefka 12y ago
    I disagree. Bail out big companies who fail miserably. THEN do the next step: Take ownership of said company. If We're paying for it, We should control it.
- rqebmm 12y ago
  >Would the country be better off if a new company could take GM's place? Theoretically this is what happened with the bailout. Everyone made fun of Mary Barra for her "this all happened at the 'old GM', so the 'new GM' can't be responsible" but there's a reason it worked. GM was replaced by a new company: GM.
csours 12y ago
All sympathy to the families involved; what makes this the worst auto scandal in history? Are there numbers for this or is this based on public opinion? Disclosure: I have been a GM employee since 2013
- viseztrance 12y ago
  I've read the article and this blew my mind: "[...] if you took all of the cars that GM has recalled this year and lined them up bumper to bumper, you’d end up with a line that would wrap around the earth four times." That's 30 million cars.
  - kefka 12y ago
    And if you took all the software that has been recalled, it would line up to nothing... Because software doesn't have dimension. And aside radiological software incidents (Therac 25 and similar), software errors usually don't kill people. Engineering design failures usually do.
- wglb 12y ago
  Can you list some of the others?
  - csours 12y ago
    Bronco, Explorer, and F150 rollovers -fixed by adding stability and Tire pressure monitors Toyota's gas pedal and floor mat recall - halfway similar to this issue in that it involved very poor affordance (drivers could not immediately tell what was wrong, they thought they were pressing the break, but the gas was pressed as well) GM Streetcar conspiracy Cars from the Big 3 used to literally fall apart in the 70's due to poor assembly. (Welcome Toyota) Ford Pintos in the 1970s had badly under-designed gas tanks that ruptured and caused fires in rear-end collisions. Media put the death toll in the "hundreds", NHTSA had a death toll of 27. The scandal of the Big 3 killing smaller car companies like the Tucker 48. Incredibly stupid design decisions like planned obsolescence. GMs policy of "build a bunch of cars and make people buy them" from the 1970's to the early 2000's - responsible for peoples poor opinion of GM and competitors that followed the same mantra (It even bit Toyota in the mid 2000's)
ploxiln 12y ago
When I read this story, I thought of parallels to large complex software projects (of course). You could call a large software system that would take more effort fix than has been been spent to create it so far "totalled", like a badly crashed car. It happens (usually with government payroll systems it seems). GM seems to have "totalled" their entire lineup of cars designed and produced over the last 20 years - the quality of their organization's work has been so low that it's not really possible for GM to find and fix all the serious problems on all the cars they've made. They would have to dissect all the parts from a sampling of cars of each year, since they can't even trust that the same part number is really the same in different years! It's probably cheaper to make a couple new car models, really make sure they're not defective, and provide everyone who still has a GM car with a replacement car. Cheaper, but maybe still not possible. "Good enough" massive recalls will have to do.
lovelearning 12y ago
I feel the root cause here is the culture of setting hard deadlines by management. Getting a deadline moved even for logical reasons involves pushing against massive internal red tape. Missing a deadline invites greater punishment from management than compromising on quality. Naturally, quality loses out. Happens all the time in our own industry, and sadly, this case shows that it happens even in safety critical industries. All engineers, regardless of industry, should be trained in how to negotiate with management. All managers, regardless of industry, should be trained to respect opinions of subordinates.
- amirmc 12y ago
  That almost sounds like you're absolving the engineers (as though they can do no wrong). Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). This is a complex issue and I do not agree with the (over-simplistic) reduction to 'management' and 'deadlines'.
  - lovelearning 12y ago
    Not absolving the engineers. The engineer in this case - and other cases - may well be guilty - both legally and morally - of mistakes. What I'm trying to understand is, what would make an experienced senior engineer prefer such shortcuts over doing the right thing. The article says, "Faced with a deadline, DeGiorgio replied: If increasing the torque will destroy the switch then do nothing. Maintain present course. Under no circumstances do we want to compromise the electrical performance of the switch.". I interpret it as the deadline being a root cause. At the heart of any complex issue lies one or more root cause(s). What were they here? What made him behave that way? Fudging the records is a consequence, not a root cause.
    - amirmc 12y ago
      You're taking four words from an article (not original sources) and deducing a root cause. In the absence of those four words -- which is the only time 'deadline' is mentioned -- would you draw the same conclusion? I do not. I could argue that fudging the records is a 'root cause' of why the investigation took longer. If you want to go reductio ad absurdum then really the root cause is ultimately human fallibility.
yason 12y ago
Still wondering why make physical ignition switches that actually mechanically cut the flow of electricity. For years, cars have been equipped with all kinds of 'convenience' electric components such as Bluetooth, touch panels, and on top of that all critical elements in the engine have been controlled by an ECU for decades -- yet still only some cars have electrically, not mechanically, controlled ignition. The car key doesn't even have to be wireless: you could still have the lock and ignition key as now if you really wanted but turning the key would only give a signal to the car's computer to wake up from sleep, authenticate the key, close the main circuits electrically to let the components start drawing the big current from the battery, and start booting sequence, self-tests etc. and eventually drive current to the starter motor to crank the engine up. That's a huge number of things to do, and it had best be left for the computer to handle. Pulling the key from the physical ignition switch is aking to starting and stopping your PC by pulling and inserting the plug from/to the wall socket. You'd better push the suspend button and let your laptop put itself to sleep and let it wake up by itself. It's also much easier to design a system that will handle startup and shutdown by itself, rather than a system that must survive opening and closing the main circuits at any time.
- toomuchtodo 12y ago
  Because its cheaper to put a $1-5 mechanical switch in than to put a smart key system. Yet again, cost is the driving factor. How many lawsuits would you need due to ignition switch failures before smart keys for all cars was cheaper? A lot.
- cnvogel 12y ago
  I think there's a change happening, some new cars can be started by button-press when the transponder (replacement for the key) is somewhere in the car (e.g. in your trousers' pocket). And then, of course there's the other aspect that in some cases you absolutely want to be able to cut power to your engine. Wasn't that exactly the case with Toyotas drive-by-wire throttles that apparently made people unable to stop their cars? Still: When you look at the electrical system for, e.g. a commercial airplane, you'll see a dozen different independent power-supply busses that can be fed from each other, different engines, or batteries. So that in case of power-loss in one or a few of them, not all safety critical systems black-out. I don't claim that such complexity is helpful or desirable in a car, but with the majority of drivers relying on computerized systems it probably makes sense to make power supplies more redundant. {probably that's already happening, and I'm just unaware of it?}
function_seven 12y ago
The ignition switch in question seems totally unremarkable. I wonder why it was redesigned in the first place? Why not use one of the many existing parts GM had at the time? Also—somewhat of a nitpick to an otherwise good article—the Bronco II is a completely different model from the one driven by Al Cowlings during the O.J. Simpson chase. That was the full-sized Bronco. When details like that are wrong, it usually makes me suspect the accuracy of other facts in the story (although in this case, the article seems otherwise accurate).
- spydum 12y ago
  I also wondered this since the story broke.. Why do they redesign core components like this, instead of just iterating on previous designs? I fight this every day in the enterprise space. My only conclusion is ignorance: sharing previous design or infrastructure knowledge is hard (or at least, not prioritized). So folks end up recreating the proverbial wheel.
  - kbart 12y ago
    I guess that might be due to parts' supply and logistics problems. When production counts in hundreds of thousands and millions, getting all parts together and in time is quite a challenge.
markvdb 12y ago
Waffle_ss , change your user name.
- markvdb 12y ago
  I should have added some historical context to this request for those unaware of it. It's hard not to see the user name Waffle_SS as an unnecessarily hurtful trivialisation of the Waffen-SS, the Nazi paramilitary force. See https://en.wikipedia.org/wiki/Waffen-SS https://en.wikipedia.org/wiki/Waffen-SS for more information.
mherdeg 12y ago
If anyone else is feeling a sense of déjà vu in reading this article — there was another in depth story about some of the technical details behind this fault in the NYT in March 2014, http://www.nytimes.com/2014/03/29/business/a-florida-engineer-unlocked-the-mystery-of-gms-ignition-flaw.html http://www.nytimes.com/2014/03/29/business/a-florida-enginee... . This Pando article has re-reported the story with lots of additional details and does some great storytelling, so I'm glad to see it. Just wanted to add a note for anyone who is wondering (like I was) "whoa, haven't I seen this before?".
- mreiland 12y ago
  I liked the storytelling as well, the only part that annoyed me was the way he continued to drop back into the romanticizing of the characters. You've already hooked me, I want to know what happens next, I don't give a shit about the lifestory of the engineer, I only care that he's an engineer and how he got involved in the story. Other than that, I thought it was a great writeup.
S_A_P 12y ago
Not really relevant to the article but bugs me because it was incorrect. The Bronco II is not what OJ Simpson drove on his infamous low speed chase. He was driving the full size Bronco. The bronco II was the mid size ford ranger based suv(at least the article gets that right)
- __david__ 12y ago
  Perhaps you misread the article? > A cultural touchstone, the Bronco II was not only infamous for having been the little brother to the Bronco, which ferried a fleeing O.J. Simpson as a battalion of police cars and news helicopters gave chase on national television… That sounds correct to me.
vermontdevil 12y ago
This article illustrates the need for trial lawyers and keeping that route option open. See how Texas shut down the medical lawsuits under the guide of malpractice reform where it didn't really work as advertised. In an ideal world, the consumer protection agencies, etc would do their jobs. But they can't because of the revolving door syndrome, the industry lobbying, and so on. So what are our alternatives to force companies to fix their product defects? People actually died due to their shoddy design/production and these companies tried to hide or cover up. I wouldn't be surprised to see an increased effort to pass laws to remove the ability for us to sue companies in case something goes wrong. This on top of the media campaign to paint trial lawyers as the scum of earth. Some may but many are actually very good and very dedicated at what they do.
- zaroth 12y ago
  > In an ideal world, the consumer protection agencies, etc would do their jobs. But they can't because of the revolving door syndrome, the industry lobbying, and so on. The article mentions the possibility of criminal charges against the switch engineer, if someone kept digging there could very well be evidence of criminal wrongdoing by the oversight agencies as well. When people start going to jail real change will be effected. But who's going the champion the multi-year investigation without a multi-million dollar carrot? There's been quite a string of front page posts the last week; mercury dumping in rivers, tainted generic drugs, faulty ignition switches. Perhaps thousands of cases of criminally negligent homicide and no charges filed in any case. Cut off the criminal and civil routes for justice and I guess vigilantism is all that's left.
- Shivetya 12y ago
  What Texas is doing/has done would not have impacted this issue if a similar law were applied to manufacturing. The key here is that people at GM knew they had a problem and successfully hid it. If you were to apply a loosely copied version of the Texas law it would start a timer two years from revelation to get the suit going. What annoys me most is the US government fined Toyota a billion dollars for not quickly identifying problems and so far GM has taking a walk comparatively. I really want to see a real fine on GM for this , given their CEO her positions coming up through the ranks would have/should have known about this problem. Maybe not its extent, but she would have at least heard aboutit
teh_klev 12y ago
Previous HN discussion when this appeared in the NY Times: https://news.ycombinator.com/item?id=7492595 https://news.ycombinator.com/item?id=7492595
segmondy 12y ago
For those of us who are programmers, if you wrote software that lives depended on, how confident will you be that lives won't be lost? My take away from this is to work harder towards excellent, better quality control, more documentation and don't change a working system without following process that informs other. In the software domain, there's this bravado about hotfixing an issue in production. Raymond DeGiorgio did that, he fixed the problem without going through proper channels, not thinking of "legacy" cars, and those one's failing to get patched, cost lives.
a3n 12y ago
These kinds of scenarios could appear in the "negative" column of a positives and negatives analysis of whether to work in a safety critical, regulated industry. Because even if an individual does his job diligently and ethically, there is a risk that corporate motivations can coat you in shit. Articles like this, and the Ranbaxy pharmaceutical article, make me wonder if the satisfaction of working in safety critical industries is worth the risk.
derekp7 12y ago
Question: What were they trying to achieve with the 2005 version of the ignition switch, which required a new design, that couldn't be provided from a 1975 switch design? It seems that a lot of effort and money is poured into redesigning every component of products every few years, when this was a solved problem many decades ago. After all, the ignition switch has to 1) be turned with an authorized key, 2) provide electrical contact for accessory, run, and start positions.
- Thimothy 12y ago
  Have you had a car made in the last 15 or so years? Nowadays ignition switches do everything but starting the ignition system. As the car is fully controlled by a computer, the key is an electronic gizmo that authenticates the driver as the owner and signals the starting sequence. The revolving movement is completely unnecessary and just provided for familiarity purposes, there are cars that instead of a key have a card and start by pushing a button. My bet is that maybe in 2005 GM was building the first switches under this principle, thus the need to preserve the electrical characteristics of the system, so the data transmision channels between the key and the computer wouldn't get compromised.
  - derekp7 12y ago
    Ah, that explains it -- my F150 pickup truck is 14 years old, the only technology it has is an RFID chip in the key (which I assume is read by a sensor in the location of the ignition switch, fed to the engine computer, but otherwise not requiring any other re-design of the key switch).
- Too 12y ago
  A 1975 switch probably doesn't have the same crash characteristics as a modern one, it's quite common that keys in older cars will penetrate your knee during a crash.
kazinator 12y ago
> While Brooke’s lap belt glued her waist to the seat, her shoulder harness went slack the instant the engine shut off. Jawdrop ... what? Those things should be purely mechanical, locked in place by masses undergoing acceleration. Shoulder belts requiring engine power is just monumentally stupid.
- aidenn0 12y ago
  This was a case of 12V power disappearing to a lot of systems; that still doesn't explain the slack harness, but electric pretensioners would certainly fail to work in a crash under such circumstances.
gregpilling 12y ago
So the writer describes the guy who does the CSI on the car as a "grease monkey" and implies he is an uneducated hick. Yet it was his work that figured it all out. Have you ever used a Tech 2 scanner to diagnose one of those cars? They are quite complex, with mutliple computers networked over the canbus system. Grease Monkey. Like Code Monkey or Porch Monkey. Derogatory.
- exogen 12y ago
  I didn't read anything implying he is an uneducated hick unless you think the term "grease monkey" itself has that implication. But I don't that that term has the connotation you're saying it does. Whenever I've heard "grease monkey" it seems more like the car equivalent of "hacker" – that tinkering with cars is something you deeply understand and enjoy, not just a job.
  - NamTaf 12y ago
    Yeh, I've always taken the term 'grease monkey' as the mechanic equivalent to 'hacker', and not bad at all. IIRC it was popularised during WW2 where guys with a mechanical knack would hack together repairs to their vehicles out of spare parts. Those guys were the grease monkeys and were well-respected for keeping the mechanised fleet operating when everything was going to hell.
  - gregpilling 12y ago
    "Charlie Miller has been a grease monkey for almost his entire life. Born in Tippo, a tiny town in the heart of the Mississippi Delta, Miller overhauled his first engine when he was 12. He put himself through a local college working as a mechanic while supporting an addiction to drag racing and after he graduated he become a professional hot rod driver. In 1974 he opened an auto repair shop in Merigold, but it was 15 years before he was asked to offer an expert opinion, and only because of happenstance" That didn't sound very complimentary to me. Maybe it could have been written like: "Charlie Miller was asked by a friend to help in a court case. His friend has asked for help because of Charlie's unique background. Charlie had a love for cars since he was a child, and was smart enough to overhaul an engine at the age of 12. He was not only good with tuning an engine, he could also drive. He competed in drag racing as an amateur before becoming a professional driver. A graduate from a local college, Charlie was able to combine his love of cars, his self directed education and his keen diagnostic sense to solve his friends problem as an expert witness, and that is how he started his path to become the guy who solved the biggest recall in automotive history. "
gfwilliams 12y ago
I'll probably get flamed, but... I'm shocked that everyone here is so against GM on this. Given the complexity of a car and all the things that could go wrong, this is a very minor issue. The brakes still work (no ABS though), steering still works (it's just harder to turn), and lights still work. It basically turns into a car that's 20 years older, not a coffin on wheels. The driver also knew there was a problem, wasn't happy with it, but still drove her car regardless and then wasn't able to cope when it happened again. She wasn't forced to drive it. Maybe it's because I tend to drive older cars, but I'm all too aware that they're machines and they break. I've had all kinds of things fail when driving and I've had to try and cope with them safely. Luckily I have, but if I hadn't been so lucky I wouldn't blame the car manufacturer. 100 years ago you could get killed by all kinds of things and it was more or less just accepted - why are we now at the stage where a switch that's a bit too easy to turn is the cause of all this blame?
- stephenitis 12y ago
  Can't tell if this is supposed to be sarcasm. I think the coverup or moral dilution at GM stands to be pretty damn awful. engineering a failure situation and not playing out how it's most dangerous scenarios might end is pretty bad. I don't think anyone would drive a car over 20 mph if they knew it would shut off and literally have to "cope" with the stress and impending accident and hope to manually steer the car to safety. I'm shocked you don't hold manufacturers to a higher standard.
  - gfwilliams 12y ago
    It's not sarcasm, I'm just very surprised that everyone's opinion is so one-sided about this. I think if you put yourself in the engineer's position, he probably didn't believe that having a switch that was a bit easier to turn would lead to anything more than mild inconvenience. I think if GM honestly thought this would cause even one death they would have changed the lock barrels. Even if you ignore the human side, the potential costs damages from something like this would totally outweigh the cost of changing them.
- PhasmaFelis 12y ago
sbhere 12y ago
> 'Despite the conclusions of the police accident report, which claimed that Brooke had been driving too fast for the road conditions and lost control when she hydroplaned over a sheet of water...' I would love to see an article on statistics of how many police reports are factually inaccurate, even if only proven inaccurate by later reporting/exposition.

269 points by waffle_ss 12y ago

latch 12y ago

Seems to be a lot of things at play, but what surprised me was the ability to issue a modified part without a change to the model number. Seems like something that could be enforced automatically? Like a version control system? Are we running out of numbers?

bagels 12y ago
Well, it's either a way to perform a coverup, or a cost savings by not needing to make new tooling with the new part number on it.
- mirkules 12y ago
  A new part number doesn't necessarily mean new tooling. However, you are correct in saying that it is a way to cut cost by not issuing a new part number. A new part number means: - Entirely new release/approval cycle. That means going through design, release and QA stages (QA is extremely expensive in automotive industry). Remember auto industry is not the software industry, there is no "agile" - waterfall is king because once the product is released, it is infinitely more expensive to change it - Recalling old part numbers and shipping new ones - Even the cost of printing new labels and making new packing materials can be non-insignificant That said, a change in part equals a new part number, period. So, this was very likely a coverup AND an improper cost-reduction exercise.
  - bagels 12y ago
    I agree with you. I did happen to notice from pictures that these particular parts have the part number cast on to them, so new tooling may be required.
    - mirkules 12y ago
      Ah yeah, good observation.
- frankydp 12y ago
  Doesnt this mean that the part manufacturer had to be in on the process and that it probably wasnt the first time?
Gargoyle888 12y ago
I was assume that this was part of a structured bill of materials. Changing the rev level should be sufficient to identify the change and would not entail changing the BOM. I'm going to make a strong guess that the rev level was incremented (the article doesn't mention rev levels). If so, I don't see anything wrong with using the same P/N. It's not like that part number exists elsewhere in the vehicle.

scotty79 12y ago

Absolutely horrified. > The Cobalt’s engine shut off and the lights inside and outside the car dimmed. Melton hit the brakes, but no power from the engine meant no anti-lock brakes and no power steering. I wasn't aware that anti-lock brakes (is that ABS?) is not a purely mechanical system. Also, was power steering hydraulic or electric? Shouldn't it be run of battery for short period of time? > While Brooke’s lap belt glued her waist to the seat, her shoulder harness went slack the instant the engine shut off. Now that's just criminally stupid. EDIT: I guess I need to read through to the end to the article but that's a hard read for me for some reason.

mirkules 12y ago
ABS is indeed the Anti-lock Braking System and I think it depends on sensors to determine the skidding amount and on software to determine the interval at which the brakes are applied and let go. So, the ABS is run off electricity and not a purely mechanical system. However, brakes themselves (without the "anti-lock" part) are purely mechanical/hydraulic, but since the brakes locked up, it didn't matter. I'm not entirely sure about electric power steering systems and how they work. ABS: https://en.wikipedia.org/wiki/Anti-lock_braking_system https://en.wikipedia.org/wiki/Anti-lock_braking_system EPS: http://www.caranddriver.com/features/electric-vs-hydraulic-steering-a-comprehensive-comparison-test-feature http://www.caranddriver.com/features/electric-vs-hydraulic-s... https://en.wikipedia.org/wiki/Electric_power_steering https://en.wikipedia.org/wiki/Electric_power_steering
- toomuchtodo 12y ago
  ABS depends on power not only for wheel speed measurement, but also for power to drive the brake piston.
- bonestamp2 12y ago
  > I'm not entirely sure about electric power steering systems and how they work. It comes in a variety of forms, but it's usually an electric motor that adds torque as you turn the steering wheel to make it easier to turn.

concernedhner 12y ago

At some level, I am horrified at what was done with prior knowledge by this institution. At another, this incident makes me wonder how this ties into the diffused nature of responsibility within corporations. Was the diffusion of responsibility within these corporations was so great that they could perpetuate these acts without any one individual bearing the moral responsibility of what happened? If so, then these corporations inadvertently have created an entity that is independent of the individual stakeholders and their moral pangs. That is a very frightening prospect...

csours 12y ago
It is not only responsibility but also knowledge. The ignition switch engineer may not be aware of what exactly it turns on and off; even if that engineer is, other engineers are not. (See page 1 of the GM Valukas report) Analogy: does your DBA know about flushing to disk?
- Too 12y ago
  Very good point. In huge systems such as a car everybody can not know everything. But in this case, if the ignition switch team does not know how their component affects everything about the car or how important it is the responsibility should move up to whoever writes the specs on the ignition switch. First of all for designing a system with a single point of failure in the first place, secondly for not loudly notifying ignition switch team that their component is vitally important and a single point of failure. As i also said in a post above, way too much attention is given to the faulty switch when this is actually a much larger system design error.

PhantomGremlin 12y ago

Along with the other posters here, I am also "horrified". But isn't this more-or-less the same at most/all other companies? They become old and sclerotic and hidebound and resistant to change. Aren't most old companies guilty of the equivalent of the "GM nod"? a staple of GM managers, nodding in agreement at steps that should be taken then doing nothing Maybe companies like GM eventually run their course, and should be allowed to die. Maybe we were wrong to bail them out in 2009? Would the country be better off if a new company could take GM's place? But, of course, GM couldn't be replaced "as is". It could only be replaced by something more in tune with the times. It's only a {relatively speaking} tiny company, but Elon Musk and Telsa could be "the next GM". Until Tesla eventually runs its course!

djloche 12y ago
Not one company should be bailed out. If a business fails, those who think they can run that sort of company better should step in and buy the assets and give it a go. In the many or most cases where companies have built what has become core infrastructure, regulations already exist for orderly shutdown and sale of assets without disruption of service. If a company becomes 'too big to fail', this means regulatory capture has already occurred and you have far greater problems to resolve than mere bankruptcy.
- kefka 12y ago
  I disagree. Bail out big companies who fail miserably. THEN do the next step: Take ownership of said company. If We're paying for it, We should control it.
rqebmm 12y ago
>Would the country be better off if a new company could take GM's place? Theoretically this is what happened with the bailout. Everyone made fun of Mary Barra for her "this all happened at the 'old GM', so the 'new GM' can't be responsible" but there's a reason it worked. GM was replaced by a new company: GM.

csours 12y ago

All sympathy to the families involved; what makes this the worst auto scandal in history? Are there numbers for this or is this based on public opinion? Disclosure: I have been a GM employee since 2013

viseztrance 12y ago
I've read the article and this blew my mind: "[...] if you took all of the cars that GM has recalled this year and lined them up bumper to bumper, you’d end up with a line that would wrap around the earth four times." That's 30 million cars.
- kefka 12y ago
  And if you took all the software that has been recalled, it would line up to nothing... Because software doesn't have dimension. And aside radiological software incidents (Therac 25 and similar), software errors usually don't kill people. Engineering design failures usually do.
wglb 12y ago
Can you list some of the others?
- csours 12y ago
  Bronco, Explorer, and F150 rollovers -fixed by adding stability and Tire pressure monitors Toyota's gas pedal and floor mat recall - halfway similar to this issue in that it involved very poor affordance (drivers could not immediately tell what was wrong, they thought they were pressing the break, but the gas was pressed as well) GM Streetcar conspiracy Cars from the Big 3 used to literally fall apart in the 70's due to poor assembly. (Welcome Toyota) Ford Pintos in the 1970s had badly under-designed gas tanks that ruptured and caused fires in rear-end collisions. Media put the death toll in the "hundreds", NHTSA had a death toll of 27. The scandal of the Big 3 killing smaller car companies like the Tucker 48. Incredibly stupid design decisions like planned obsolescence. GMs policy of "build a bunch of cars and make people buy them" from the 1970's to the early 2000's - responsible for peoples poor opinion of GM and competitors that followed the same mantra (It even bit Toyota in the mid 2000's)

ploxiln 12y ago

When I read this story, I thought of parallels to large complex software projects (of course). You could call a large software system that would take more effort fix than has been been spent to create it so far "totalled", like a badly crashed car. It happens (usually with government payroll systems it seems). GM seems to have "totalled" their entire lineup of cars designed and produced over the last 20 years - the quality of their organization's work has been so low that it's not really possible for GM to find and fix all the serious problems on all the cars they've made. They would have to dissect all the parts from a sampling of cars of each year, since they can't even trust that the same part number is really the same in different years! It's probably cheaper to make a couple new car models, really make sure they're not defective, and provide everyone who still has a GM car with a replacement car. Cheaper, but maybe still not possible. "Good enough" massive recalls will have to do.

lovelearning 12y ago

I feel the root cause here is the culture of setting hard deadlines by management. Getting a deadline moved even for logical reasons involves pushing against massive internal red tape. Missing a deadline invites greater punishment from management than compromising on quality. Naturally, quality loses out. Happens all the time in our own industry, and sadly, this case shows that it happens even in safety critical industries. All engineers, regardless of industry, should be trained in how to negotiate with management. All managers, regardless of industry, should be trained to respect opinions of subordinates.

amirmc 12y ago
That almost sounds like you're absolving the engineers (as though they can do no wrong). Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). This is a complex issue and I do not agree with the (over-simplistic) reduction to 'management' and 'deadlines'.
- lovelearning 12y ago
  Not absolving the engineers. The engineer in this case - and other cases - may well be guilty - both legally and morally - of mistakes. What I'm trying to understand is, what would make an experienced senior engineer prefer such shortcuts over doing the right thing. The article says, "Faced with a deadline, DeGiorgio replied: If increasing the torque will destroy the switch then do nothing. Maintain present course. Under no circumstances do we want to compromise the electrical performance of the switch.". I interpret it as the deadline being a root cause. At the heart of any complex issue lies one or more root cause(s). What were they here? What made him behave that way? Fudging the records is a consequence, not a root cause.
  - amirmc 12y ago
    You're taking four words from an article (not original sources) and deducing a root cause. In the absence of those four words -- which is the only time 'deadline' is mentioned -- would you draw the same conclusion? I do not. I could argue that fudging the records is a 'root cause' of why the investigation took longer. If you want to go reductio ad absurdum then really the root cause is ultimately human fallibility.

yason 12y ago

Still wondering why make physical ignition switches that actually mechanically cut the flow of electricity. For years, cars have been equipped with all kinds of 'convenience' electric components such as Bluetooth, touch panels, and on top of that all critical elements in the engine have been controlled by an ECU for decades -- yet still only some cars have electrically, not mechanically, controlled ignition. The car key doesn't even have to be wireless: you could still have the lock and ignition key as now if you really wanted but turning the key would only give a signal to the car's computer to wake up from sleep, authenticate the key, close the main circuits electrically to let the components start drawing the big current from the battery, and start booting sequence, self-tests etc. and eventually drive current to the starter motor to crank the engine up. That's a huge number of things to do, and it had best be left for the computer to handle. Pulling the key from the physical ignition switch is aking to starting and stopping your PC by pulling and inserting the plug from/to the wall socket. You'd better push the suspend button and let your laptop put itself to sleep and let it wake up by itself. It's also much easier to design a system that will handle startup and shutdown by itself, rather than a system that must survive opening and closing the main circuits at any time.

toomuchtodo 12y ago
Because its cheaper to put a $1-5 mechanical switch in than to put a smart key system. Yet again, cost is the driving factor. How many lawsuits would you need due to ignition switch failures before smart keys for all cars was cheaper? A lot.
cnvogel 12y ago
I think there's a change happening, some new cars can be started by button-press when the transponder (replacement for the key) is somewhere in the car (e.g. in your trousers' pocket). And then, of course there's the other aspect that in some cases you absolutely want to be able to cut power to your engine. Wasn't that exactly the case with Toyotas drive-by-wire throttles that apparently made people unable to stop their cars? Still: When you look at the electrical system for, e.g. a commercial airplane, you'll see a dozen different independent power-supply busses that can be fed from each other, different engines, or batteries. So that in case of power-loss in one or a few of them, not all safety critical systems black-out. I don't claim that such complexity is helpful or desirable in a car, but with the majority of drivers relying on computerized systems it probably makes sense to make power supplies more redundant. {probably that's already happening, and I'm just unaware of it?}

function_seven 12y ago

The ignition switch in question seems totally unremarkable. I wonder why it was redesigned in the first place? Why not use one of the many existing parts GM had at the time? Also—somewhat of a nitpick to an otherwise good article—the Bronco II is a completely different model from the one driven by Al Cowlings during the O.J. Simpson chase. That was the full-sized Bronco. When details like that are wrong, it usually makes me suspect the accuracy of other facts in the story (although in this case, the article seems otherwise accurate).

spydum 12y ago
I also wondered this since the story broke.. Why do they redesign core components like this, instead of just iterating on previous designs? I fight this every day in the enterprise space. My only conclusion is ignorance: sharing previous design or infrastructure knowledge is hard (or at least, not prioritized). So folks end up recreating the proverbial wheel.
- kbart 12y ago
  I guess that might be due to parts' supply and logistics problems. When production counts in hundreds of thousands and millions, getting all parts together and in time is quite a challenge.

markvdb 12y ago

Waffle_ss , change your user name.

markvdb 12y ago
I should have added some historical context to this request for those unaware of it. It's hard not to see the user name Waffle_SS as an unnecessarily hurtful trivialisation of the Waffen-SS, the Nazi paramilitary force. See https://en.wikipedia.org/wiki/Waffen-SS https://en.wikipedia.org/wiki/Waffen-SS for more information.

mherdeg 12y ago

If anyone else is feeling a sense of déjà vu in reading this article — there was another in depth story about some of the technical details behind this fault in the NYT in March 2014, http://www.nytimes.com/2014/03/29/business/a-florida-engineer-unlocked-the-mystery-of-gms-ignition-flaw.html http://www.nytimes.com/2014/03/29/business/a-florida-enginee... . This Pando article has re-reported the story with lots of additional details and does some great storytelling, so I'm glad to see it. Just wanted to add a note for anyone who is wondering (like I was) "whoa, haven't I seen this before?".

mreiland 12y ago
I liked the storytelling as well, the only part that annoyed me was the way he continued to drop back into the romanticizing of the characters. You've already hooked me, I want to know what happens next, I don't give a shit about the lifestory of the engineer, I only care that he's an engineer and how he got involved in the story. Other than that, I thought it was a great writeup.

S_A_P 12y ago

Not really relevant to the article but bugs me because it was incorrect. The Bronco II is not what OJ Simpson drove on his infamous low speed chase. He was driving the full size Bronco. The bronco II was the mid size ford ranger based suv(at least the article gets that right)

__david__ 12y ago
Perhaps you misread the article? > A cultural touchstone, the Bronco II was not only infamous for having been the little brother to the Bronco, which ferried a fleeing O.J. Simpson as a battalion of police cars and news helicopters gave chase on national television… That sounds correct to me.

vermontdevil 12y ago

This article illustrates the need for trial lawyers and keeping that route option open. See how Texas shut down the medical lawsuits under the guide of malpractice reform where it didn't really work as advertised. In an ideal world, the consumer protection agencies, etc would do their jobs. But they can't because of the revolving door syndrome, the industry lobbying, and so on. So what are our alternatives to force companies to fix their product defects? People actually died due to their shoddy design/production and these companies tried to hide or cover up. I wouldn't be surprised to see an increased effort to pass laws to remove the ability for us to sue companies in case something goes wrong. This on top of the media campaign to paint trial lawyers as the scum of earth. Some may but many are actually very good and very dedicated at what they do.

zaroth 12y ago
> In an ideal world, the consumer protection agencies, etc would do their jobs. But they can't because of the revolving door syndrome, the industry lobbying, and so on. The article mentions the possibility of criminal charges against the switch engineer, if someone kept digging there could very well be evidence of criminal wrongdoing by the oversight agencies as well. When people start going to jail real change will be effected. But who's going the champion the multi-year investigation without a multi-million dollar carrot? There's been quite a string of front page posts the last week; mercury dumping in rivers, tainted generic drugs, faulty ignition switches. Perhaps thousands of cases of criminally negligent homicide and no charges filed in any case. Cut off the criminal and civil routes for justice and I guess vigilantism is all that's left.
Shivetya 12y ago
What Texas is doing/has done would not have impacted this issue if a similar law were applied to manufacturing. The key here is that people at GM knew they had a problem and successfully hid it. If you were to apply a loosely copied version of the Texas law it would start a timer two years from revelation to get the suit going. What annoys me most is the US government fined Toyota a billion dollars for not quickly identifying problems and so far GM has taking a walk comparatively. I really want to see a real fine on GM for this , given their CEO her positions coming up through the ranks would have/should have known about this problem. Maybe not its extent, but she would have at least heard aboutit

teh_klev 12y ago

Previous HN discussion when this appeared in the NY Times: https://news.ycombinator.com/item?id=7492595 https://news.ycombinator.com/item?id=7492595

segmondy 12y ago

For those of us who are programmers, if you wrote software that lives depended on, how confident will you be that lives won't be lost? My take away from this is to work harder towards excellent, better quality control, more documentation and don't change a working system without following process that informs other. In the software domain, there's this bravado about hotfixing an issue in production. Raymond DeGiorgio did that, he fixed the problem without going through proper channels, not thinking of "legacy" cars, and those one's failing to get patched, cost lives.

a3n 12y ago

These kinds of scenarios could appear in the "negative" column of a positives and negatives analysis of whether to work in a safety critical, regulated industry. Because even if an individual does his job diligently and ethically, there is a risk that corporate motivations can coat you in shit. Articles like this, and the Ranbaxy pharmaceutical article, make me wonder if the satisfaction of working in safety critical industries is worth the risk.

derekp7 12y ago

Question: What were they trying to achieve with the 2005 version of the ignition switch, which required a new design, that couldn't be provided from a 1975 switch design? It seems that a lot of effort and money is poured into redesigning every component of products every few years, when this was a solved problem many decades ago. After all, the ignition switch has to 1) be turned with an authorized key, 2) provide electrical contact for accessory, run, and start positions.

Thimothy 12y ago
Have you had a car made in the last 15 or so years? Nowadays ignition switches do everything but starting the ignition system. As the car is fully controlled by a computer, the key is an electronic gizmo that authenticates the driver as the owner and signals the starting sequence. The revolving movement is completely unnecessary and just provided for familiarity purposes, there are cars that instead of a key have a card and start by pushing a button. My bet is that maybe in 2005 GM was building the first switches under this principle, thus the need to preserve the electrical characteristics of the system, so the data transmision channels between the key and the computer wouldn't get compromised.
- derekp7 12y ago
  Ah, that explains it -- my F150 pickup truck is 14 years old, the only technology it has is an RFID chip in the key (which I assume is read by a sensor in the location of the ignition switch, fed to the engine computer, but otherwise not requiring any other re-design of the key switch).
Too 12y ago
A 1975 switch probably doesn't have the same crash characteristics as a modern one, it's quite common that keys in older cars will penetrate your knee during a crash.

kazinator 12y ago

> While Brooke’s lap belt glued her waist to the seat, her shoulder harness went slack the instant the engine shut off. Jawdrop ... what? Those things should be purely mechanical, locked in place by masses undergoing acceleration. Shoulder belts requiring engine power is just monumentally stupid.

aidenn0 12y ago
This was a case of 12V power disappearing to a lot of systems; that still doesn't explain the slack harness, but electric pretensioners would certainly fail to work in a crash under such circumstances.

gregpilling 12y ago

So the writer describes the guy who does the CSI on the car as a "grease monkey" and implies he is an uneducated hick. Yet it was his work that figured it all out. Have you ever used a Tech 2 scanner to diagnose one of those cars? They are quite complex, with mutliple computers networked over the canbus system. Grease Monkey. Like Code Monkey or Porch Monkey. Derogatory.

exogen 12y ago
I didn't read anything implying he is an uneducated hick unless you think the term "grease monkey" itself has that implication. But I don't that that term has the connotation you're saying it does. Whenever I've heard "grease monkey" it seems more like the car equivalent of "hacker" – that tinkering with cars is something you deeply understand and enjoy, not just a job.
- NamTaf 12y ago
  Yeh, I've always taken the term 'grease monkey' as the mechanic equivalent to 'hacker', and not bad at all. IIRC it was popularised during WW2 where guys with a mechanical knack would hack together repairs to their vehicles out of spare parts. Those guys were the grease monkeys and were well-respected for keeping the mechanised fleet operating when everything was going to hell.
- gregpilling 12y ago
  "Charlie Miller has been a grease monkey for almost his entire life. Born in Tippo, a tiny town in the heart of the Mississippi Delta, Miller overhauled his first engine when he was 12. He put himself through a local college working as a mechanic while supporting an addiction to drag racing and after he graduated he become a professional hot rod driver. In 1974 he opened an auto repair shop in Merigold, but it was 15 years before he was asked to offer an expert opinion, and only because of happenstance" That didn't sound very complimentary to me. Maybe it could have been written like: "Charlie Miller was asked by a friend to help in a court case. His friend has asked for help because of Charlie's unique background. Charlie had a love for cars since he was a child, and was smart enough to overhaul an engine at the age of 12. He was not only good with tuning an engine, he could also drive. He competed in drag racing as an amateur before becoming a professional driver. A graduate from a local college, Charlie was able to combine his love of cars, his self directed education and his keen diagnostic sense to solve his friends problem as an expert witness, and that is how he started his path to become the guy who solved the biggest recall in automotive history. "

gfwilliams 12y ago

I'll probably get flamed, but... I'm shocked that everyone here is so against GM on this. Given the complexity of a car and all the things that could go wrong, this is a very minor issue. The brakes still work (no ABS though), steering still works (it's just harder to turn), and lights still work. It basically turns into a car that's 20 years older, not a coffin on wheels. The driver also knew there was a problem, wasn't happy with it, but still drove her car regardless and then wasn't able to cope when it happened again. She wasn't forced to drive it. Maybe it's because I tend to drive older cars, but I'm all too aware that they're machines and they break. I've had all kinds of things fail when driving and I've had to try and cope with them safely. Luckily I have, but if I hadn't been so lucky I wouldn't blame the car manufacturer. 100 years ago you could get killed by all kinds of things and it was more or less just accepted - why are we now at the stage where a switch that's a bit too easy to turn is the cause of all this blame?

stephenitis 12y ago
Can't tell if this is supposed to be sarcasm. I think the coverup or moral dilution at GM stands to be pretty damn awful. engineering a failure situation and not playing out how it's most dangerous scenarios might end is pretty bad. I don't think anyone would drive a car over 20 mph if they knew it would shut off and literally have to "cope" with the stress and impending accident and hope to manually steer the car to safety. I'm shocked you don't hold manufacturers to a higher standard.
- gfwilliams 12y ago
  It's not sarcasm, I'm just very surprised that everyone's opinion is so one-sided about this. I think if you put yourself in the engineer's position, he probably didn't believe that having a switch that was a bit easier to turn would lead to anything more than mild inconvenience. I think if GM honestly thought this would cause even one death they would have changed the lock barrels. Even if you ignore the human side, the potential costs damages from something like this would totally outweigh the cost of changing them.
PhasmaFelis 12y ago

sbhere 12y ago

> 'Despite the conclusions of the police accident report, which claimed that Brooke had been driving too fast for the road conditions and lost control when she hydroplaned over a sheet of water...' I would love to see an article on statistics of how many police reports are factually inaccurate, even if only proven inaccurate by later reporting/exposition.