5 ms·

I feel the root cause here is the culture of setting hard deadlines by management. Getting a deadline moved even for logical reasons involves pushing against m

I feel the root cause here is the culture of setting hard deadlines by management. Getting a deadline moved even for logical reasons involves pushing against massive internal red tape. Missing a deadline invites greater punishment from management than compromising on quality. Naturally, quality loses out. Happens all the time in our own industry, and sadly, this case shows that it happens even in safety critical industries. All engineers, regardless of industry, should be trained in how to negotiate with management. All managers, regardless of industry, should be trained to respect opinions of subordinates.

amirmc 12y ago
That almost sounds like you're absolving the engineers (as though they can do no wrong). Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). This is a complex issue and I do not agree with the (over-simplistic) reduction to 'management' and 'deadlines'.
- lovelearning 12y ago
  Not absolving the engineers. The engineer in this case - and other cases - may well be guilty - both legally and morally - of mistakes. What I'm trying to understand is, what would make an experienced senior engineer prefer such shortcuts over doing the right thing. The article says, "Faced with a deadline, DeGiorgio replied: If increasing the torque will destroy the switch then do nothing. Maintain present course. Under no circumstances do we want to compromise the electrical performance of the switch.". I interpret it as the deadline being a root cause. At the heart of any complex issue lies one or more root cause(s). What were they here? What made him behave that way? Fudging the records is a consequence, not a root cause.
  - amirmc 12y ago
    You're taking four words from an article (not original sources) and deducing a root cause. In the absence of those four words -- which is the only time 'deadline' is mentioned -- would you draw the same conclusion? I do not. I could argue that fudging the records is a 'root cause' of why the investigation took longer. If you want to go reductio ad absurdum then really the root cause is ultimately human fallibility.
    - Natsu 12y ago
      > then really the root cause is ultimately human fallibility. I'm not sure why you see this as 'absurdity', because it is a problem and it is something we try to design around with everything from good UIs to fatigue management.
      - amirmc 12y ago
        I see it as absurd because when you get to that level, there's no information about what you can practically do to fix it. You've gone way past the threshold of usefulness — if you get my meaning (in other words, this reductionist view could end with "because, physics", but that's not very helpful). You don't attempt to fix human fallibility itself but instead create systems and process to reduce both it's likelihood and impact (as you describe) for specific scenarios. Those systems and processes can then be improved and new scenarios added. In this particular case, I find the reduction to 'management and hard deadlines' to be a useless summary (as well as incorrect) as it goes beyond the threshold.
        Natsu 12y ago
        > I see it as absurd because when you get to that level, there's no information about what you can practically do to fix it. I disagree. I see it as important to recognize our shortcomings and find successful ways to deal with them. It may be that we haven't yet figured this out in general, but figuring out specific cases may allow us to solve individual pieces of the problem one at a time and improve things even if we cannot fully overcome our flaws.
        amirmc 12y ago
        I don't see how your comment disagrees with mine.
  - cynicalkane 12y ago
    > what would make an experienced senior engineer prefer such shortcuts over doing the right thing One thing I've learned from software engineering is that experienced people are capable of choosing lazy shortcuts by themselves.
  - fnordfnordfnord 12y ago
    >What I'm trying to understand is, what would make an experienced senior engineer prefer such shortcuts over doing the right thing. Organizational failure. An onerous documentation change/approval management scheme. Covering up a mistake. A culture that causes people to fear losing their jobs, especially for cost reasons, or the feeling that company management are looking for reasons to cut staff. Having your group be blamed for an expensive product recall makes you an easy target. One of the last things a middle-aged engineer in the US auto industry wants to be right now is laid-off. We naively expect a senior engineer to do the ethically correct thing without respect to whether it has negative personal consequences. The problem with this idea is that the consequences for being "that guy" are too great; and both engineering societies and state licensing boards have been derelict in their roles of helping to protect engineers who refuse to do the ethically gray/wrong. >I interpret it as the deadline being a root cause. At the heart of any complex issue lies one or more root cause(s). There are so many things wrong with product engineering management, but yeah laying a deadline that no engineer has the courage to break would definitely be one of them.
    - zaroth 12y ago
      It doesn't sound like such a horrible position to be in; working for a multi-billion dollar company that was trying to force you to ship a part that could kill people. Whistleblowers get 1/3rd of damages, it's like wining the lottery for ethical people. How long does it normally take to design one of these switches, when you've been doing it most of your life? Knowing absolutely nothing about the complexity of these switches, but knowing what small hardware teams have been able to accomplish at companies I've worked at, 3 years sounds like a more than reasonable time frame.
      - fnordfnordfnord 12y ago
        >It doesn't sound like such a horrible position to be in; working for a multi-billion dollar company that was trying to force you to ship a part that could kill people. It's probably not a case of some moustache twisting monocle wearing fiend who was the manager and the engineer was absolutely convinced that it was a faulty part that would ultimately be responsible for death and destruction. It was more likely one of a number of similar sub-assemblies, all having some questionable attributes and people trying to ship things under the guise of "perfect is the enemy of the good". At some points along this road, various people probably began realizing that there was a real problem, one that they might have some culpability in. Why the people involved continued to make poor decisions, I don't know. >Whistleblowers get 1/3rd of damages, it's like wining the lottery for ethical people. It's a nice theory. I've yet to see it happen consistently in practice (or pretty much ever). Engineering ethics case studies are littered with cases where engineers tried to alert management of problems and were punished for their efforts. >How long does it normally take to design one of these switches, when you've been doing it most of your life? Not very long given ample resources and nothing else to do. I doubt they were working under those conditions.
  - brudgers 12y ago
    Engineering is the art of technical and economic tradeoffs. Engineers use their judgement, experienced engineers simply have a larger pool of good and bad judgements to draw on. They still get things wrong for the same reasons Uncle Bob has to debug the code he writes, they're human and make mistakes. The domain of automobile use encompasses more variables than a website - there's no sleet over TCP/IP and broken internationalization doesn't result in crushed vertebrae. The vast majority of software engineers and programmers live in a world where the calculus for balancing technical/economic tradeoffs don't involve life safety and billions of dollars of capital investment in the pans.
- fnordfnordfnord 12y ago
  >Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). No. It is not.
- bonestamp2 12y ago
  > Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). This really depends. If the original part was not being made to spec and the "new" part was to spec, the specs never changed and therefore the part number shouldn't change. They would just identify that parts with serial numbers or stamp dates up to X were not made to spec or defective.
loup-vaillant 12y ago
> […] managers […] should be trained to respect opinions of subordinates. That's your problem right there: the subordinacy of engineers. We are biased to listen to high-status people more than low-status people. The strength of this bias varies from people to people, and from culture to culture. But if you are asking someone to listen to the opinion of a direct subordinate, you are asking for trouble. This is one reason why Michael O. Church is so big on guilds, or "professions". If instead of being a subordinate, the engineer was a (possibly certified) master craftsman, things would be different. Just picture the manager calling the the engineer "master", then not listen to her opinion. > […] engineers […] should be trained in how to negotiate with management Conversely, it's harder to negotiate when you have lower status than the manager. You may feel it is not your place. But if you're a master programmer, offering your services to said manager, then you could feel more confident about talking to him as a peer. You could feel more confident about saying things like "your idea won't work —trust me, it's my speciality."
- lovelearning 12y ago
  I agree with you. I'd like to live and work in such a world myself. But my rather pessimistic opinion is that status, hierarchy, etc are deeply embedded in us as a species - just like animal species who have pack leaders or dominant males or whatever - and this won't be easy to get rid of. The ones who hold financial strings, the ones who act more arrogant, more confident, more powerful, more knowledgeable - they are unconsciously seen as having higher status.
  - ppereira 12y ago
    The hierarchy could be inverted with a very small change to the laws governing engineers, just as it has been done for lawyers. The details vary by jurisdiction, but generally lawyers must practice in some form of partnership where non-lawyer members are restricted. Work similar to project management is frequently performed by secretaries that are subordinate to the lawyers. Whether or not strengthening the profession of engineering would be good for society is another question. It is inherently anti-competitive. It could easily slip into a system with very few and highly paid "qualified" engineers whose small number would hinder development. This is a problem with medicine and the supply of doctors in some places.
brudgers 12y ago
That's one way of looking at it. Another is engineers wanting to reinvent the wheel. Ignition switch springs are a solved problem, but specifying a new system smells more like cleverness than using a part from the warehouse shelf. In manufacturing, production lines at all the tiers have to retool at the right time. This year's new models began design over a decade or more ago.
technofiend 12y ago
It's an understandable byproduct of a stack ranked organization. Any time you have the threat of being ranked and rewarded based on delivery well anything that threatens the delivery reflects badly on the rankee and is actively avoided.
- zaroth 12y ago
  The threat of being ranked? We are ranked constantly starting at 1st grade. Competitive ranking should be a very familiar feeling for every American at least. When humans face the prospect of being ranked, the only option then becomes to lie, cheat, take shortcuts, or put lives at risk?! It may be an understandable byproduct for the weak, the proud, or the immature. Cheating is certainly endemic in our society, but I don't believe that absolves the cheaters; it's still criminal/unethical and it should still get you fired, throw off the team, or sent to jail. There is something important to study and learn from this disaster, but my personal opinion is it's not so much a problem with performance evaluations, but rather a spectacular engineering failure combined with a company rotten to the core that completely lost its sense of purpose (hint: serving customers, not short term profit). I do expect that adult humans can be subjected to significant workplace stress and pressure and still be expected to behave ethically and responsibly. I think that the law and justice expect the same.
zaroth 12y ago
Did you read the email from the chief switch engineer about the 'switch from hell'? Seems like this guy seriously fucked up and then tried to cover it up and then continued to lie about it at depositions? Management is absolutely also to blame but it all started with the engineer. His negligence and 'fuck this switch' attitude killed people. He failed technically and ethically to do his job and people died.
- lovelearning 12y ago
  As reported in [1], it looks like he wasn't negligent - "DeGiorgio discussed the problem with Delphi...in February 2002, he had a choice: do nothing to fix it or change the switch and delay production." He certainly knew about the problem, but then decided to not fix it, to keep the production schedule on track. This is what makes me talk about management and deadlines. It's easy to blame it all on that single engineer - and I'm sure GM would love to do so - but isn't it possible the environment in which he worked made it difficult - even risky - to implement the right solution? Perhaps biased by some of my own experiences, I tend to favor the organisational failure explanation that other comments here describe quite well. Of course, everything he did later - the cover up, the lies - were attempts to cover up this one bad decision, and show that his ethics were less than stellar. [1]: http://fortune.com/2014/06/11/gm-switch-ray-degiorgio/ http://fortune.com/2014/06/11/gm-switch-ray-degiorgio/
  - zaroth 12y ago
    I guess I see it differently. Management definitely failed to react properly once it became obvious the switch was faulty. I mean, apparently it came up in car reviews of all places. However, if you're the lead engineer on the switch, you have 23 years of experience on the job, it's your responsibility not to say "fuck it, just ship it." It's his signature on the paperwork. This wasn't a one-time one-day bad decision. This is a switch he worked on for three years. The right course of action is refuse to sign off on a faulty part, and then not try to conceal that the part was faulty once people start dying. Every project has schedule pressure, so if schedule pressure is somehow a defense then engineers are never responsible. I simply don't buy that. Management can decide they want to override the engineer and ship it anyway. GM can fire the guy for not agreeing to ship a faulty part, but then DeGiorgio could have been a whistle blower collecting 1/3rd of the damages against GM right now. I don't blame it singularly on the engineer. TFA says that GM’s Director of Product Investigations shut off the car with her knee, so this goes way beyond just the engineer. But I don't think that lets the engineer off the hook either. The engineer failed spectacularly, possibly criminally, and I hope he has to face a jury of his peers for this. Management also failed the same way, and if the investigators were in place to really push this, I'm sure the paper trail is there to at least indite some of these greedy complacent fools.

amirmc 12y ago

That almost sounds like you're absolving the engineers (as though they can do no wrong). Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). This is a complex issue and I do not agree with the (over-simplistic) reduction to 'management' and 'deadlines'.

lovelearning 12y ago
Not absolving the engineers. The engineer in this case - and other cases - may well be guilty - both legally and morally - of mistakes. What I'm trying to understand is, what would make an experienced senior engineer prefer such shortcuts over doing the right thing. The article says, "Faced with a deadline, DeGiorgio replied: If increasing the torque will destroy the switch then do nothing. Maintain present course. Under no circumstances do we want to compromise the electrical performance of the switch.". I interpret it as the deadline being a root cause. At the heart of any complex issue lies one or more root cause(s). What were they here? What made him behave that way? Fudging the records is a consequence, not a root cause.
- amirmc 12y ago
  You're taking four words from an article (not original sources) and deducing a root cause. In the absence of those four words -- which is the only time 'deadline' is mentioned -- would you draw the same conclusion? I do not. I could argue that fudging the records is a 'root cause' of why the investigation took longer. If you want to go reductio ad absurdum then really the root cause is ultimately human fallibility.
  - Natsu 12y ago
    > then really the root cause is ultimately human fallibility. I'm not sure why you see this as 'absurdity', because it is a problem and it is something we try to design around with everything from good UIs to fatigue management.
    - amirmc 12y ago
      I see it as absurd because when you get to that level, there's no information about what you can practically do to fix it. You've gone way past the threshold of usefulness — if you get my meaning (in other words, this reductionist view could end with "because, physics", but that's not very helpful). You don't attempt to fix human fallibility itself but instead create systems and process to reduce both it's likelihood and impact (as you describe) for specific scenarios. Those systems and processes can then be improved and new scenarios added. In this particular case, I find the reduction to 'management and hard deadlines' to be a useless summary (as well as incorrect) as it goes beyond the threshold.
      - Natsu 12y ago
        > I see it as absurd because when you get to that level, there's no information about what you can practically do to fix it. I disagree. I see it as important to recognize our shortcomings and find successful ways to deal with them. It may be that we haven't yet figured this out in general, but figuring out specific cases may allow us to solve individual pieces of the problem one at a time and improve things even if we cannot fully overcome our flaws.
        amirmc 12y ago
        I don't see how your comment disagrees with mine.
- cynicalkane 12y ago
  > what would make an experienced senior engineer prefer such shortcuts over doing the right thing One thing I've learned from software engineering is that experienced people are capable of choosing lazy shortcuts by themselves.
- fnordfnordfnord 12y ago
  >What I'm trying to understand is, what would make an experienced senior engineer prefer such shortcuts over doing the right thing. Organizational failure. An onerous documentation change/approval management scheme. Covering up a mistake. A culture that causes people to fear losing their jobs, especially for cost reasons, or the feeling that company management are looking for reasons to cut staff. Having your group be blamed for an expensive product recall makes you an easy target. One of the last things a middle-aged engineer in the US auto industry wants to be right now is laid-off. We naively expect a senior engineer to do the ethically correct thing without respect to whether it has negative personal consequences. The problem with this idea is that the consequences for being "that guy" are too great; and both engineering societies and state licensing boards have been derelict in their roles of helping to protect engineers who refuse to do the ethically gray/wrong. >I interpret it as the deadline being a root cause. At the heart of any complex issue lies one or more root cause(s). There are so many things wrong with product engineering management, but yeah laying a deadline that no engineer has the courage to break would definitely be one of them.
  - zaroth 12y ago
    It doesn't sound like such a horrible position to be in; working for a multi-billion dollar company that was trying to force you to ship a part that could kill people. Whistleblowers get 1/3rd of damages, it's like wining the lottery for ethical people. How long does it normally take to design one of these switches, when you've been doing it most of your life? Knowing absolutely nothing about the complexity of these switches, but knowing what small hardware teams have been able to accomplish at companies I've worked at, 3 years sounds like a more than reasonable time frame.
    - fnordfnordfnord 12y ago
      >It doesn't sound like such a horrible position to be in; working for a multi-billion dollar company that was trying to force you to ship a part that could kill people. It's probably not a case of some moustache twisting monocle wearing fiend who was the manager and the engineer was absolutely convinced that it was a faulty part that would ultimately be responsible for death and destruction. It was more likely one of a number of similar sub-assemblies, all having some questionable attributes and people trying to ship things under the guise of "perfect is the enemy of the good". At some points along this road, various people probably began realizing that there was a real problem, one that they might have some culpability in. Why the people involved continued to make poor decisions, I don't know. >Whistleblowers get 1/3rd of damages, it's like wining the lottery for ethical people. It's a nice theory. I've yet to see it happen consistently in practice (or pretty much ever). Engineering ethics case studies are littered with cases where engineers tried to alert management of problems and were punished for their efforts. >How long does it normally take to design one of these switches, when you've been doing it most of your life? Not very long given ample resources and nothing else to do. I doubt they were working under those conditions.
- brudgers 12y ago
  Engineering is the art of technical and economic tradeoffs. Engineers use their judgement, experienced engineers simply have a larger pool of good and bad judgements to draw on. They still get things wrong for the same reasons Uncle Bob has to debug the code he writes, they're human and make mistakes. The domain of automobile use encompasses more variables than a website - there's no sleet over TCP/IP and broken internationalization doesn't result in crushed vertebrae. The vast majority of software engineers and programmers live in a world where the calculus for balancing technical/economic tradeoffs don't involve life safety and billions of dollars of capital investment in the pans.
fnordfnordfnord 12y ago
>Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). No. It is not.
bonestamp2 12y ago
> Changing a part, without also changing it's part-number, doesn't sound like a normal thing to do (and then forget about). This really depends. If the original part was not being made to spec and the "new" part was to spec, the specs never changed and therefore the part number shouldn't change. They would just identify that parts with serial numbers or stamp dates up to X were not made to spec or defective.

loup-vaillant 12y ago

> […] managers […] should be trained to respect opinions of subordinates. That's your problem right there: the subordinacy of engineers. We are biased to listen to high-status people more than low-status people. The strength of this bias varies from people to people, and from culture to culture. But if you are asking someone to listen to the opinion of a direct subordinate, you are asking for trouble. This is one reason why Michael O. Church is so big on guilds, or "professions". If instead of being a subordinate, the engineer was a (possibly certified) master craftsman, things would be different. Just picture the manager calling the the engineer "master", then not listen to her opinion. > […] engineers […] should be trained in how to negotiate with management Conversely, it's harder to negotiate when you have lower status than the manager. You may feel it is not your place. But if you're a master programmer, offering your services to said manager, then you could feel more confident about talking to him as a peer. You could feel more confident about saying things like "your idea won't work —trust me, it's my speciality."

lovelearning 12y ago
I agree with you. I'd like to live and work in such a world myself. But my rather pessimistic opinion is that status, hierarchy, etc are deeply embedded in us as a species - just like animal species who have pack leaders or dominant males or whatever - and this won't be easy to get rid of. The ones who hold financial strings, the ones who act more arrogant, more confident, more powerful, more knowledgeable - they are unconsciously seen as having higher status.
- ppereira 12y ago
  The hierarchy could be inverted with a very small change to the laws governing engineers, just as it has been done for lawyers. The details vary by jurisdiction, but generally lawyers must practice in some form of partnership where non-lawyer members are restricted. Work similar to project management is frequently performed by secretaries that are subordinate to the lawyers. Whether or not strengthening the profession of engineering would be good for society is another question. It is inherently anti-competitive. It could easily slip into a system with very few and highly paid "qualified" engineers whose small number would hinder development. This is a problem with medicine and the supply of doctors in some places.

brudgers 12y ago

That's one way of looking at it. Another is engineers wanting to reinvent the wheel. Ignition switch springs are a solved problem, but specifying a new system smells more like cleverness than using a part from the warehouse shelf. In manufacturing, production lines at all the tiers have to retool at the right time. This year's new models began design over a decade or more ago.

technofiend 12y ago

It's an understandable byproduct of a stack ranked organization. Any time you have the threat of being ranked and rewarded based on delivery well anything that threatens the delivery reflects badly on the rankee and is actively avoided.

zaroth 12y ago
The threat of being ranked? We are ranked constantly starting at 1st grade. Competitive ranking should be a very familiar feeling for every American at least. When humans face the prospect of being ranked, the only option then becomes to lie, cheat, take shortcuts, or put lives at risk?! It may be an understandable byproduct for the weak, the proud, or the immature. Cheating is certainly endemic in our society, but I don't believe that absolves the cheaters; it's still criminal/unethical and it should still get you fired, throw off the team, or sent to jail. There is something important to study and learn from this disaster, but my personal opinion is it's not so much a problem with performance evaluations, but rather a spectacular engineering failure combined with a company rotten to the core that completely lost its sense of purpose (hint: serving customers, not short term profit). I do expect that adult humans can be subjected to significant workplace stress and pressure and still be expected to behave ethically and responsibly. I think that the law and justice expect the same.

zaroth 12y ago

Did you read the email from the chief switch engineer about the 'switch from hell'? Seems like this guy seriously fucked up and then tried to cover it up and then continued to lie about it at depositions? Management is absolutely also to blame but it all started with the engineer. His negligence and 'fuck this switch' attitude killed people. He failed technically and ethically to do his job and people died.

lovelearning 12y ago
As reported in [1], it looks like he wasn't negligent - "DeGiorgio discussed the problem with Delphi...in February 2002, he had a choice: do nothing to fix it or change the switch and delay production." He certainly knew about the problem, but then decided to not fix it, to keep the production schedule on track. This is what makes me talk about management and deadlines. It's easy to blame it all on that single engineer - and I'm sure GM would love to do so - but isn't it possible the environment in which he worked made it difficult - even risky - to implement the right solution? Perhaps biased by some of my own experiences, I tend to favor the organisational failure explanation that other comments here describe quite well. Of course, everything he did later - the cover up, the lies - were attempts to cover up this one bad decision, and show that his ethics were less than stellar. [1]: http://fortune.com/2014/06/11/gm-switch-ray-degiorgio/ http://fortune.com/2014/06/11/gm-switch-ray-degiorgio/
- zaroth 12y ago
  I guess I see it differently. Management definitely failed to react properly once it became obvious the switch was faulty. I mean, apparently it came up in car reviews of all places. However, if you're the lead engineer on the switch, you have 23 years of experience on the job, it's your responsibility not to say "fuck it, just ship it." It's his signature on the paperwork. This wasn't a one-time one-day bad decision. This is a switch he worked on for three years. The right course of action is refuse to sign off on a faulty part, and then not try to conceal that the part was faulty once people start dying. Every project has schedule pressure, so if schedule pressure is somehow a defense then engineers are never responsible. I simply don't buy that. Management can decide they want to override the engineer and ship it anyway. GM can fire the guy for not agreeing to ship a faulty part, but then DeGiorgio could have been a whistle blower collecting 1/3rd of the damages against GM right now. I don't blame it singularly on the engineer. TFA says that GM’s Director of Product Investigations shut off the car with her knee, so this goes way beyond just the engineer. But I don't think that lets the engineer off the hook either. The engineer failed spectacularly, possibly criminally, and I hope he has to face a jury of his peers for this. Management also failed the same way, and if the investigators were in place to really push this, I'm sure the paper trail is there to at least indite some of these greedy complacent fools.