6 ms·

Some programmer - or team of programmers - implemented this software. Sure, it may have been a PHB who came up with the idea and told them to code this into th

by edent 11y ago

Some programmer - or team of programmers - implemented this software. Sure, it may have been a PHB who came up with the idea and told them to code this into the algorithm - but it was programmers like you or I who actually cranked out the code that made this possible. Can you imagine this happening with, say, an architect? When unsafe buildings fall down, it makes international news. By and large, professionals have a code of conduct which they must follow or there is a very real chance that they will lose their livelihood. I'm a Member of the British Computing Society - it has a fairly simple code of conduct for members. http://www.bcs.org/category/6030 http://www.bcs.org/category/6030 I can certainly see adding this functionality would probably be a breach of ... > 1 a) have due regard for public health, privacy, security and wellbeing of others and the environment. > 2 d) ensure that you have the knowledge and understanding of Legislation* and that you comply with such Legislation, in carrying out your professional responsibilities. > 2 f) avoid injuring others, their property, reputation, or employment by false or malicious or negligent action or inaction. > 3 e) NOT misrepresent or withhold information on the performance of products, systems or services (unless lawfully bound by a duty of confidentiality not to disclose such information), or take advantage of the lack of relevant knowledge or inexperience of others. But, here's the kicker - if I were kicked out of the BCS for adding this code, nothing would happen to me. Employers don't care about professional bodies - except in terms of certification and, possibly, indemnity. I'm quite happy being a member of a Trade Union, because I believe it offers me the best protection against a malicious employer - I wonder how long before more codes start joining professional bodies to help protect themselves from being asked to act counter to their best interests?

TheCapn 11y ago
In the same vein is why I advocate for "Software Engineering" being brought into the realm of other engineers and having the status protected so people who call themselves engineers are held to the ethics and governing regulations that all engineers are. Holding the P.Eng. designation myself I would be liable to lose my license had I published this software, I really hope we can reach a point in our profession where a higher standard of care to public and environnment is not only expected, but enforced.
- sbov 11y ago
  I see this a lot. And I generally agree. However... For software it's hard to see where to draw the line. For other engineering disciplines this can be pretty easy. I can release a little mechanical toy and not be an engineer. The public won't try to use it to drive around town. But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves. I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)? When I first started programming I built a D&D utility for a friend. He passed it to his friends. Soon around 10 of them used it. If it had a serious bug, should I have been prosecuted? Should I have been unable to build it for him in the first place? If you say all of this activity should be restricted, how do you get from here to there? Some of the best "software engineers" in the world aren't technically software engineers.
  - ams6110 11y ago
    Software you write for fun or to pass around to friends would (should) have the standard disclaimer of implied warranty of suitability. Software developed to control an automotive emissions system by a manufacturer and in the context of being used for that purpose would not be able to escape that.
  - krschultz 11y ago
    This is a very common counter argument to software engineering as a discipline. > But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves. That is a legitimately unique aspect of software and deserves consideration. > I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)? Likely some portions of the system matter and some don't. Which ones likely require engineering judgement. For physical products it's pretty easy to make that call for a paper weight and for an airplane, but all the products in between need that decision. We are generally building software systems that are about as complex as a Black & Decker power drill. When designing something like that you need to make damn well sure you are not electrocuting the user, but if the gears break and it doesn't function you aren't hurting anyone. Therefore most drills are 'double insulated', but there is no backup for the actual function. In software we should make that same call. What is the consequence of failure? If it's small then we live with it. If it's not, then we should build a fail safe / backup system. I'm super in favor of software engineering licensing and all that it entails, but I do think large swathes of software are completely immune because failure has no consequences. Your D&D utility is an example. Most open source licenses rightly have a clause that should protect them from civil lawsuits, much less prosecution.
thoman23 11y ago
Well said! I did think about the programmer or programmers when this story first broke. I am not shocked that some executive would come up with the idea...but I am shocked that someone would actually code it. You have to wonder how many people knew and said nothing.
- A_Beer_Clinked 11y ago
  I could imagine a scenario where this started out as something less evil. Seems to me that cars will perform differently in a lab than on the road; off the top of my head: - Stationary cars have different air-intake characteristics - The temperature of the fuel and air is likely different - The humidity of the air might be different - The fuel in the tank is stationary not sloshing around There may well be valid reasons for the software to vary engine parameters test mode; maybe to compensate for the above. Then sometime later someone said "This NOx Reduction module is killing fuel efficiency on the open road, how about we turn it off" The net result is the same but motives and intentions of the individuals are important.
afarrell 11y ago
What steps do people take to comply with #2d ? Does the British Computing Society maintain a service which keeps people up to date with legislation and relevant court rulings and where you can ask hypothetical questions of actual lawyers?
- gregmac 11y ago
  This one does seem quite onerous: how do you even know what legislation applies to the software you're working on, to even ask the right questions? There are already specialists in legislation (we call them lawyers), so why should the programmers take on this responsibility? I agree with the spirit of the rule, but I don't think it makes sense as it's basically asking all programmers to become lawyers. It should be more along the lines of "comply with legislation you are (made) aware of" and require some due diligence like "solicit the services of a lawyer if there is a reasonable belief there is legislation that could apply".
  - edent 11y ago
    That's part of your professional responsibility. How do you know not to store passwords in plain-text? How do you know that you have to make your software accessible to people with visual impairments? I honestly don't think you need a legal degree to know that "dramatically alter how the pollution controls works when in a test environment" is likely to be extremely dodgy! Now, maybe these programmers did flag this as a problem. And, maybe VW's high-priced lawyers said it was OK. In which case, responsibility is probably shifted. I'm not saying every programmer should be a lawyer. But you can't say "how was I supposed to know that running a red light while drunk is illegal?! I'm just a delivery driver - not a lawyer."
- edent 11y ago
  I do receive emails from them talking about policy changes - and the BCS's position on government consultations. They also have a number of reports & white papers which members can read. Members also have access to a legal helpline. Don't think I can link to it, but it says... > As part of the ongoing commitment from BCS to invest in services and benefits, the BCS has launched a new and improved Legal Advisory Service comprising two new online legal information websites and a dedicated telephone helpline for both Business and Personal legal issues (restrictions apply*). These services are totally FREE and are provided by Law Express, the UK's leading specialist provider of legal advice. I've not had cause to use it, so I can't comment on how well it works in practice.
jsprogrammer 11y ago
Do we have the bad VW code? Do we know it happened how you allege? Do we know that the code was written by hand and not auto-generated based on real-world testing/calibration? Do we know that a programmer has final say on the code that is shipped? Are you sure management couldn't select from a number of programs based on the criteria they like, without needing additional input from a programmer? Why didn't management review the codes and put in place procedures to ensure the codes reviewed are the ones actually being used? Why didn't management implement testing procedures so that the reviewed codes could be verified? Do you have the specs/requirements the programmers were given? Edit: Really? Downmods?
jonah 11y ago
The engine control software in question was produced by Bosch. They say they warned Volkswagen that the feature in question was for development only and not to be used in production.
- cptskippy 11y ago
  It seems unlikely that Bosch would implemented an algorithm to detect an emissions test and alter the engine behavior. It seems much more likely that Bosch implemented diagnostic features to force the engine into a particular operating mode (e.g. Enable/Disable EGR) and that Volkswagen activated those features when it detected certain qualifying criteria (i.e. emissions test conditions). Unless there's direct evidence implicating Bosch, then blaming them for the cheating is like blaming a bullet manufacturer when you get caught shooting people.
- logfromblammo 11y ago
  Was it a stern warning, loaded with gravitas, or was it accompanied with a wink that passed unnoticed by the corporate attorneys? Bosch might have sold them the pistol, but VW is still the one that shot itself in the foot with it. But was the pistol pre-loaded with toe-seeking bullets, or did VW carefully aim at its own extremity before firing? There is still some question in my mind as to how complete a defeat device package Bosch shipped. If there were several separate functions like LowestEmissions() and LowestFuelConsumption() and LowestMaintenance() and BestPerformance() and LowAmbientTemp() and HighAmbientTemp(), that's quite a bit different from DetectEmissionsTest() or OperateAsDefeatDevice(). Who was it that wrote the code to switch modes when an emissions test was detected?
therobot24 11y ago
> Sure, it may have been a PHB who came up with the idea and told them to code this into the algorithm - but it was programmers like you or I who actually cranked out the code that made this possible. So i think there's been a lot of assumption that the software engineers made this code explicitly for production. For all we know they were making what they thought was code for specific in-house experiments and a subset or different set knowingly kept it for production.
- omouse 11y ago
  Actually that was the purpose stated; they developed it for testing in-house and it makes sense to do so. However, at some point there must have been someone who was a front-line employee who could have blown the whistle on this kind of disgusting ethics violation.
- ams6110 11y ago
  That's possible, but I find it to be a stretch. Unless one group were told "write some code to meet this emission standard" and another were told to "write some code to detect when emmission testing is happening" that could be the case. But somewhere along the line someone was given the task to combine those and "disable the emission controls unless testing is happening." In other scenarios, yeah it's perfectly possible for an developer to complete a task that seems innocent enough and unbeknownst to him will later become a part of something nefarious.
SimonPStevens 11y ago
It's not accurate to compare this to a building falling down. Nothing went wrong, the code did exactly what it was designed to do.
draw_down 11y ago
There's a whole business structure that drove and enabled this decision, it wasn't just a bunch of engineers who had a bright idea one day. (Not meant to be exculpatory for the people who worked on this.) But you never hear anyone going around calling for businesspeople to conduct themselves ethically.
amelius 11y ago
Your story may not be 100% accurate, but still, it may be a good idea to introduce a "Hippocratic oath" for programmers, or engineers in general.
omouse 11y ago
I'm a member of the ACM (Association of Computing Machinery) and every year I read over the IEEE-ACM Software Engineering Code Of Ethics: https://www.acm.org/about/se-code https://www.acm.org/about/se-code Due to manager and business pressure to make more money, multiple sections are subverted and there's absolutely no consequence to them. The only consequence they understand is if programmers start to refuse jobs or tasks or quit based on violations of the code. This is sometimes why I would favour certification or more responsibility placed on the programmer. Some programs should not be written or if they are written they should be held to a very high standard. We wouldn't have issues with high frequency trading if it cost a lot to provide insurance for that kind of development. Sure simple websites don't really need high standards, but as soon as you're past 1 million in revenue you're dealing with real money, real data and any bugs could have serious consequences. On the other hand, I like being able to write little scripts that say "hey if something breaks, it ain't my fault".
hinkley 11y ago
From what I understand of the situation, It's entirely likely that the devs got a rule table from the engine and emissions guys and couldn't actually tell if the metrics were good or sketchy as hell. It sounds like the main problem is with high fidelity O2 sensors at a cheap enough price point that these lookup tables can be banished. We legislated people carrying a block of platinum around in their car. Maybe $200 O2 sensors is the next step if the manufacturers can't be trusted.

by edent 11y ago

TheCapn 11y ago

In the same vein is why I advocate for "Software Engineering" being brought into the realm of other engineers and having the status protected so people who call themselves engineers are held to the ethics and governing regulations that all engineers are. Holding the P.Eng. designation myself I would be liable to lose my license had I published this software, I really hope we can reach a point in our profession where a higher standard of care to public and environnment is not only expected, but enforced.

sbov 11y ago
I see this a lot. And I generally agree. However... For software it's hard to see where to draw the line. For other engineering disciplines this can be pretty easy. I can release a little mechanical toy and not be an engineer. The public won't try to use it to drive around town. But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves. I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)? When I first started programming I built a D&D utility for a friend. He passed it to his friends. Soon around 10 of them used it. If it had a serious bug, should I have been prosecuted? Should I have been unable to build it for him in the first place? If you say all of this activity should be restricted, how do you get from here to there? Some of the best "software engineers" in the world aren't technically software engineers.
- ams6110 11y ago
  Software you write for fun or to pass around to friends would (should) have the standard disclaimer of implied warranty of suitability. Software developed to control an automotive emissions system by a manufacturer and in the context of being used for that purpose would not be able to escape that.
- krschultz 11y ago
  This is a very common counter argument to software engineering as a discipline. > But if I make a pet software project millions of people might decide to use it to "drive around town", and some portion will hurt themselves. That is a legitimately unique aspect of software and deserves consideration. > I think that if you build stuff for certain things (such as 1 ton objects zooming around at 70+ miles per hour), it's obvious. But what about financial software? What about social networks (privacy issues)? Likely some portions of the system matter and some don't. Which ones likely require engineering judgement. For physical products it's pretty easy to make that call for a paper weight and for an airplane, but all the products in between need that decision. We are generally building software systems that are about as complex as a Black & Decker power drill. When designing something like that you need to make damn well sure you are not electrocuting the user, but if the gears break and it doesn't function you aren't hurting anyone. Therefore most drills are 'double insulated', but there is no backup for the actual function. In software we should make that same call. What is the consequence of failure? If it's small then we live with it. If it's not, then we should build a fail safe / backup system. I'm super in favor of software engineering licensing and all that it entails, but I do think large swathes of software are completely immune because failure has no consequences. Your D&D utility is an example. Most open source licenses rightly have a clause that should protect them from civil lawsuits, much less prosecution.

thoman23 11y ago

Well said! I did think about the programmer or programmers when this story first broke. I am not shocked that some executive would come up with the idea...but I am shocked that someone would actually code it. You have to wonder how many people knew and said nothing.

A_Beer_Clinked 11y ago
I could imagine a scenario where this started out as something less evil. Seems to me that cars will perform differently in a lab than on the road; off the top of my head: - Stationary cars have different air-intake characteristics - The temperature of the fuel and air is likely different - The humidity of the air might be different - The fuel in the tank is stationary not sloshing around There may well be valid reasons for the software to vary engine parameters test mode; maybe to compensate for the above. Then sometime later someone said "This NOx Reduction module is killing fuel efficiency on the open road, how about we turn it off" The net result is the same but motives and intentions of the individuals are important.

afarrell 11y ago

What steps do people take to comply with #2d ? Does the British Computing Society maintain a service which keeps people up to date with legislation and relevant court rulings and where you can ask hypothetical questions of actual lawyers?

gregmac 11y ago
This one does seem quite onerous: how do you even know what legislation applies to the software you're working on, to even ask the right questions? There are already specialists in legislation (we call them lawyers), so why should the programmers take on this responsibility? I agree with the spirit of the rule, but I don't think it makes sense as it's basically asking all programmers to become lawyers. It should be more along the lines of "comply with legislation you are (made) aware of" and require some due diligence like "solicit the services of a lawyer if there is a reasonable belief there is legislation that could apply".
- edent 11y ago
  That's part of your professional responsibility. How do you know not to store passwords in plain-text? How do you know that you have to make your software accessible to people with visual impairments? I honestly don't think you need a legal degree to know that "dramatically alter how the pollution controls works when in a test environment" is likely to be extremely dodgy! Now, maybe these programmers did flag this as a problem. And, maybe VW's high-priced lawyers said it was OK. In which case, responsibility is probably shifted. I'm not saying every programmer should be a lawyer. But you can't say "how was I supposed to know that running a red light while drunk is illegal?! I'm just a delivery driver - not a lawyer."
edent 11y ago
I do receive emails from them talking about policy changes - and the BCS's position on government consultations. They also have a number of reports & white papers which members can read. Members also have access to a legal helpline. Don't think I can link to it, but it says... > As part of the ongoing commitment from BCS to invest in services and benefits, the BCS has launched a new and improved Legal Advisory Service comprising two new online legal information websites and a dedicated telephone helpline for both Business and Personal legal issues (restrictions apply*). These services are totally FREE and are provided by Law Express, the UK's leading specialist provider of legal advice. I've not had cause to use it, so I can't comment on how well it works in practice.

jsprogrammer 11y ago

Do we have the bad VW code? Do we know it happened how you allege? Do we know that the code was written by hand and not auto-generated based on real-world testing/calibration? Do we know that a programmer has final say on the code that is shipped? Are you sure management couldn't select from a number of programs based on the criteria they like, without needing additional input from a programmer? Why didn't management review the codes and put in place procedures to ensure the codes reviewed are the ones actually being used? Why didn't management implement testing procedures so that the reviewed codes could be verified? Do you have the specs/requirements the programmers were given? Edit: Really? Downmods?

jonah 11y ago

The engine control software in question was produced by Bosch. They say they warned Volkswagen that the feature in question was for development only and not to be used in production.

cptskippy 11y ago
It seems unlikely that Bosch would implemented an algorithm to detect an emissions test and alter the engine behavior. It seems much more likely that Bosch implemented diagnostic features to force the engine into a particular operating mode (e.g. Enable/Disable EGR) and that Volkswagen activated those features when it detected certain qualifying criteria (i.e. emissions test conditions). Unless there's direct evidence implicating Bosch, then blaming them for the cheating is like blaming a bullet manufacturer when you get caught shooting people.
logfromblammo 11y ago
Was it a stern warning, loaded with gravitas, or was it accompanied with a wink that passed unnoticed by the corporate attorneys? Bosch might have sold them the pistol, but VW is still the one that shot itself in the foot with it. But was the pistol pre-loaded with toe-seeking bullets, or did VW carefully aim at its own extremity before firing? There is still some question in my mind as to how complete a defeat device package Bosch shipped. If there were several separate functions like LowestEmissions() and LowestFuelConsumption() and LowestMaintenance() and BestPerformance() and LowAmbientTemp() and HighAmbientTemp(), that's quite a bit different from DetectEmissionsTest() or OperateAsDefeatDevice(). Who was it that wrote the code to switch modes when an emissions test was detected?

therobot24 11y ago

> Sure, it may have been a PHB who came up with the idea and told them to code this into the algorithm - but it was programmers like you or I who actually cranked out the code that made this possible. So i think there's been a lot of assumption that the software engineers made this code explicitly for production. For all we know they were making what they thought was code for specific in-house experiments and a subset or different set knowingly kept it for production.

omouse 11y ago
Actually that was the purpose stated; they developed it for testing in-house and it makes sense to do so. However, at some point there must have been someone who was a front-line employee who could have blown the whistle on this kind of disgusting ethics violation.
ams6110 11y ago
That's possible, but I find it to be a stretch. Unless one group were told "write some code to meet this emission standard" and another were told to "write some code to detect when emmission testing is happening" that could be the case. But somewhere along the line someone was given the task to combine those and "disable the emission controls unless testing is happening." In other scenarios, yeah it's perfectly possible for an developer to complete a task that seems innocent enough and unbeknownst to him will later become a part of something nefarious.

SimonPStevens 11y ago

It's not accurate to compare this to a building falling down. Nothing went wrong, the code did exactly what it was designed to do.

draw_down 11y ago

There's a whole business structure that drove and enabled this decision, it wasn't just a bunch of engineers who had a bright idea one day. (Not meant to be exculpatory for the people who worked on this.) But you never hear anyone going around calling for businesspeople to conduct themselves ethically.

amelius 11y ago

Your story may not be 100% accurate, but still, it may be a good idea to introduce a "Hippocratic oath" for programmers, or engineers in general.

omouse 11y ago

I'm a member of the ACM (Association of Computing Machinery) and every year I read over the IEEE-ACM Software Engineering Code Of Ethics: https://www.acm.org/about/se-code https://www.acm.org/about/se-code Due to manager and business pressure to make more money, multiple sections are subverted and there's absolutely no consequence to them. The only consequence they understand is if programmers start to refuse jobs or tasks or quit based on violations of the code. This is sometimes why I would favour certification or more responsibility placed on the programmer. Some programs should not be written or if they are written they should be held to a very high standard. We wouldn't have issues with high frequency trading if it cost a lot to provide insurance for that kind of development. Sure simple websites don't really need high standards, but as soon as you're past 1 million in revenue you're dealing with real money, real data and any bugs could have serious consequences. On the other hand, I like being able to write little scripts that say "hey if something breaks, it ain't my fault".

hinkley 11y ago

From what I understand of the situation, It's entirely likely that the devs got a rule table from the engine and emissions guys and couldn't actually tell if the metrics were good or sketchy as hell. It sounds like the main problem is with high fidelity O2 sensors at a cheap enough price point that these lookup tables can be banished. We legislated people carrying a block of platinum around in their car. Maybe $200 O2 sensors is the next step if the manufacturers can't be trusted.