3 ms·

I wholly object to this line of reasoning. I don't see how this wouldn't blow-back on software developers and make them an easier scapegoat. Take these car man

I wholly object to this line of reasoning. I don't see how this wouldn't blow-back on software developers and make them an easier scapegoat. Take these car manufacturers gaming the emissions test, for example. I find it very easy to imagine management making a series of requests for changes to the software that seem innocent but allow for this exact outcome: changing the way the system performs when being tested for emissions. They may have asked the development team to add some kind of a switch so that they could toggle the behavior on and off in order to assess how overall performance was impacted. That sounds innocent enough to me that I wouldn't be eager to risk my job (and perhaps the ability to get work in the future) over suspicions. Certainly it's not enough that I'd try to get law enforcement involved. This idea that software developers (or engineers, if you prefer that term) should be responsible for the entire system, even though they likely only work on one component, sounds to my ear like nothing more than setting those developers up to be easy scapegoats. If you wanted to get me on board in any way, I'd need to see some system where all code is signed and, perhaps, stored in some tamper-proof write-only storage system managed by a third party. I'd need to be able to verify beyond any doubt that code a particular developer wrote hasn't been tampered with before I'd be willing to send that developer to prison, or at the least, ruin their career. Management is at fault for requesting the feature that enabled them to game the system. Management should pay the cost, I see no reason to cast any blame on developers. I have yet to see any compelling evidence that the developers knowingly added code specifically to game the system. If evidence of that is out there, I would be super interested in reading it and would appreciate any links anyone might have.

geodel 8y ago
You are right. From what I have seen that most of the time even quite senior engineers are not aware of larger implications beyond technical architecture of software they are developing.
Zimahl 8y ago
> I find it very easy to imagine management making a series of requests for changes to the software that seem innocent but allow for this exact outcome: changing the way the system performs when being tested for emissions. Off the top of my head, I've written many pieces of code that 'game the system' but are used only for testing purposes. I know full well that if those pieces get into production they could be hazardous, but there is nothing ethically wrong with me creating them. If a superior takes that code and uses it, that's not my fault. It would be supremely easy for management to put in a bunch of requirements for the VW emissions system that are blatantly dangerous, but 'for testing purposes'. The developers need to be able to trust the higher-ups and certainly shouldn't be held responsible for their decisions.
draw_down 8y ago
Exactly! Thank you!!
godelski 8y ago
To clarify on my OP, I want to say that there is a distinction between a legal and ethical obligation. In engineering only a PE really has the legal obligation. Their on the hook if something goes wrong. But at the same time everyone involved has an ethical obligation to ensure that best practices are being followed and nothing is being done that would result in harm. This is even if they wouldn't get in any legal trouble. Or that a doctor has both legal and ethical obligations. Their ethical obligations can many times supersede their legal ones. I am not arguing that we who write software need to be held legally accountable, but that we need to think about any potential ethical concerns with what we write. And if you are saying "well yeah" then we're in agreement. And I think this should be pervasive. I think we should frequently talk about it. But we are in high demand right now. Many of us have the leverage to say "I don't agree with this" and go work somewhere else. If you personally have that ability I think it should be used as well. But I understand not everyone has that leverage.
- cmiles74 8y ago
  I think I understand what you're saying and I agree that no software developer should willingly write code that will be used to break or circumvent the law. I believe that if I was put in such a position that I would walk rather than implement the feature. I don't think we are alone in that feeling, I believe many developers feel the same way. I worry that a feature meant to circumvent or break the law will not be easy to spot. Many projects are broken into smaller pieces that interact, many times the laws or rules change from state to state. Even in regular day to day work I'll see a feature request that effectively undoes another request. Often developers are working in areas in which they are not subject experts. I think it's easy to write code that could (or even will) be misused; it may not look troubling until it's interacting with the larger system.
  - godelski 8y ago
    But that's why I'm arguing that we have to think about it. As well as why I'm not arguing for legal accountability. We have to be aware that the things we create aren't always going to be used for what we intend them to be used for. So you frequently have to ask yourself "how could this be abused?" That is something that engineers frequently ask themselves. I don't understand why developers can't be asked to do the same.

geodel 8y ago

You are right. From what I have seen that most of the time even quite senior engineers are not aware of larger implications beyond technical architecture of software they are developing.

Zimahl 8y ago

> I find it very easy to imagine management making a series of requests for changes to the software that seem innocent but allow for this exact outcome: changing the way the system performs when being tested for emissions. Off the top of my head, I've written many pieces of code that 'game the system' but are used only for testing purposes. I know full well that if those pieces get into production they could be hazardous, but there is nothing ethically wrong with me creating them. If a superior takes that code and uses it, that's not my fault. It would be supremely easy for management to put in a bunch of requirements for the VW emissions system that are blatantly dangerous, but 'for testing purposes'. The developers need to be able to trust the higher-ups and certainly shouldn't be held responsible for their decisions.

draw_down 8y ago

Exactly! Thank you!!

godelski 8y ago

To clarify on my OP, I want to say that there is a distinction between a legal and ethical obligation. In engineering only a PE really has the legal obligation. Their on the hook if something goes wrong. But at the same time everyone involved has an ethical obligation to ensure that best practices are being followed and nothing is being done that would result in harm. This is even if they wouldn't get in any legal trouble. Or that a doctor has both legal and ethical obligations. Their ethical obligations can many times supersede their legal ones. I am not arguing that we who write software need to be held legally accountable, but that we need to think about any potential ethical concerns with what we write. And if you are saying "well yeah" then we're in agreement. And I think this should be pervasive. I think we should frequently talk about it. But we are in high demand right now. Many of us have the leverage to say "I don't agree with this" and go work somewhere else. If you personally have that ability I think it should be used as well. But I understand not everyone has that leverage.

cmiles74 8y ago
I think I understand what you're saying and I agree that no software developer should willingly write code that will be used to break or circumvent the law. I believe that if I was put in such a position that I would walk rather than implement the feature. I don't think we are alone in that feeling, I believe many developers feel the same way. I worry that a feature meant to circumvent or break the law will not be easy to spot. Many projects are broken into smaller pieces that interact, many times the laws or rules change from state to state. Even in regular day to day work I'll see a feature request that effectively undoes another request. Often developers are working in areas in which they are not subject experts. I think it's easy to write code that could (or even will) be misused; it may not look troubling until it's interacting with the larger system.
- godelski 8y ago
  But that's why I'm arguing that we have to think about it. As well as why I'm not arguing for legal accountability. We have to be aware that the things we create aren't always going to be used for what we intend them to be used for. So you frequently have to ask yourself "how could this be abused?" That is something that engineers frequently ask themselves. I don't understand why developers can't be asked to do the same.