4 ms·

Except that the start and goal are so far away you can legitimately claim you can't see the actual harm in writing something. You write a tool for let's say re

by Ygg2 6y ago

Except that the start and goal are so far away you can legitimately claim you can't see the actual harm in writing something. You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. What if you start as something 100% ethical. But your company pivots to unethical application?

tfehring 6y ago
This is a great point, and reasonable people can disagree about when an application's abuse or potential for abuse crosses the line. The same goes for pivots or for general-purpose code that's used elsewhere. (Is it ethical to contribute to internal tools at Facebook? What if those tools make other engineers way more effective at doing things that ultimately undermine democratic systems?) My point here isn't to dictate what software is or isn't ethical, but to argue that if a program is unethical, its ethical implications are the responsibility of the engineer(s) who wrote it.
- deleted 6y ago
  [deleted]
BeetleB 6y ago
> You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. I mentioned this in another comment, but I'll say it again: Irrespective of any legal/ethical concerns, yes, I would like to know! If my boss just came to me and said "build a facial recognition system" I certainly will ask how it is going to be used. Not because I care about ethics, but it's a basic aspect of the job. You can replace "facial recognition" with "CMS" and I'd still ask. If they tell me the facial recognition is for logging into computers, and then later decide to use it to track dissidents, that is a different concern. But I'll at least ask! > What if you start as something 100% ethical. But your company pivots to unethical application? If they pivot after my work is done, I won't feel responsible. If they never used it for the original application and pivoted to this, I may get upset and quit, but my conscience would be clear.
- Ygg2 6y ago
  > If they pivot after my work is done, I won't feel responsible. So if you invented dynamite you wouldn't feel responsible for its use? But, let's change it a bit more personal. You write an awesome OSS yaml parser. It's so good, that GFW of China uses it as a main component, and this gets published in the news. What would you do? Nothing you did changed, but suddenly your work is powering an unethical component.
  - Nasrudith 6y ago
    Dynamite is a good example of why the philosophy is complete bullshit. How about you blame the stupid evil fuckers who started all of the futile wars to try to get rich in the quagmire that was European geopolitics instead of tbe person who made them safer? It is the same sort of stupid blame shifting involved with the hippocratic oath for x nonsense. Oaths are majorly outmoded in the zeitgeist anyway because everyone recognizes lies are commonplace.
    - Ygg2 6y ago
      > How about you blame the stupid evil fuckers who started all of the futile wars And I fully agree. Expecting people to individually bear the burden of "some oath", is a fool's errand. My point was software on its own, much like a fridge, is amoral. You can use it to store your groceries, or you can use it to store corpses. That said, there are some extreme cases (like a gun), that have very limited non-violent uses. And IMO, that should be regulated, instead of depending on people Doing The Right Thing™.
  - BeetleB 6y ago
    > So if you invented dynamite you wouldn't feel responsible for its use? I would if I were inventing dynamite, but that's not what this scenario is. A person working for a knife manufacturer need not worry about it being used for murder, as that's not what the primary use. And facial recognition is a lot less harmful than even that. Trust me: I work for a company that produces certain goods used for all kinds of good and nefarious purposes depending on who buys it. My conscience is clear. > But, let's change it a bit more personal. You write an awesome OSS yaml parser. It's so good, that GFW of China uses it as a main component, and this gets published in the news. > What would you do? Nothing you did changed, but suddenly your work is powering an unethical component. I wouldn't do anything: 1. This is milder than the knife scenario above. Of course I don't care if people use it in a poor way - unless there is a straightforward technical mitigation I could do. In your example, given that the source code is available, that is not an option. 2. There's a certain hypocrisy in releasing something as open source and then complaining about how it is used. If it bothers you, then modify your license!

by Ygg2 6y ago

tfehring 6y ago

This is a great point, and reasonable people can disagree about when an application's abuse or potential for abuse crosses the line. The same goes for pivots or for general-purpose code that's used elsewhere. (Is it ethical to contribute to internal tools at Facebook? What if those tools make other engineers way more effective at doing things that ultimately undermine democratic systems?) My point here isn't to dictate what software is or isn't ethical, but to argue that if a program is unethical, its ethical implications are the responsibility of the engineer(s) who wrote it.

deleted 6y ago
[deleted]

BeetleB 6y ago

> You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. I mentioned this in another comment, but I'll say it again: Irrespective of any legal/ethical concerns, yes, I would like to know! If my boss just came to me and said "build a facial recognition system" I certainly will ask how it is going to be used. Not because I care about ethics, but it's a basic aspect of the job. You can replace "facial recognition" with "CMS" and I'd still ask. If they tell me the facial recognition is for logging into computers, and then later decide to use it to track dissidents, that is a different concern. But I'll at least ask! > What if you start as something 100% ethical. But your company pivots to unethical application? If they pivot after my work is done, I won't feel responsible. If they never used it for the original application and pivoted to this, I may get upset and quit, but my conscience would be clear.

Ygg2 6y ago
> If they pivot after my work is done, I won't feel responsible. So if you invented dynamite you wouldn't feel responsible for its use? But, let's change it a bit more personal. You write an awesome OSS yaml parser. It's so good, that GFW of China uses it as a main component, and this gets published in the news. What would you do? Nothing you did changed, but suddenly your work is powering an unethical component.
- Nasrudith 6y ago
  Dynamite is a good example of why the philosophy is complete bullshit. How about you blame the stupid evil fuckers who started all of the futile wars to try to get rich in the quagmire that was European geopolitics instead of tbe person who made them safer? It is the same sort of stupid blame shifting involved with the hippocratic oath for x nonsense. Oaths are majorly outmoded in the zeitgeist anyway because everyone recognizes lies are commonplace.
  - Ygg2 6y ago
    > How about you blame the stupid evil fuckers who started all of the futile wars And I fully agree. Expecting people to individually bear the burden of "some oath", is a fool's errand. My point was software on its own, much like a fridge, is amoral. You can use it to store your groceries, or you can use it to store corpses. That said, there are some extreme cases (like a gun), that have very limited non-violent uses. And IMO, that should be regulated, instead of depending on people Doing The Right Thing™.
- BeetleB 6y ago
  > So if you invented dynamite you wouldn't feel responsible for its use? I would if I were inventing dynamite, but that's not what this scenario is. A person working for a knife manufacturer need not worry about it being used for murder, as that's not what the primary use. And facial recognition is a lot less harmful than even that. Trust me: I work for a company that produces certain goods used for all kinds of good and nefarious purposes depending on who buys it. My conscience is clear. > But, let's change it a bit more personal. You write an awesome OSS yaml parser. It's so good, that GFW of China uses it as a main component, and this gets published in the news. > What would you do? Nothing you did changed, but suddenly your work is powering an unethical component. I wouldn't do anything: 1. This is milder than the knife scenario above. Of course I don't care if people use it in a poor way - unless there is a straightforward technical mitigation I could do. In your example, given that the source code is available, that is not an option. 2. There's a certain hypocrisy in releasing something as open source and then complaining about how it is used. If it bothers you, then modify your license!