7 ms·

Writing software is not a neutral act. You are morally responsible for its uses. Restricting those uses is both a responsibility and a good. Circumventing this

Writing software is not a neutral act. You are morally responsible for its uses. Restricting those uses is both a responsibility and a good. Circumventing this restriction is itself an immoral act.

tkzed49 6y ago
So if I just write "you're not allowed to do bad things with this" in the license agreement, I'm covered?
nathanaldensr 6y ago
No, you're not. This is an argument that doesn't hold water. Why am I morally responsible but none of the abstractions below my code are not? What about chip manufacturers? The developers of CPU instruction sets? Compiler developers?
- iandanforth 6y ago
  They are. Why wouldn't they be? We put people in jail for distributing potentially harmful tools to people who are known to desire to harm others. This is not a new or controversial idea.
  - umvi 6y ago
    Name one tool that has not been used to kill someone or commit an immoral act
    - iandanforth 6y ago
      This attitude disturbs me, it tries to absolve responsibility by denying the possibility that responsible behavior is possible. That's not just obviously false, whole troves of law cover exactly this point, individuals and corporations have responsibility that follows their products.
      - umvi 6y ago
        Your attitude disturbs me. You want the ability to yank the rug out from under individuals and corporations if they ever do something that runs afoul of your personal morality system. How do I know your sense of morality isn't capricious and subject to the whims of twitter mobs? You could destroy my whole business if you arbitrarily deem my current or past actions immoral. For that reason, I will never, ever use software with a license like that. Therefore, I will never, ever create software with a license like that.
      - DSMan195276 6y ago
        I just want to say I agree 100%. All these arguments that people are making that you can just absolve yourself from responsibility for anything you create because "you don't know what they'll do with it" worries me. Sure, a JS Linter is probably not a thing to be worried about, and adding an entry to your software license doesn't really help, but people (and developers) are asked to do arguably bad things, and we should encourage people to use their moral judgement in deciding whether to do something rather than just decide "I'm not responsible, so it doesn't matter". I think it's also somewhat interesting to compare these comments against the ones on articles where we find out a company is helping China (or a different country) censor their website, either locally or globally. In those situations people are upset they were willing to do it - but why wouldn't they if they have no moral responsibility for what they're making and it brings in more dollars? Someone in those companies had to approve it and eventually add those lines of code or those entries in a database, and good or bad they should be willing to stand up for those decisions and say no if they truly disagree with what they're doing. They are, quite literally, our last line of defense against such actions - and it might not be "fair" to them, but it is reality.
        umvi 6y ago
        I don't think Linus Torvalds loses sleep at night just because North Korea has created a controlled linux distribution for use in their country[0]. Would you if you were the creator of Linux? If so, what would you have done that Linus Torvalds didn't do? [0] https://en.wikipedia.org/wiki/Red_Star_OS https://en.wikipedia.org/wiki/Red_Star_OS
- DSMan195276 6y ago
  I think adding the term to the license doesn't solve this problem (and were silly), but I think your argument is the wrong way to look at it. People should take a moral responsibility for the code they write, and refuse to write code if they view its uses as immoral, but you have to weigh for yourself what that means and what those criteria are, and adding a license requirement doesn't solve this problem. It's not hard to justify developing a compiler or CPU (or a JS lintier) and deciding that it's just a generic tool and you have no control over the code compiled with it - and that's perfectly fine. You might however find it harder to justify writing functionality that's designed to track people or that you know is intended to be used for nefarious purposes, and in those situations you should be willing to hold yourself to whatever moral standard you have determined. Just because everyone below you hasn't prevented you from doing it isn't a reason to go along with it, you know more directly what your software is going to be used for than they do and they're effectively trusting you to make these decisions. Again though, a JS linter doesn't exactly have these concerns ;)
  - wolco 6y ago
    Tracking someone isn't evil. Killing someone isn't evil. Pushing a kid into the bushes isn't evil either. Your tracking someone explore a new cave and recording paths found. You kill someone who is about to kill other innocent people. Pushing a kid into bushes to avoid a car could save their life. Evil depends on context. What's evil to you is not evil to me either. Courts generally throw out clauses like these unless the definition of evil is part of the agreement.
    - DSMan195276 6y ago
      I think it's worth pointing out I agree the clause is silly. I was talking on a more broader context of software development.
foxrider 6y ago
A license is not a pact with a demon, known as the original developer. A license is a legal document to be enforced by many different courts across the world.
fennecfoxen 6y ago
You know what? I’d like a world where people don’t decide that software licensing is an ideal place to wage ideological war and inflict harm on their enemies. I want collaboration and communication between people not in perfect agreement. I understand this is an overtly political stance; I am willing to commit to it as a positive and trust that it will benefit communities and the world at large, and that fragmentation into polarized ideological splinters will harm communities and the world at large (and enable our well resourced enemies to strike us in our weakness). Because this is a linter, that will help coders code well, and not a facial recognition suite that will facilitate totalitarian regimes as they crack down on dissidents.
- DSMan195276 6y ago
  I think it's worth pointing out that software licenses are what enable the kind of world you're describing. Unless you're suggesting massively changing copyright law (Which may be a fair thing to suggest), the typical well known software licenses enable lots of code sharing and contributions by making it very easy to understand the terms on which you can use it. Without them, you could never easily share or take code contributions without potentially having tons of copyright issues. Put another way, the issue here is not software licenses in general, it's the fact the author picked an obscure one with with dubious requirements. Picking no license would have actually been worse than the license he had.
TimTheTinker 6y ago
> Writing software is not a neutral act. You are morally responsible for its uses. That's an unsound argument - it suffers from a confusion of agency. The most I can be responsible for as an author is the intent of my software. If I'm a wrench manufacturer, I have no moral obligation to attempt to control how people will use my wrenches. They are amoral tools with a good intent (to help with construction projects). Whether my wrenches or those projects will end up being used for evil is beyond my purview.
- yunruse 6y ago
  This I would generally agree with. However, care should be taken with powerful tools – deepfake software, explosive material, and so on. The laissez-faire “guns don’t kill people” attitude isn’t wrong, per se, but it can be inconsiderate if used blindly.
  - umvi 6y ago
    deepfake software will eventually be so easy that script kiddies can and will flood the internet with deep fakes. That will hopefully "cheapen" the value of online video and audio clips and force people to learn where reliable sources of truth come from.
    - wolco 6y ago
      Verified instagram/twitter accounts with text?
    - zapzupnz 6y ago
      In a world where people still get fooled by Onion or Onion-like articles, I'm not holding my breath for it.
  - TimTheTinker 6y ago
    True, but I think your caveat is covered by my assertion regarding the intent of the tool. The intent of deepfake software is clearly either fraud or development of deepfake detection tools. If you're a vendor and your intent is the latter, it would be morally commendable to do your best to sell to reputable customers only... but with security tools like this the authorial intent almost doesn't (and shouldn't) factor in. Unfortunately, if you try to go soft so it won't "hurt someone" then the fraud detection R&D can't be as robust. Regarding guns, they're either for killing people, killing varmints/animals, or defending oneself against bad guys. As a salesperson, you can reserve the right to refuse sales to known crooks -- that would be morally commendable (as far as it's possible). But it's your decision based on your own judgment. As far as the gun itself is concerned, if as a manufacturer your intent is defense against bad guys (or legal hunting/varmint control) and your product reflects that intent, then you're morally in the clear. Incidentally, I'm strongly against sales of surplus military gear to police departments, because of the mixing of intentions that are at-odds. The intent of military gear is to fight foreign armies commanded by bad guys -- which is an intent that police departments should never have, as they're dealing with civilians. That's why in the US, we have the National Guard if an army of bad guys ever appears within our borders--it's different from the police, with different training and different intent.
falcor84 6y ago
Is a vegetable farmer responsible that people who eat the veggies only do good deeds? I suppose if you really wanted to, with GMO seeds sold as a service, you could conceivably enforce that.
xen0 6y ago
This is too simplistic and requires an objective sense of morality. I have to accept that some of my work may be used for things I do not agree with. I do not believe this is a stain upon my honour.
- iandanforth 6y ago
  Asking software authors to make moral considerations is less simple than saying, "You're free to ignore them." precisely because morality is subjective. Being irresponsible is much easier than being responsible after all.
  - xen0 6y ago
    There is some nuance. If I were writing software to control missiles then I had better be comfortable with how those missiles get used. But if I were writing a compiler for a mainstream programming language, then I don't think I could be expected to lose sleep over the fact that _someone_ might be using it to write missile control systems.
    - wolco 6y ago
      It goes further. Unless you stop the project the missile control will be built with less qualified developers. If a missiles keep hitting buses of average citizens you are at fault because you could have prevented that by finishing the project. Then you can even extend and say because the missile system wasn't built (because you left) when another country bombs and kills kids you are at fault. You could even say because the missile control wasn't built the next Hitler was born and all of those deaths are your fault. It's no wonder all presidents age so rapidly for those 4/8 years.
deleted 6y ago
[deleted]
frenchyatwork 6y ago
You're not entirely wrong, but looking at things this way will take you down the wrong path. The big question here is who decides what is right and wrong? Is it the author of JSHint? Is the the random dev tasked with adding a lintier to a project? Is it the lawyer who's vetting the organization's licenses? Ultimately, these questions of how we translate morality into a system of rules, that we use to punish people, shouldn't be decided by a few technical experts, but by the everyone, and we already have a system for that.
jahewson 6y ago
This makes no sense. If I write a todo list app I bare no responsibility for what people put on their list.
kelnos 6y ago
I think it's reasonable to expect people to refuse to write software that has only or mainly immoral uses. But suggesting that a software developer is responsible for any and all possible uses of their software, whether they could have reasonably thought of all of those uses, is absurd. Regardless, a tool is a tool. If something has, say 60% good use and 40% bad use, should it not be built? Where is the cutoff? 90%/10%? 99%/1%? Should hammer manufacturers be responsible for murder done using their hammers?
mhh__ 6y ago
There's a difference between writing a spreadsheet program that ends up being used in a concentration camp to writing the AI that decides who is sub-human. Neutrality doesn't mean much, although I agree as an industry software can be very naive ethically.
zzo38computer 6y ago
I disagree. If someone publishes something, or otherwise makes something available, then it is not available. It is not the author's/inventor's job to decide who is going to use it and how; they only decide how they will use it by themself. That includes money, too: they can charge money to provide a copy to someone else (or refuse to provide a copy at all), if they want to do, but it should not be their job to stop the customer from making additional copies with or without any kinds of payment and with or without modification. I think copyright and patents is bad. Of course something can be used immorally, but that does not necessarily make the author immoral. If you write a program or a book or whatever and you think bad things will be made from it, then you might consider to not write it at all (if that is your choice), but you might also have a different consideration, such as, that good things will be made from it, or whatever other consideration it might be.

Writing software is not a neutral act. You are morally responsible for its uses. Restricting those uses is both a responsibility and a good. Circumventing this restriction is itself an immoral act.

tkzed49 6y ago

So if I just write "you're not allowed to do bad things with this" in the license agreement, I'm covered?

nathanaldensr 6y ago

No, you're not. This is an argument that doesn't hold water. Why am I morally responsible but none of the abstractions below my code are not? What about chip manufacturers? The developers of CPU instruction sets? Compiler developers?

iandanforth 6y ago
They are. Why wouldn't they be? We put people in jail for distributing potentially harmful tools to people who are known to desire to harm others. This is not a new or controversial idea.
- umvi 6y ago
  Name one tool that has not been used to kill someone or commit an immoral act
  - iandanforth 6y ago
    This attitude disturbs me, it tries to absolve responsibility by denying the possibility that responsible behavior is possible. That's not just obviously false, whole troves of law cover exactly this point, individuals and corporations have responsibility that follows their products.
    - umvi 6y ago
      Your attitude disturbs me. You want the ability to yank the rug out from under individuals and corporations if they ever do something that runs afoul of your personal morality system. How do I know your sense of morality isn't capricious and subject to the whims of twitter mobs? You could destroy my whole business if you arbitrarily deem my current or past actions immoral. For that reason, I will never, ever use software with a license like that. Therefore, I will never, ever create software with a license like that.
    - DSMan195276 6y ago
      I just want to say I agree 100%. All these arguments that people are making that you can just absolve yourself from responsibility for anything you create because "you don't know what they'll do with it" worries me. Sure, a JS Linter is probably not a thing to be worried about, and adding an entry to your software license doesn't really help, but people (and developers) are asked to do arguably bad things, and we should encourage people to use their moral judgement in deciding whether to do something rather than just decide "I'm not responsible, so it doesn't matter". I think it's also somewhat interesting to compare these comments against the ones on articles where we find out a company is helping China (or a different country) censor their website, either locally or globally. In those situations people are upset they were willing to do it - but why wouldn't they if they have no moral responsibility for what they're making and it brings in more dollars? Someone in those companies had to approve it and eventually add those lines of code or those entries in a database, and good or bad they should be willing to stand up for those decisions and say no if they truly disagree with what they're doing. They are, quite literally, our last line of defense against such actions - and it might not be "fair" to them, but it is reality.
      - umvi 6y ago
        I don't think Linus Torvalds loses sleep at night just because North Korea has created a controlled linux distribution for use in their country[0]. Would you if you were the creator of Linux? If so, what would you have done that Linus Torvalds didn't do? [0] https://en.wikipedia.org/wiki/Red_Star_OS https://en.wikipedia.org/wiki/Red_Star_OS
DSMan195276 6y ago
I think adding the term to the license doesn't solve this problem (and were silly), but I think your argument is the wrong way to look at it. People should take a moral responsibility for the code they write, and refuse to write code if they view its uses as immoral, but you have to weigh for yourself what that means and what those criteria are, and adding a license requirement doesn't solve this problem. It's not hard to justify developing a compiler or CPU (or a JS lintier) and deciding that it's just a generic tool and you have no control over the code compiled with it - and that's perfectly fine. You might however find it harder to justify writing functionality that's designed to track people or that you know is intended to be used for nefarious purposes, and in those situations you should be willing to hold yourself to whatever moral standard you have determined. Just because everyone below you hasn't prevented you from doing it isn't a reason to go along with it, you know more directly what your software is going to be used for than they do and they're effectively trusting you to make these decisions. Again though, a JS linter doesn't exactly have these concerns ;)
- wolco 6y ago
  Tracking someone isn't evil. Killing someone isn't evil. Pushing a kid into the bushes isn't evil either. Your tracking someone explore a new cave and recording paths found. You kill someone who is about to kill other innocent people. Pushing a kid into bushes to avoid a car could save their life. Evil depends on context. What's evil to you is not evil to me either. Courts generally throw out clauses like these unless the definition of evil is part of the agreement.
  - DSMan195276 6y ago
    I think it's worth pointing out I agree the clause is silly. I was talking on a more broader context of software development.

foxrider 6y ago

A license is not a pact with a demon, known as the original developer. A license is a legal document to be enforced by many different courts across the world.

fennecfoxen 6y ago

You know what? I’d like a world where people don’t decide that software licensing is an ideal place to wage ideological war and inflict harm on their enemies. I want collaboration and communication between people not in perfect agreement. I understand this is an overtly political stance; I am willing to commit to it as a positive and trust that it will benefit communities and the world at large, and that fragmentation into polarized ideological splinters will harm communities and the world at large (and enable our well resourced enemies to strike us in our weakness). Because this is a linter, that will help coders code well, and not a facial recognition suite that will facilitate totalitarian regimes as they crack down on dissidents.

DSMan195276 6y ago
I think it's worth pointing out that software licenses are what enable the kind of world you're describing. Unless you're suggesting massively changing copyright law (Which may be a fair thing to suggest), the typical well known software licenses enable lots of code sharing and contributions by making it very easy to understand the terms on which you can use it. Without them, you could never easily share or take code contributions without potentially having tons of copyright issues. Put another way, the issue here is not software licenses in general, it's the fact the author picked an obscure one with with dubious requirements. Picking no license would have actually been worse than the license he had.

TimTheTinker 6y ago

> Writing software is not a neutral act. You are morally responsible for its uses. That's an unsound argument - it suffers from a confusion of agency. The most I can be responsible for as an author is the intent of my software. If I'm a wrench manufacturer, I have no moral obligation to attempt to control how people will use my wrenches. They are amoral tools with a good intent (to help with construction projects). Whether my wrenches or those projects will end up being used for evil is beyond my purview.

yunruse 6y ago
This I would generally agree with. However, care should be taken with powerful tools – deepfake software, explosive material, and so on. The laissez-faire “guns don’t kill people” attitude isn’t wrong, per se, but it can be inconsiderate if used blindly.
- umvi 6y ago
  deepfake software will eventually be so easy that script kiddies can and will flood the internet with deep fakes. That will hopefully "cheapen" the value of online video and audio clips and force people to learn where reliable sources of truth come from.
  - wolco 6y ago
    Verified instagram/twitter accounts with text?
  - zapzupnz 6y ago
    In a world where people still get fooled by Onion or Onion-like articles, I'm not holding my breath for it.
- TimTheTinker 6y ago
  True, but I think your caveat is covered by my assertion regarding the intent of the tool. The intent of deepfake software is clearly either fraud or development of deepfake detection tools. If you're a vendor and your intent is the latter, it would be morally commendable to do your best to sell to reputable customers only... but with security tools like this the authorial intent almost doesn't (and shouldn't) factor in. Unfortunately, if you try to go soft so it won't "hurt someone" then the fraud detection R&D can't be as robust. Regarding guns, they're either for killing people, killing varmints/animals, or defending oneself against bad guys. As a salesperson, you can reserve the right to refuse sales to known crooks -- that would be morally commendable (as far as it's possible). But it's your decision based on your own judgment. As far as the gun itself is concerned, if as a manufacturer your intent is defense against bad guys (or legal hunting/varmint control) and your product reflects that intent, then you're morally in the clear. Incidentally, I'm strongly against sales of surplus military gear to police departments, because of the mixing of intentions that are at-odds. The intent of military gear is to fight foreign armies commanded by bad guys -- which is an intent that police departments should never have, as they're dealing with civilians. That's why in the US, we have the National Guard if an army of bad guys ever appears within our borders--it's different from the police, with different training and different intent.

falcor84 6y ago

Is a vegetable farmer responsible that people who eat the veggies only do good deeds? I suppose if you really wanted to, with GMO seeds sold as a service, you could conceivably enforce that.

xen0 6y ago

This is too simplistic and requires an objective sense of morality. I have to accept that some of my work may be used for things I do not agree with. I do not believe this is a stain upon my honour.

iandanforth 6y ago
Asking software authors to make moral considerations is less simple than saying, "You're free to ignore them." precisely because morality is subjective. Being irresponsible is much easier than being responsible after all.
- xen0 6y ago
  There is some nuance. If I were writing software to control missiles then I had better be comfortable with how those missiles get used. But if I were writing a compiler for a mainstream programming language, then I don't think I could be expected to lose sleep over the fact that _someone_ might be using it to write missile control systems.
  - wolco 6y ago
    It goes further. Unless you stop the project the missile control will be built with less qualified developers. If a missiles keep hitting buses of average citizens you are at fault because you could have prevented that by finishing the project. Then you can even extend and say because the missile system wasn't built (because you left) when another country bombs and kills kids you are at fault. You could even say because the missile control wasn't built the next Hitler was born and all of those deaths are your fault. It's no wonder all presidents age so rapidly for those 4/8 years.

deleted 6y ago

[deleted]

frenchyatwork 6y ago

You're not entirely wrong, but looking at things this way will take you down the wrong path. The big question here is who decides what is right and wrong? Is it the author of JSHint? Is the the random dev tasked with adding a lintier to a project? Is it the lawyer who's vetting the organization's licenses? Ultimately, these questions of how we translate morality into a system of rules, that we use to punish people, shouldn't be decided by a few technical experts, but by the everyone, and we already have a system for that.

jahewson 6y ago

This makes no sense. If I write a todo list app I bare no responsibility for what people put on their list.

kelnos 6y ago

I think it's reasonable to expect people to refuse to write software that has only or mainly immoral uses. But suggesting that a software developer is responsible for any and all possible uses of their software, whether they could have reasonably thought of all of those uses, is absurd. Regardless, a tool is a tool. If something has, say 60% good use and 40% bad use, should it not be built? Where is the cutoff? 90%/10%? 99%/1%? Should hammer manufacturers be responsible for murder done using their hammers?

mhh__ 6y ago

There's a difference between writing a spreadsheet program that ends up being used in a concentration camp to writing the AI that decides who is sub-human. Neutrality doesn't mean much, although I agree as an industry software can be very naive ethically.

zzo38computer 6y ago

I disagree. If someone publishes something, or otherwise makes something available, then it is not available. It is not the author's/inventor's job to decide who is going to use it and how; they only decide how they will use it by themself. That includes money, too: they can charge money to provide a copy to someone else (or refuse to provide a copy at all), if they want to do, but it should not be their job to stop the customer from making additional copies with or without any kinds of payment and with or without modification. I think copyright and patents is bad. Of course something can be used immorally, but that does not necessarily make the author immoral. If you write a program or a book or whatever and you think bad things will be made from it, then you might consider to not write it at all (if that is your choice), but you might also have a different consideration, such as, that good things will be made from it, or whatever other consideration it might be.