7 ms·
Can you stop your open-source project from being used for evil?
- yjftsjthsd-h 4y agoThe road to hell is paved with good intentions, but throwing out freedom zero is a lovely way to get there in short order.
- rootusrootus 4y agoI sure hope not. The definition of evil is too subject to interpretation.
- ccn0p 4y agoExactly my first thought. With a libertarian hat on, of course anyone can do anything with whatever they create. That's someone's right. But to call it "open source" with some license that attempts to define morality? Good luck with that.
- giraffe_lady 4y agoone of the examples given in the article is torture.
- happyopossum 4y agoThat’s an example, not a limitation on the definition of the word ‘evil’.
- didgetmaster 4y ago
- mjfl 4y agoanother could be genital mutilation, or giving puberty blocking hormones to children.
- prox 4y agoAre you serious? There might be some grey areas but ethics and morality is a lot more developed than that.
- snapetom 4y agoYes, OP is serious. I have to ask if you're serious. There was an extensive interview with an Australian ISIS recruiter in The Atlantic (I think) several years ago. The guy wholeheartedly believed he was doing good, and felt sadness and compassion for Muslims outside the caliphate because they weren't getting into heaven. What if he was also a developer and forbade you from using his software because you weren't using it for his definition of good?
- prox 4y agoI think we are looking at this from different angles. Certain persons might not have a clue on right/wrong and/or simply lack a moral compass. That doesn’t mean there is a whole field of study on ethics and morality, from which I am viewing this.
- dleslie 4y agoIndeed, private property is a good example. To some, protecting private property ownership is an evil. To others, undermining private property is an evil.
- elcritch 4y agoAnother aspect would be copyright protections. Depending on the circumstance people have widely varying ideas of whether copyright protection is good or bad. If my Linux distro includes some “do no evil” license what if they decide my anti-copyright program is evil? Or vice-versa? Making a living as an indie game developer, perhaps one would be more inclined to view copyright protection as “good”. A decade later someone trying to archive the same indie devs game which made a big impact on their own childhood and copyright becomes “bad”.
- dleslie 4y agoAs a game developer, going on over two decades now, I have _complicated_ feelings about copyright. ;)
- 2OEH8eoCRo0 4y agoRight. I'm the good guy and will gladly help build autonomous killer robots for my government.
- newaccount2021 4y ago
- AnimalMuppet 4y agoNo, of course you can't. If nothing else, evil doesn't feel obligated to obey copyright laws. If it's open source, they can get the source, and do whatever they want. Laws? Why would they care? I'm on the "do not call" registry. That doesn't seem to stop spammers.
- bee_rider 4y agoThey cover this in the article -- while some underground evil organizations might not care, there are also companies like Palantir. They work with the government, they probably want to avoid any legal headaches where possible.
- AnimalMuppet 4y agoI'm aware of Palantir, though I have no actual detailed information about their internal workings. But my impression is that they would probably be willing to violate a copyright if it enabled them to do what they were asked to do. They might even do it with the (perhaps unstated) blessing of the part of the government they were working with. As I said, I don't have actual information. I may be misjudging them.
- commandlinefan 4y agoWell, if you delve just a tiny bit into the source material, you see that they use evil as a shorthand for "perfectly legal things that a plurality (if not a majority) of people have absolutely no problem with but that they do", like immigration enforcement - in which case, some legal maneuvering might actually help them, although I personally consider their goals to be more evil than the uses they're trying to prevent.
- coffeecat 4y agoThis article makes the mistake of conflating open source and free software. Open source software may be distributed under licenses that prohibit use by orgs/industries that the developer finds unsavory. But the resulting software is proprietary - not FOSS.
- seba_dos1 4y agoIt does not. "Open Source" and "Free Software" are pretty much synonymous, with the difference between them being mostly political rather than practical (which doesn't mean that the difference isn't important, it's just not what you think it is). If the software isn't Free, it can't be Open Source. Consult the definition for more details: https://opensource.org/osd https://opensource.org/osd
- sys_64738 4y ago> It does not. "Open Source" and "Free Software" are pretty much synonymous It depends who you ask and who the decision maker is.
- modzu 4y agothis is a pet peve of mine because we should be able to distinguish between closed source software and open source but non free software. its extremely useful to be able to see/verify what some code does on your computer even without any other rights to that code.
- madeofpalk 4y ago"source available".
- seba_dos1 4y agoSuch software is usually called "source available".
- jahewson 4y agoNo, you are mistaken. That is not the commonly accepted definition of open source, see the OSI: https://opensource.org/osd https://opensource.org/osd > 6. No Discrimination Against Fields of Endeavor > The license must not restrict anyone from making use of the program in a specific field of endeavor. For example, it may not restrict the program from being used in a business, or from being used for genetic research.
- kevingadd 4y agoWhen considering license measures for this purpose, remember that evil people don't care about software licenses or copyright. So if your license inconveniences good people to "hurt" evil ones, you're probably screwing up. The person making NFTs of Uyghur prisoners, for example, definitely doesn't care about whether that action was legal and didn't go secure likeness rights or check the license.
- h2odragon 4y agoIf you're deeply concerned about who uses your software and for what purpose they do so; "open source" isn't what you want. You want a real commercial license, even if you're not charging money. There's nothing wrong with that, and no reason to try making "open source" attempt to conform to moral stances. I'd say that the core value that open source serves is the notion that sharing our thoughts is better than not doing so. That is enough of a stance to take. People can agree to that even in the face of differing values in other areas.
- LinuxBender 4y agoAdding to this, even a commercial license may not be sufficient in some cases. If the people doing evil things are truly evil people or organizations, it may not be feasible or even possible to take legal action against them. Some groups are effectively untouchable by the legal systems and some people are within the boundaries of governments or organizations that exemplify the concept of total corruption and hegemonic domination.
- phire 4y agoYes. And your commercial license can still offer the source code. Unreal engine is an example of such a licence. Anyone can access the source code for free (just need to jump though a hoop of signing up of an account and agreeing to the terms and conditions). Anyone can release a game without paying unreal a cent. But it is not open source. The two major restrictions are: You can't share the source code to people who haven't agreed to the terms and conditions; And if you earn more that $1 million in revenue, you must pay epic games 5% of all additional revenue. Epic have even gone to the additional step of removing their right to revoke the license from the license agreement. They can take you to court if you don't pay, but they can't simply revoke the license.
- Andrew_nenakhov 4y agoYou can use whatever no-evil license for your open-source project, but if you want your project to be FOSS [1], you can't. If you restrict your project from being used for evil, it'll violate the basic freedoms that proper Free licenses grant to their users. Also, check this video by Douglas Crockford [2]. ("I give permission for IBM, its customers, partners, and minions, to use JSLint for evil.") [1]: https://en.wikipedia.org/wiki/Free_and_open-source_software https://en.wikipedia.org/wiki/Free_and_open-source_software [2]: https://youtu.be/-C-JoyNuQJs?t=39m45s https://youtu.be/-C-JoyNuQJs?t=39m45s
- seba_dos1 4y agoA no-evil clause makes your license not Open Source either. There's no practical difference between "Open Source" and "FOSS". You probably meant something like "source available".
- verdverm 4y agoI think what you are actually talking about is permissioned vs permissionless licenses. We typically think of open source as the later, but licenses like GPLv3, the licenses like Elastic & Mongo (commercial restrictions), and ethics are all licenses that only grant you permission in certain situations.
- jpablo 4y agoOSI couldn't trademark "Open Source" and the common usage of the term (source available) has steered away from the OSI definition.
- seba_dos1 4y ago[citation needed] Common usage has matched the OSI definition for decades. Even Wikipedia acknowledges it [0]. There are always some people who are confused about the terms, sure, but those usually come simply from lack of exposure and experience around FLOSS communities. Trademarks have nothing to do with it; it's not OSI that owns the term, but the community that uses it and widely agrees with OSI's definition. [0] https://en.wikipedia.org/wiki/Open-source_software#Definitions https://en.wikipedia.org/wiki/Open-source_software#Definitio...
- bjt2n3904 4y ago"I asked Coraline Ada Ehmke, a leader in..." Oh good grief. This tired topic again. How'd all this nonsense do for the Node TSC and Rod Vagg? Ehmke isn't anyone I'd trust making ethical decisions for anything. In Ehmke's own words, "the Contributor Covenant is a political document". It has zero to do with fixing real world problems, and everything to do with using software development as a tool to advance political ideology. And Ehmke has the audacity to lecture about ethics. Here's something unethical: weaponizing a neutral thing (software, medicine, sports) in pursuit of political power.
- woodruffw 4y ago> Ehmke isn't anyone I'd trust making ethical decisions for anything. In Ehmke's own words, "the Contributor Covenant is a political document". Independent of your trust, can you explain the connection between these two points? The Bill of Rights is a nakedly political document, and yet a large swath (probably the overwhelming majority) of the US attributes some moral value to its contents.
- bjt2n3904 4y agoThis is an excellent question, and I struggle to answer it. Part of why I don't trust the "Code of Conduct" things is because of how they started off -- the desire to "protect marginalized people's safety over privileged people's comfort" is inherently prejudiced. Who decides who is privileged and who is marginalized? What this boils down to is there are favored people to which the rules do not apply. Although the language is no longer there, in many instances the result is the same. See the aforementioned Node TSC drama. But none of that is listed in the Contributor Covenant, which Ehmke authored. Version 2.1 pledges to make participation a "harassment free experience for everyone". As a software maintainer, how can I make good on that pledge? How can I arbitrate disputes between members? Especially when this involves conduct outside of git, and with sensitive users who consider many innocuous things as harassment? But furthermore, WHY should I arbitrate members conduct? I have enough on my plate just writing software. Now I have to decide who is right in a "he/him said, they/xir said", and decide who gets voted off the island? You'll note that I have referred to Ehmke without pronouns, and this is intentional. While I do not wish to be offensive, I can't use Ehmke's pronouns in good conscience. According to one version of the Stack Overflow guidelines on pronoun usage, this constitutes harassment. I firmly disagree -- harassment is "you call me a woman, or I will have you removed from the project". As Ehmke said, this is a political document. But politics does not belong in software development.
- ghaff 4y agoNo. But you can release software code under a license that places usage-based restrictions on the code. Some may ignore it, yes. It won't be open source under the usual definitions. But it may prevent organizations you disapprove of (along with many others) from using your code. Whether is still makes sense as mostly a source-available project at that point is up to you.
- happyopossum 4y agoShort answer: no. Long answer: still no. you have zero control over what anyone chooses to do with your open-source project. You gave up any control you had when you released the source code to the world.
- Silverback_VII 4y agolet's ask the developers of the open-source deepfake program what they think is an ethical use of their tool.
- sneak 4y agoI have yet to see a single piece of evidence that deepfakes are actually harmful and not simply distasteful. People cited the one of Zelensky telling soldiers to surrender - but did any actually simply as a result of such a video? It's a stretch. Show me specific, real harm from a deepfake, or let's stop the hand-wringing.
- yjftsjthsd-h 4y ago> let's ask the developers of the open-source deepfake program what they think is an ethical use of their tool. We... did? Like, that was 90% of the comments when it was posted.
- dheera 4y agoDon't use a FOSS license. You can still open your source, take pull requests and everything, and use a proprietary license with a clause that says the code is not to be used for evil, where you (the author) hold the sole final judgement of what constitutes evil. Basically, copyright the shit out of your code, but selectively enforce it, and hint to the community that you are unlikely to enforce it on people who you think aren't evil. Most people will likely steer clear of your code, except for a handful of hobbyists and maybe some academics.
- powersnail 4y agoTangentially, what is the limit of what can be put into a license or contract (that remains enforceable)? Can you just put any arbitrary requirements, as long as it's not against the law? Like, if I lease my house, but with a clause that the resident should never change their name to "Jack" in their entire life, will that be laughed out of the court?
- bilsbie 4y agoTo further riff on this could your license prevent evil outcomes regardless of whether the original intent?
- verdverm 4y agoRiffing even more, what if the usage is through a chain of dependencies and I sit in the middle with my project?
- nomdep 4y agoFor someone with difficulties to see the grays of the world (like Coraline Ehmke, the source of the article) it must be very comforting to think there might be a scenario where the answer is “yes”. However, by definition, open source cannot
- guelo 4y agoYes. OSI's ridiculous and unsuccessful attempts to legally own the phrase "open source" not withstanding.
- cdiddy2 4y agoInteresting article timing since github just removed the Tornado Cash project as well as all of the developers accounts from the site. I guess github doesn't think open-source code that can be used for evil should be allowed to stay up.
- yjftsjthsd-h 4y agoThere's a difference between licensing a project to try and limit what users can do, and being legally obligated to stop hosting a copy of it.
- pdntspa 4y agoOnce your intellectual work is out there in the open, it is no longer yours. It is effectively public domain, regardless of the license -- because the license assumes that the user actually cares about it, and that it is actually enforceable. But there are a variety of scenarios where this is not true. This applies to any IP. Remember: sharing is caring
- thesuperbigfrog 4y agoSQLite is public domain, but offers a blessing instead of a license: "The author disclaims copyright to this source code. In place of a legal notice, here is a blessing: May you do good and not evil. May you find forgiveness for yourself and forgive others. May you share freely, never taking more than you give." Source: https://sqlite.org/src/artifact/df5091916dbb40e6 https://sqlite.org/src/artifact/df5091916dbb40e6
- loa_in_ 4y agoYou could develop SkyNet and have it invade every computer on the world to monitor usage. It still leaves many problems like codifying the definition of evil, airgapped systems, detectability and how evil would you consider invading everyone to be.
- deleted 4y ago[deleted]
- abetusk 4y agoI don't know if it's too conspiratorial but this does seem like a sideways attack to "embrace, extend, extinguish" FOSS. By couching it in terms of FOSS being "unethical", FOSS licenses can be whiddled away until there's either enough non-libre licenses to cause confusion or have a widely adopted alternative license that's unusable for business. I do notice that Stack Overflow is using a Microsoft technology stack. The author of the article has been part of Microsoft Research [1]. [0] https://en.wikipedia.org/wiki/Stack_Overflow#Technology https://en.wikipedia.org/wiki/Stack_Overflow#Technology [1] https://stackoverflow.blog/author/david-gray-widder/ https://stackoverflow.blog/author/david-gray-widder/
- phendrenad2 4y agoNo, because not many (perhaps none at all) people have objective measures of morality. Is waterboarding torture? You say yes, I say yes, but if it's 2003 and the Iraq War is in full swing, do you think enough people are going to agree with us to mount an effective lawsuit? Maybe you can instead choose to explicitly enumerate all things you consider "evil" in the license, but that license is going to be prohibitively long.
- deleted 4y ago[deleted]
- jahewson 4y agoAs a society, we have laws for this kind of thing. If you want to ban something, go to the legislature and ban it. Abusing our position as creators to illiberally force our own personal "ethics" on others is nothing but petty tyranny.
- davidork 4y agoWhoever originally invented the wheel ultimately is responsible for all weapons of war. once the cat is out of the bag, it's not going back. This smells like someone inventing a moral dilemma because the FUD didn't work on the users to make them stop using open source, so they're gonna try to get the developers to stop making it.
- hosh 4y agoI don't know if software can be written in a way that cannot be used for harm. There are several issues with this, among them being (1) there is always a "shadow side" (a reference to the Jungian shadow), and (2) this is a form of "Do Less Harm" paradigm, and it is problematic because morality is not universal, monolithic, or consistent. Imposing what "less harm" means on someone else, will always polarize somewhere. It's the same problem with the "Do More Good" paradigm. I have been lately thinking of a different way of developing and funding improvements of software, by adding on top of the idea of open source of free software. That is, "community-supported software", similar to the idea of "community-supported agriculture". No one is going to get rich off of it, but it should allow developers to get their fair share (a reference to the permaculture ethical principles). This doesn't stop software from being used in harmful ways. However, what I'm thinking is that it helps software developed in ways that are beneficial for local communities instead of being hijacked by large corporate interests in markets that are not directly connected with the community (national stock exchanges). The end-users and local community decides what is harmful or beneficial, and pool resources to maintain and expand features.
- ryandvm 4y agoI'm generally not a fan of paternalistic license clauses, but I dunno, I think I could get behind an open source license clause that prohibited click-through user agreements. In my opinion, they are a complete nuisance with no upside. Nobody is reading them and I doubt they are even enforceable. From what I can tell, they are nothing but economic drag introduced by the corporate lawyers.