6 ms·
> He also denied that Amazon had any responsibility for the negative impacts of their AI/ML technologies, or role to play in industry efforts to self-regulate.
by vorpalhex 8y ago
> He also denied that Amazon had any responsibility for the negative impacts of their AI/ML technologies, or role to play in industry efforts to self-regulate.
For a period of time, there was a lot of chatter about all developers having some kind of professional oath, like doctors. Many of the approaches that were taken have issues (preclude working on smart weapons program or legal surveillance).
I wonder if what we really need is a developers oath along the following:
"Anything I build can and will be abused. I am responsible for my designs, for my products, and for the data I collect and store. If my technology is used for evil, I am responsible."
- bbddg 8y agoIt's almost as if workers should own and control the means of production.
- j4kp07 8y agoThere is absolutely nothing stopping you and your friends from starting such a company. I won't be holding my breath as you start on your new venture.
- ben_w 8y agoWe do. In software, at least. The world has moved on a lot since 1848 — someone needs to promote genuinely new political ideas for our era, not rehash ones that came from the transition from agrarian-feudalism to industrial-capitalism.
- meesles 8y agoI don't know if the oath comparison works out so well for development. Let us not forget that each physician is directly responsible for a life every time they see someone. They also have a more direct effect on the success or failure of the procedure as they tend to produce a large portion of the work. I'd argue few companies ever reach this level of risk, and those that do are so large that the individual contributor cannot reasonably take on that burden of responsibility. In the example of some Amazon surveillance 'big-brother' software: Max the software dev is just making facial recognition software to the best of their ability. They aren't privy to the motivations, long-term plans, and potential consequences of those decisions. The oath is always a fun topic to discuss though: In reality it holds no meaning other than to the one who takes the oath. Correct me if I'm wrong, but in malpractice cases I doubt they cite the oath as evidence since all students are essentially forced to recite it.
- pixl97 8y agoNo snowflake is responsible for the avalanche, therefore I must pile on as much snow as possible -- The modern developer.
- whatshisface 8y agoNo individual litterer is responsible for a trashed park.
- vallavaraiyan 8y agoBut the snowflake doesn't know that it is falling on a mountain slope.
- ben_w 8y agoIndividual snowflakes are not capable of abstract reasoning. Can we make it a requirement for ourselves to limit our power to our ability to keep that power safe? I think that’s a superset of the problem of incentive alignment in AI safety, so probably not... but we also shouldn’t let the perfect to be the enemy of the improved.
- 8y ago
- ohitsdom 8y agoI like the idea of improving a sense of developer responsibility. But the challenge here is how to you define evil? How would you interpret this oath when building a web browser? Have a blacklist of "evil" sites? Obviously, a lot of issues to sort through there.
- devoply 8y ago"Anything I build can and will be abused. I am not responsible for my designs, for my products, and for the data I collect and store. If my technology is used for evil, the corporation employing me is responsible and the government is at fault for not doing something about it."
- deadmetheny 8y agoYes, you are. You chose to follow the design and build a system you knew would be abused. This mindset is what got us here to begin with. YOU ARE AT FAULT.
- arbitrary_name 8y agoAgreed. The USA is all about individual liberty, but not responsibility, it would appear.
- sib 8y agoSo, if I'm an assembly line worker at GM and someone buys a pickup truck and intentionally drives it into a crowd, murdering 20 people, I should get the death penalty as a serial killer? Obviously hyperbolic, but not very different at the core.
- ryandrake 8y agoA car’s primary function is not to kill people. If you are an engineer and develop a tool whose primary purpose is mass surveillance, you should bear that ethical burden.
- sib 8y agoAnd yet, cars kill many, many more people (1.25 million in 2013, per WHO) than mass surveillance does. And parent didn't qualify his statement as to which products the oath would apply to: "Anything I build can and will be abused. I am responsible for my designs, for my products, and for the data I collect and store. If my technology is used for evil, I am responsible."
- whatshisface 8y agoMass surveillance does something a lot more insidious than car wrecks, it sets up a future where people are killed. It's like global warming.
- Koshkin 8y agoWhatever the "primary function," car accidents is the leading cause of young people's death in the US.
- freehunter 8y agoI'm not arguing for or against that oath, but that would 100% kill Open Source dead the minute it becomes a requirement. Absolutely no one would share their code if they could be held responsible for its misuse.
- RcouF1uZ4gsC 8y agoYep. How much do you want to bet that the Chinese surveillance systems that target the Uighur run on some version of Linux? I bet that they have some parts that are written Python, Java, Javascript, C or C++ for which they use an open source compiler.
- skybrian 8y agoIf this effort succeeds, the end result will be something like "know your customer" for cloud providers. So, just like with app stores, you'll have to worry about getting your service approved. There seems to be an eternal cycle of abstraction creating and breaking, in both finance and software. - Someone builds a general-purpose abstraction that becomes popular. - Inevitably, the abstraction gets abused. - To prevent abuse, the abstraction gets violated. It's not general-purpose anymore because some usages are disallowed. (This seems to be why open API's to online services tend not to last very long.)
- ForHackernews 8y agoHas anyone actually quit over this? http://neveragain.tech/ http://neveragain.tech/
- RcouF1uZ4gsC 8y agoSo if you write an encrypted chat app, and people use it to plan a crime, are you responsible?
- jf- 8y agoIt depends on how far down the line you want moral culpability to apply. To take this to it’s logical extreme, it effectively makes the developer responsible for anything the end user decides to use their software for. The same reasoning is not only not applied to software, it is not applied to any product class with the exception of those that are immediately and obviously lethal. Engineers in google would be responsible for drug cartels operating in Columbia. Higher order causation must be separated from moral culpability. In other words, you should not hold people responsible for things that happen far downstream. Things that happen several links down the causal chain have occurred due to the decisions of many other people further along that chain, and a higher culpability should fall upon them. That is not to say that a software developer cannot be directly responsible for bad outcomes, maybe for example working on weapon systems for a nefarious state, where you’re fairly close causally to the point of application. My point is that it’s not a good idea to push this to it’s limits.
- jka 8y agoSomething like this would be a great start. I have a feeling the industry has a tendency to try to recruit young and perhaps naive developers who might not be fully ready to appreciate all of the potential misuses (and the probability of those outcomes) of what they build, however. Or worse, have some misaligned incentives to build them regardless.
- CamperBob2 8y ago"Anything I build can and will be abused... If my technology is used for evil, I am responsible." Well, which is it? Both of these statements can't be true. Does this creed apply to gun manufacturers? Knife manufacturers? Car manufacturers? Hammer manufacturers?
- cronix 8y ago> If my technology is used for evil, I am responsible. So, by that logic, Tatu Ylönen should be held responsible for all hacks/crimes committed via ssh? You can't blame someone for making a tool that someone else uses for evil. Should Ford be responsible for all auto-related deaths? Should Edison be responsible for all deaths where the state put someone to death via electrocution? Should Jobs be responsible because a hacker used an OSX machine to hack into another machine? I don't get this line of thinking, although I understand the sentiment.
- canada_dry 8y agoWorth noting the Project Managers who are certified "PMP"s attest to a code of conduct that includes: "take actions based on the best interests of society, public safety, and the environment." It requires that PMP's report any unethical practices to "appropriate management". Pretty weak, but hey, it's better than nothing.
- ferongr 8y agoWhat is "evil"?
- deleted 8y ago[deleted]
- int_19h 8y agoThat is insanely broad and vague. You can be responsible for knowingly designing a system for abuse, or if you give it to someone you know, or have reasonable cause to suspect, will abuse it. But what you're saying here is that if you ship anything at all, and someone somewhere finds a way to use it for some nefarious purpose, then you're responsible. That's not how things work with anything else; why should they work that way with software? That aside, in US specifically, there's already something that is more narrowly tailored to our current reality. They aren't accepting new signatures because of how many there were, but you can still make the same pledge (and e.g. share it publicly to have some skin in the game): http://neveragain.tech/ http://neveragain.tech/
- crawfordcomeaux 8y agoI think it's time to acknowledge the impact of culture in our oaths. It IS how we're all naively programming ourselves and others, so we might as well address it. If my tech is used to do harm, the responsibility to heal is shared between myself, the cultures involved in leading to harm, the perpetrators of the harm, those harmed, those witnessing harm, and those in denial of responsibility. I'll do my part while encouraging and believing in others' willingness to grow together.
- Nasrudith 8y agoI think oaths are completely outmoded as a social construct - everyone knows that people lie regardless of what they claim. The actual teeth to it were their own sources of problems, guilds and not being able to run your business if violated. They were supplanted by licenses. Even if we somehow had that measure in place it would be a cure worse than the disease - well you can't get a job doing something legitimate because it turns out that your safety human recognition algorithm got abused by somebody else? Just crime for you to make a living.