10 ms·
Google employees reportedly quit over military drone AI project
- archagon 8y agoI wish there was a legally-enforcable version of Douglas Crockford's "Good, not Evil" license. I haven't released any source code that could have military applications yet, but if I ever do, I want to make it 100% clear that it's not to be used for any task related to the killing or injuring of other people. We're in a unique position as programmers where even the tiniest bit of our code can affect thousands or millions of people across the globe, and this terrifies me. The GPL has already shown us that a license has the power to change culture and behavior (in however small a way). We should be able to extend this approach to other values we hold dear.
- raverbashing 8y agoPacifism is nice on paper, too bad not everybody adheres to it
- archagon 8y agoThat's fine; they can use someone else's code. I just want the power to license my software in line with my own values. I'd find it very hard to live with myself if some code I wrote ended up being used to fire missiles at people, even if they "deserved it".
- option2 8y agoKudos to those who is ready to stand for their principles. If you are at G and thinking whether you should resign or not, remembers this - the market for AI talent is super hot. You will immediately find lots of great and challenging AI work pushing humanity forward
- thirduncle 8y agoYou will immediately find lots of great and challenging AI work pushing humanity forward. They'll find jobs quickly, that's for sure. But "work pushing humanity forward"? There's precious little of that in any skill sector - not at FAANG salary levels, anyways. The vast bulk of the work that the vast majority of us do is simply about pushing the investor's balance sheets forward - not "humanity".
- rm_-rf_slash 8y agoCorollary: plenty of skilled engineers with fewer moralistic constraints will jump at the chance to do interesting work for high pay. For a company as large and wealthy as Google, they can continue to raise offer salaries until they are adequately staffed. There is a school of thought that recommends “moral” people doing “immoral” work because if those people left then other “immoral” people will take those jobs and more readily implement “immoral” features. So the “moral” engineers have an incentive to stay and act as a front line against “immoral” actions, or at least have an insider’s position for whistleblowing. Military drones are here to stay, and whether or not the US builds them, other military powers certainly will. Ultimately, I don’t think this changes anything.
- stochastic_monk 8y agoIt at least makes a statement. It may not prevent the work from being done, but at least these people have washed their hands of it.
- option2 8y agoIt changes the answer you give to your kid's question: "Daddy/Mommy what do you do at work?"
- Consultant32452 8y agoDo you think there is a moral difference between the person who carries a gun in the military and the person who designs it? It seems like whatever feelings, positive or negative, we have about military personnel should carry at least in part to weapons designers.
- ChuckMcM 8y agoThese two comments, the parent and grandparent, capture the essential challenge of integrity in engineering. You can have integrity and choose not to work on things you feel are morally wrong, or your can choose to set aside your reservations and continue the work. It is not an easy place to be and it causes many people stress. I don't know if my experience is typical or not but I do know that several times in my career I have encountered or been put into a position where it was clear that 'success' was tied to doing something which I felt was also wrong. I have always chosen not to set aside my principles for that success. And still I know people who have made the immoral choice and reaped the rewards, and then they have used that success to step into places of higher influence or control. They would no doubt argue that they were in a much better place to do good now, because they chose to do something wrong once before. It is not surprising that this conflict is the underpinning of many dramatic stories.
- oh_sigh 8y agoHard to tell if those people were planning on leaving anyway, and just wanted to make a splash. How many people quit each week in a company with 90,000 employees? Only a small percent of engineers at google are AI experts. Sure, more people use it, but they probably just make a service call and get some magic results back.
- joebadmo 8y agoseems like a dupe of https://news.ycombinator.com/item?id=17074148 https://news.ycombinator.com/item?id=17074148 which has more comments
- DINKDINK 8y agoYou'd think that killing people would be covered by "Do no evil"
- ythn 8y agoIt's not directly killing people. That's like saying helping improve GPS satellites kills people because weapons systems use GPS and would benefit from increased accuracy.
- astrodust 8y agoJust like the "do no harm" oath taken by doctors means "If you gotta kill like twenty people to save a hundred, get on with it."
- peterwwillis 8y agoIn 2015 Alphabet changed their motto to "do the right thing". If you interpret "right" as "correct", then "doing the right thing" can mean 'correctly' bombing civilians.
- oh_sigh 8y agoIf project maven worked as planned, wouldn't the USG kill fewer people?
- carlosrg 8y ago> runs contrary to Google's ethos -- the mantra "don't be evil" has long been at the heart of Google's principles Thinking that the military of your own country is "evil" seems a bit puerile to me. Watching too many movies and TV shows can have that effect.
- sincerely 8y agoWhy does the "evil"ness of a military have to do with whether it's your country or someone else's?
- option_greek 8y agoBecause F16s are developed in USA and used only by US military ? /s
- briffle 8y agoF16's are used in many countries. They were developed in the US, but many, many countries use them (some of our NATO partners since the beginning) Since they were first developed, NATO partners: Belgium, Denmark, Netherlands, Norway Other European countries as soon as the mid-80's: Croatia, Greece, Italy, Poland, Portugal, Romania Middle east since the mid/late 80's: Bahrain, Egypt, Israel, Iraq, Pakistan, Turkey, Jordan, UAE Africa: Morocco, Indonesia, Asia: Singapore, South Korea, Taiwan, Thailand South America: Chile, Venezuela.
- option_greek 8y agoMy point exactly :) (check the /s = satire tag)
- astrodust 8y ago/s = Sarcasm
- ImSkeptical 8y agoI believe that's the point of the comment you are replying to. You may develop weapons believing in the goodness of your country's military, but those weapons may come into the hands of people with more dubious moral positions.
- ythn 8y ago> Google recently sponsored the Conservative Political Action Conference, for instance. Of course, there would be no conscientious objectors if it were a liberal event
- rainbowmverse 8y agoPeople who disagree with you often disagree with each other as well. This doesn't come through in a media (of all political leanings) that tends toward a narrow set of well-funded viewpoints.
- flyinglizard 8y agoI wouldn’t say that developing military hardware necessarily negates the “don’t be evil” principal (especially if the developed articles are dual use, for both military and civilian applications). The western world, our principals and values have prospered for more than half a century in Pax Americana afforded, in a large part, by the prosperous US Military Industrial Complex. I totally get the objection to developing combative AI - that’s a separate ethical question - but you can contribute to the military and still maintain your humane values.
- titzer 8y agoGiven that the US military has killed nearly 4,000 people with drone strikes in Pakistan over the past decade, a country in which no formal military conflict exists, nor any formal enemy, just vague accusations of terrorist networks (and likely a bunch of political dissidents fed to them by the Pakistani government), I am really wondering where the "Pax" is coming from. Because if you do the math the odds of any given person knowing someone who was killed by a drone strike, or someone who knows someone who was, are pretty damn high. The US has brought hell to Pakistan. I say fuck that.
- dragonwriter 8y ago> Given that the US military has killed nearly 4,000 people with drone strikes in Pakistan over the past decade, a country in which no formal military conflict exists Pakistan is a country with which no formal military conflict with the US exists, but it is not one in which no formal military conflict involving the US exists. The 2001 AUMF is a (ludicrously open ended) conditional exercise of Congress' Constitutional power to declare war, and the parties targeted in Pakistan are parts of groups to which the executive branch has determined that the conditions in that act apply.
- walshemj 8y agoAs the number of police shootings (with a high number of collateral ones) is about 1k per year over the last decade American police have killed 10,000 and America doesn't have large tribal area's and the related problems like Pakistan.
- detaro 8y agodupe of https://news.ycombinator.com/item?id=17064776 https://news.ycombinator.com/item?id=17064776 (98 comments) (discussion of the Gizmodo article thats the source of this article)
- mmjaa 8y agoI once worked for a large industrial group in Europe. The kind that has a bit of a piece of every little pie there is - military, transportation, etc. I was pretty happy working there .. until I got a demo from the 'defence' group. They demonstrated the willingness to push the company's technology into heinous, heinous territory. The kind of thing where a drone would be able to follow a single person in a crowd, and target them for execution - unguided, of course. I quit the next day. Those of us who make technology, need to be very sure we see that it is not used destructively against the human species. The responsibility is very, very high. And, the danger is extreme. These people were revelling in the fact that they could develop targeted assassination drones and sell them to any country in the world. Heinous.
- dsfyu404ed 8y agoI can beat you to death with a hammer or stab you to death with a knife. Should we not build hammers and knives? Should hammers and knives be subject to KYC laws? Sure you can argue that it's different because those are things the common man can get whereas on the state can afford surveillance dragnets and drones but it wasn't that long ago that only the state could afford computers. Edit: Apparently I struck a nerve. Technology transfers between military and civilian application all the time. Propeller technology that helped submarines that are now obsolete stay quiet is fine tuned in a different manner to yield more environmentally friendly watercraft. A drone that can disperse insecticides on only the crops that need it can deliver chemical weapons with some slightly different fine tuning. An 1984 (or 2018 UK if you like) surveillance and law enforcement system could be used to track down corruption in government, suppressing dissidents, identifying insider trading, identifying human tracking, etc. It all depends on who's using it. (I personally don't trust any government to properly wield that kind of power.) The technology doesn't care. It's all how you use it.
- kevindqc 8y agoAre hammers and knives designed and bought to kill people?
- 8y ago
- whb07 8y agoI’d like to offer the point of view that if the drones become better and more surgical in their precision, it would reduce civilian casualties. Like it or not the world is full of extremists who would like nothing more than to hurt innocent people. There is no “oh just send the cops and arrest them!” route to take. Shit, just look at the time Osama bin Laden could have been bombed with a tomahawk missile during Clinton’s presidency. He didn’t do it because of the potential to kill a Saudi prince he was meeting at that time. Would those angry Googlers be against surgically killing Osama? I think not. Better drone software might help track a potential target and present with the optimal window in which a target could be shot and have reduced civilian casualties. It could also present with better intel to let a surgical ground strike which would put more American soldiers at risk but would allow for better intel and again less civilian deaths. Lastly, it could offer new knowledge and experience in tracking humans with drones during humanitarian disasters. It could also help in tracking victims of kidnapping, are the Googlers opposed to rescuing the hundreds and thousands kidnapped by Boko Haram and company? Who is going to go into the African heart of darkness to rescue those people? Is it the arm chair Googlers who pretend to know better?
- VikingCoder 8y agoCan you guarantee the technology won't fall into enemy hands? Can you guarantee our government won't initiate illegal aggression? Won't subvert democracies? Won't have another Gulf of Tonkin? Won't target the families of terrorists, as our current President has suggested? No, you can't guarantee those things. Guns don't kill people, people do. And people are sometimes evil, and sometimes break the law, and sometimes make mistakes. And sometimes the gun is stolen. So maybe some engineers don't want their company to make any guns. You don't get to ask, "Would those angry Googlers be against the technology always being used the way they intended?" Instead you have to ask, "Is it possible for this technology to be used in ways those Googlers would object to?" And of course the answer is yes. Many on the Manhattan project thought that bombing Nagasaki was completely unnecessary. Some probably thought Hiroshima was unnecessary, that a demonstration of the power would be sufficient.
- nyolfen 8y ago
- kelukelugames 8y agoRegardless of your opinion on military projects, I hope we can agree that voting with your feet is a good thing.
- mr_rheee 8y agoI've been asked to work for military industry companies before. And I have always declined for ethical reasons. But as I sit here and think about it, I wonder if its a good thing that a person like myself (that believes I'm on the ethical high-ground) decline these types of jobs. Someone is going to take the job. Perhaps someone less skilled than myself, perhaps someone less ethical than myself ? What is the result of that ? As another poster wrote, it's "good" that the targetting gets more precise, meaning less collateral damage. But to each his own. We need to be able to sleep at night aswell. And that to me also seems like a really good reason to decline. I'm kind of on the fence about wanting to work in that industry.
- djhworld 8y agoIt depends on how much of an impact you'll think you can make, I'm always of the mind that these defence contractors primary motivations are what their customers desire (i.e. military), rather than making considerations for anyone on staff.
- mr_rheee 8y agoI guess when you work in an industry where what you create or do is done for killing, as a person, you must have crossed some line of what is okay to do for yourself. Similar to people who remote control 'drones'. And at that point, I would think your will to try and change what the company is doing is very small. On that note, I jumped back off the fence. Not for me.
- obelix_ 8y agoIt's not complicated. Stick to your values. When you work on stuff that you feel ambiguous about, you aren't going to be doing you best work.
- mr_rheee 8y agoYeah, you're right. I quickly realized that after posting my comment.
- 8y ago
- flurdy 8y agoI am worried we are not too far away from this becoming a reality: https://www.youtube.com/watch?v=TlO2gcs1YvM https://www.youtube.com/watch?v=TlO2gcs1YvM
- hacker_9 8y agoDefinitely scary, but what happens if enemy states create this technology before us? That's why we need a defence sector that does, in fact, build these tools. If both sides have the technology then a stalemate is reached. Look at nuclear weapons; we can all bomb each other out of existence, and so, no one does. Now look at the countries that lagged behind.
- agitator 8y agoWow that is frightening!
- Lionsion 8y agoThat video seems to have been copied everywhere, and the copy you linked to seems to be on a fringe UFO conspiracy channel. As far as I can tell, this is the original: https://www.youtube.com/watch?v=9CO6M2HsoIA https://www.youtube.com/watch?v=9CO6M2HsoIA
- totalforge 8y agoThis video is fictional, and was made as a warning about where current tech could take us. I have no doubt that generals who have seen it are quite upset that they can't have it yet.
- Lionsion 8y agoI realize that, I just wanted to drive the traffic to the original source rather than to some weirdo who uploaded a copy.
- flurdy 8y agoThanks. I thought I had found the original upload after sifting through a lot of copies on youtube. Wish youtube was a little proactive with detecting dupes, rather than reactive. Or at least some sort of an automatic note/link to original even if it is a legal derivative.
- dfsegoat 8y agoSomebody else will fill the gap. It is simply a consequence of military science: If a human in the loop makes combat systems less effective, then other countries will seek the advantage over others by removing the human from the loop. It's a classic arms race at this point. ... This is a pandoras box that has already been opened I am afraid.
- 21 8y agoI have a question for the people who applaud them. Do you think USA should have no military? Would you vote to dismantle it?
- throwaway6497 8y agosome incoherent ramblings... I would be curious to see the list of employees who quit for this purpose. They are making a statement. They might as well publicly disclose their identities to inspire more people. Also, wondering. Are most of them financially independent to have made this decision? When money is not an worry, people have freedom to truly align themselves externally with their internal core values. If you are constantly worried paying rent or securing your kids future - people make compromises. That is not ideal to build a great society.
- neves 8y agoI just want to congratulate those Googlers. It isn't something usual to quit a good job due to ethical concerns. The World would be a lot better if there were more people like you.
- deleted 8y ago[deleted]
- cobookman 8y agoThe list internally is pretty small. Its like 2-4 people total. Which isn't statistically significant.