29 ms·

Leaked Emails Show Google Expected Military Drone AI Work to Grow Exponentially

535 points by raleighm 8y ago

deleted 8y ago
[deleted]
calibas 8y ago
Is my moral compass off, or does war profiteering completely violate the whole "don't be evil" thing?
- AmericanChopper 8y ago
  I guess that depends on whether you view either profit or war as intrinsically evil.
  - kibwen 8y ago
    It's whether one views the intersection of profit and war as intrinsically evil, which for most people is a resounding yes. Humans are already good enough at killing and causing misery for each other without adding additional economic incentives into the mix.
    - jonnybgood 8y ago
      But what if the US military wants to minimize civilian casualties and collateral damage as much as possible and wants technology developed to do so?
      - wavefunction 8y ago
        We don't know that Google will accomplish those goals.
        jonnybgood 8y ago
        Well I think that's what the Air Force wants to find out.
      - SiempreViernes 8y ago
        The obvious way to reach the goal of minimize civilian casualities is to abstain from risky strikes and use smaller yield weapons. The us military wants technology that justifies them not making more conservative choices, and I don’t see any moral justification for that. Drones are useful for cheaply eliminating risk to one actor, thus allowing more risky behaviour to the side that has them. But being a cheap risk mitigator is also all they are, they bring no new strategic capability, as evidenced by how easily their use was ramped up and down, so their main effect, the one that could provide some moral justification, is making war cheaper.
      - randcraw 8y ago
        Unfortunately I'm not aware that minimization of civilian casualties is a US military objective. Until I hear that it is, I will assume it is not, since it's much easier to achieve your mission of killing bad guys if you're also willing to kill good guys. You get more rewards for killing more bad guys, not saving more good guys. Unfortunately that's how militaries and governments work.
        AndrewKemendo 8y ago
        Unfortunately I'm not aware that minimization of civilian casualties is a US military objective. Not to put too fine of a point on it but even a basic search on the topic would yield a ton of official policy doing exactly that. There is explicit guidance and long history of minimizing civilian casualties as a goal within the DoD, and such guidance is codified in multiple legal documents under the Uniform Code of Military Justice, Rules of Engagement and international treaties. To wit: https://obamawhitehouse.archives.gov/the-press-office/2016/07/01/executive-order-united-states-policy-pre-and-post-strike-measures https://obamawhitehouse.archives.gov/the-press-office/2016/0... https://www.globalsecurity.org/military/library/policy/army/attp/attp3-37-31.pdf https://www.globalsecurity.org/military/library/policy/army/... This is also taught as a basic tenet in introductory courses for new Military Officers. One example here: https://www.trngcmd.marines.mil/Portals/207/Docs/TBS/B130936%20Law%20of%20War%20and%20Rules%20Of%20Engagement.pdf https://www.trngcmd.marines.mil/Portals/207/Docs/TBS/B130936...
        randcraw 8y ago
        A large set of such references are to be expected, of course. When a civilized people undertake killing on a large scale, you don't want others to think you act without justification. So you write serious administrative documents claiming your government actions to be justified and your military claims to minimize collateral damage. But are such claims really true? Does the part of the military chain of command that actually does the killing take those claims to heart, as much as pentagon spokesmen do? Do the superiors of lieutenants and privates hold them to that standard, even when the laws of conduct become unenforceable by collateral victims, as in drone strikes? In several US wars, such claim were not taken to heart. Vietnam became infamous in US history as lacking a sound casus belli, just as the 2003 Iraq War has. In messy wars like these, one of the principal casualties is often the enforcement of the rules of war, especially when it comes to laws of conduct that are still in the course of being written, such as the proper use of drones or AI. This lack of enforcement becomes especially fraught when novel weaponry and its use is governed not by the military under its code of conduct, but by the CIA (who now runs most drone snipery), which AFAIK lacks an equivalent martial code of conduct.
    - AmericanChopper 8y ago
      But you’re describing some commercial inventive to go to war, which is very different from selling goods and services to the military.
      - rickycook 8y ago
        those 2 things are pretty heavily correlated. guns and ammo, military rations manufacturers, and yes creators of technology that support military operations have a huge economic incentive for war the more war you create, the larger the military, and get larger their ability to spent on “protection” and “defence” *edit: which is certainly offset pretty substantially by the hit to the countries GDP, because yknow... war is expensive!
        AmericanChopper 8y ago
        There is no logic to your argument here at all. You’re saying that every company that supplies anything to the military is automatically wielding an unethical influence to create war on false pretences. I think you can certainly find examples where commercial entities have played a suspicious role in that respect, but to say that simply by supplying goods or services to the military you are war mongering is completely devoid of logic.
        krageon 8y ago
        He is saying that any company that supplies to a military has an economic incentive to make sure war happens. Seeing as they profit from the size of the military, that's not exactly far-fetched.
        AmericanChopper 8y ago
        He’s making an enormous and baseless jump that any company that supplies the military must also be involved corruption and unethical war mongering. Just because an economic insentive exists, doesn’t mean people will exploit it. That’s just as senseless as saying that I’m probably a drug dealer because there’s an economic insentive for me to be. Aside from the complete lack of logic in this claim, it also doesn’t even make sense, since the military gets a budget every year, regardless of whether there’s a war to fight.
  - swampthinker 8y ago
    War will always be evil, it's usually just about whether it is a necessary evil.
    - AmericanChopper 8y ago
      I think that’s a very oversimplified opinion. Depending on the alternative, war can certainly be the moral decision. I don’t think you’d find many would who would describe The Unions war on slavery as Evil.
      - wavefunction 8y ago
        It wasn't a war on Slavery though, as Lincoln only took the extraordinary step of releasing the Emancipation Proclamation late in the war.
        AmericanChopper 8y ago
        That’s a semantic argument. The point is that any of us could come up with an example where the decision to go to war, either offensively or defensively, is clearly not evil.
        will4274 8y ago
        Yes, it was. South Carolina was the first state to secede; South Carolina cited slavery in their declaration of secession; the war began with secession or immediately following and because of secession. The civil war was fought over slavery.
      - dragonwriter 8y ago
        The Union didn't fight a war on slavery (in fact, it included slave states). The Confederacy fought for slavery, the Union fought for, well, union.
  - mtgx 8y ago
    Was IBM's providing of tech to the Nazis to kill Jewish people more effectively neutral or intrinsically evil? I try to avoid comparisons to the Nazis (even though many more people seem to be doing it these days with Trump in power), but isn't Google basically providing the government the technology to more effectively kill people in the Middle East, too? That's what it is at the end of the day, isn't it? History will remember Google for this - especially when its technology will be used for some horrifying stuff in the future (and it will be). Right now we haven't even seen its true consequences yet, so it's easy for some to dismiss it as no big deal.
    - AmericanChopper 8y ago
      This isn’t a well reasoned position at all. You can’t summarily declare that all wars or actions taken by the military are morally equivalent to an ethnoreligios genocide.
      - John_KZ 8y ago
        Is there another use for these drones right now? Because all I've seen is pretty close to "ethnoreligious genocide". Many people straightforwardly advocate for it. Can you really pretend that you're developing "neutral" weapons technology in this climate?
        AmericanChopper 8y ago
        Appealing to mindless outrage is not a rational argument. Please list all of the genocides that the USA’s drone program has participated in. If anything, it seems this google technology would reduce the number of civilian casualties.
- vanake 8y ago
  that is why they removed it from Google’s Code of Conduct
  - cornstalks 8y ago
    It wasn’t removed. It was moved towards the end. You’re welcome to have an opinion on that if you want, but let’s be accurate and honest about it and stop spreading the incorrect claim it was removed.
    - emptybits 8y ago
      Let's be accurate, then. It's disingenuous to simply say it was moved towards the end. It was always at the end but until approximately a month ago, Google actually led its Code of Conduct with "don't be evil" in the very first sentence. That was removed. Then again "don't be evil" was in the second paragraph. That was removed. Sources: April 21, 2018: https://web.archive.org/web/20180421105327/https://abc.xyz/investor/other/google-code-of-conduct.html https://web.archive.org/web/20180421105327/https://abc.xyz/i... May 4, 2018: https://web.archive.org/web/20180504211806/https://abc.xyz/investor/other/google-code-of-conduct.html https://web.archive.org/web/20180504211806/https://abc.xyz/i...
- hazz99 8y ago
  They removed that clause recently, I believe.
  - kibwen 8y ago
    "Google Removes 'Don't Be Evil' Clause From Its Code of Conduct": https://gizmodo.com/google-removes-nearly-all-mentions-of-dont-be-evil-from-1826153393 https://gizmodo.com/google-removes-nearly-all-mentions-of-do... "“Don’t be evil” has been part of the company’s corporate code of conduct since 2000. When Google was reorganized under a new parent company, Alphabet, in 2015, Alphabet assumed a slightly adjusted version of the motto, “do the right thing.” However, Google retained its original “don’t be evil” language until [late April or early May 2018]. [...] The updated version of Google’s code of conduct still retains one reference to the company’s unofficial motto—the final line of the document is still: “And remember… don’t be evil, and if you see something that you think isn’t right – speak up!”"
    - cornstalks 8y ago
      That last sentence directly contradicts the title...
      - mtgx 8y ago
        It doesn't. The code of conduct used to start and center around the Don't be evil mantra. Now it seems to be in there just one last time before they completely eliminate, and mentioned in passing at the very end of a 3-page document: https://web.archive.org/web/20180421105327/https://abc.xyz/investor/other/google-code-of-conduct.html https://web.archive.org/web/20180421105327/https://abc.xyz/i...
  - deleted 8y ago
    [deleted]
- killjoywashere 8y ago
  [deleted]
  - deleted 8y ago
    [deleted]
  - TAForObvReasons 8y ago
    You are talking besides the point. War != "War Profiteering". You can believe war is unavoidable and still find excess profiting from war morally repugnant
    - killjoywashere 8y ago
      The company with the single best resources to execute on this mission is trying to help make sure the kinetic energy doesn't get off target. How is that evil, exactly?
      - Iv 8y ago
        It is evil when the target choice is immoral. Simple as that.
      - rdiddly 8y ago
        By putting it on the target, duh. If I shoot you I don't get hailed as a hero for saving your family's lives and neither does the gun sight manufacturer. Especially since they did it for the money.
  - Waterluvian 8y ago
    If you're trying to make a point, it is entirely lost in the rambling.
    - killjoywashere 8y ago
      Read The Dictator's Handbook. Then come back and tell me if I'm rambling. https://www.amazon.com/Dictators-Handbook-Behavior-Almost-Politics/dp/1610391845 https://www.amazon.com/Dictators-Handbook-Behavior-Almost-Po...
- mtgx 8y ago
  Good thing they removed all mentions of that from their workers' code of conduct then, no? They really dodged a bullet there (get it?) https://www.digitaltrends.com/computing/google-dont-be-evil/ https://www.digitaltrends.com/computing/google-dont-be-evil/ Moral compass out of sight, out of mind, is probably what the leadership must have been thinking, especially after the recent protest against Maven from a few thousand employees (which seems to have resulted only in the leadership trying to appease them by promising some secret bullshit "ethics rules")
  - joshuamorton 8y ago
    Except they didn't, it's still in the code of conduct ;)
    - mtgx 8y ago
      It used to be a central piece. Now it barely gets a "Oh, and don't be evil guys, k?" mention in the very last line.
      - joshuamorton 8y ago
        "removed all mention" is entirely false though. And while I don't particularly care for this change, calling the concluding sentence of a document "a passing mention" feels forced. I would agree it's been deemphasized and should be mentioned more, but a concluding sentence is pretty central to a document. It's meant to sum up the entire document.
- ChuckMcM 8y ago
  When you are dying perhaps your definitions change. I hold the controversial (and often dismissed) opinion that Google is dying. I assert they are dying in part because they built a company around search advertising before they knew how to build a company, their initial success gave them the confidence and agency to create structural impediments to their ability to do anything but search advertising. And the search advertising market is not only saturated it is getting more competitive as the easy gains to be made there have already been milked for what they are worth. I support that opinion on the meager evidence of how they more and more cravenly monetize properties that they previously kept free of such things, their repeated failed attempts at building any other business outside of search that brings anything close to the margins they need for their business model to work, and the lack of 'barriers' to other well funded and durable companies from taking away everything that is special from them. (Bing, Uber, Cruise, Facebook, Apple, Etc.) They get less and less per click and they spend more and more to buy more traffic for their search boxes as documented in their financial statements for the last decade. They have repeatedly come under attack from outside agencies which see their use of, and exploitation of data about people as either anti-competitive or a violation of basic human rights. Those efforts have resulted in fines and legislation that has been cutting off other ways to monetize the traffic of their place in search. From that misguided perspective, military contracts make total sense. What they have is a tremendous infrastructure that can be wielded by a relatively small number of engineers to do amazing things. For now, that capability is unique. By turning it from a cost center into a profit center (or even a less cost center) it can help them make the profits they need to keep things rolling. When I was working there I got to see what spending 4 billion dollars a year on infrastructure buys you. It rivals what nation states can field, it rivals what the US government can field. Renting that out to a price insensitive buyer for a unique asset? Sounds like saving yourself from dying to me.
  - thewarrior 8y ago
    Define dying ? I don't see how Google can go out of business in the next 10 years given their cash reserves. Eg: If they lost 2 billion dollars every year they could still keep doing that for decades. I'm pretty confident that they'll still be around in some form 20 years from now as well.
- deleted 8y ago
  [deleted]
- cwkoss 8y ago
  https://gizmodo.com/google-removes-nearly-all-mentions-of-dont-be-evil-from-1826153393 https://gizmodo.com/google-removes-nearly-all-mentions-of-do... Google is allowing their employees to be evil now
narrator 8y ago
Meanwhile, in Syria: https://www.youtube.com/watch?v=B5LdagBBo-k https://www.youtube.com/watch?v=B5LdagBBo-k
wavefunction 8y ago
That's exactly what I thought when the point was brought up that the contract between Google and the DoD "was only for $9,000,000." That's called getting your foot in the door.
aetherson 8y ago
I think that the leaked emails show that one or two execs within Google expressed the optimistic case for the eventual revenue leading from the project. That's how projects happen in companies. Some employee gets excited about the potential of the project and writes emails talking about all the huge numbers that project will post. Most of those emails are wrong. Maven may be wrong at any price or size. But Google's characterization of it as a minor project doesn't seem contradicted by these emails.
- aunty_helen 8y ago
  If it was only meant to be a $9 millon project, why take the risk? Google aren't that badly cash strapped.
  - bb88 8y ago
    Because the military contracts are really lucrative. Many are worth billions on their face. Performing well on a 9 million dollar contract opens the door to larger contracts.
    - ocdtrekkie 8y ago
      Isn't this, in itself, a huge problem? Shouldn't public service be a low margin project? Could the fact that military contracts are so needlessly "lucrative" (despite the government having a literal monopoly on who you can sell advanced weaponry to) be why such a huge margin of your taxes go to defense?
      - mrep 8y ago
        Eh, government contracts come with all sorts of red tape that make that "lucrative" contract not so good once all the hidden costs are considered.
        bb88 8y ago
        Yeah, there is red tape, but government contractors make this a core part of their business, so the relative risk is low. For a new company, the risk is relatively high, because there's a lot of up front work that has to be done (auditing, compliance, security, etc). But once that is ironed out, larger contracts become easier and easier. And then the only real risk is on a "firm fixed price" style of contract where the contractor assumes all the risk of product delivery. But many other are "time and materials" where the risk is shifted onto the government. If you fail to deliver a product, you're not on the hook because the government was paying for "time and materials" not the end product itself. So yes, google may actually lose money on a $9MM contract, but probably not on the next $100MM contract.
fakescience 8y ago
To any Google Employees reading this. Don't worry! There are SEVERAL ways you can reduce any cognitive dissonance you might be experiencing as you read more about the evils your company is committing : + rationalization of your own behavior : "I'm not doing this work...I'm working on projects that help people. And ads. But ads are pretty neutral compared to bombing people." + rationalization of others' behavior : "Bombing people isn't evil, since it's helping the military kill MORE _bad guys_ and LESS good guys. And gals. and children." + change your thoughts about yourself : "Maybe I'm not as good of a person as I thought I was..." [note : this one might be hardest to do!] + change your own behavior : Quit Google, or even better organize your fellow workers into some form of collective action (strike!) to remake the company into something that better fits your values and ethics.
- flatline 8y ago
  Or accept the reality that we live in a world that feels the need to kill people. I think it would be swell to not need a military but the reality is that we do. If you have qualms about working for Google as a defense contractor, consider the implications of living in the US (or in all likelihood wherever you happen to reside). Can you justify paying your taxes? Can you justify voting for one of a handful of candidates for federal government, all of whom are likely to support some form of military action or other? If a hostile foreign power was to invade your country, what would you want the response to be? To what ends and what efforts should we support the military? There are many ethically questionable choices we take for granted, especially those of us that live in the West. Everything we consume has a taint of human exploitation, environmental damage, contributes to oppressive regimes that murder people, etc. Where does it stop? On the other hand, it sounds like the work Google was doing here was to improve the relative safety of autonomous drone strikes. Oh, but if these strikes remained risky and unreliable, we wouldn’t do them! Believe me, there are a dozen other large defense contractors out there that will implement this if Google doesn’t, so it’s not like it’s a binary proposition. My point is, working for Google is not the right target on which to pin arguments for or against the ethics of autonomous drone strikes.
  - fakescience 8y ago
itronitron 8y ago
Gee, if only Google was part of a larger umbrella organization they could have just transferred the project to a separate company and been done with it.
- ocdtrekkie 8y ago
  If none of us buy that nonsense, why would Google employees? Alphabet = Google. We all know that.
sqdbps 8y ago
The irony of this particular piece is that they are publishing emails expressing concern about how the media might frame and distort such an item and doing just that. The email discussion is mainly about potential revenue if they were to win the JEDI contract it's not about maven or 'expecting the drone ai work to grow'. Also worth noting that Google isn't the only contractor for maven yet they are made as such in most articles. As for the issue at large: Microsoft and Amazon have recently surpassed Google in market value and that is largely due to cloud, they also manage to not have their employees leak internal material to the press and protest publicly, and I think Google should look into achieving that.
- repolfx 8y ago
  Google probably can't change that, at least not overnight. The reason Google are experiencing leaks and taking flak for this is that it's only Google that ever claimed to be a superior moral actor. Amazon and Microsoft, for all their faults, never claimed to be much more than tech companies that want to make money - the extent of Microsoft's moral ambition was "computers are good, let's put a computer in every home", a goal which is hard to argue with given the attendant positive consequences of computing for society. Google, on the other hand, has frequently made the following claims: • It won't "be evil", whatever that means. • It has an ethics board, for AI specifically. • That it withdrew from the Chinese search market due to being unwilling to go along anymore with the government's censorship. • That it "believes in the empowering and democratizing effect of putting information in the hands of everyone, everywhere" • From its code of conduct, that "everything we do in connection with our work at Google will be, and should be, measured against the highest possible standards of ethical business conduct" And I would say in general, having once been a part of it, that parts of Google's workforce are quite convinced of their own moral superiority. Look at the relentless feminist virtue circus around the firing of Damore, a man whose crime was merely to point out that men and women have different interests and that's why so many jobs have unbalanced ratios. The company's top management went on TV and said they were convinced firing him was the "right" thing to do. Well you can't do all those things and then get upset when employees start leaking behaviour they feel to be unethical. Live by the sword, die by the sword. Summary: Google gets shit for working on the drone strike program for money, because (a) it has more money than it knows what to do with already and (b) it has constantly claimed to be a new kind of tech company that is ethically superior to the previous generation. Other firms lack either (a) or (b) or both.
mturmon 8y ago
This is actually very interesting, being the first detailed info on this work that I have seen. First, the scale (dollars/FTEs) of the work (and proposed work) seems larger than in some earlier reporting. Bizdev people will have incentive to boost projections, but even the current numbers seem higher. Second, the little fact that “...On October 27, 2017, a team from Google Cloud visited Beale Air Force Base...”. This indicates they were not just doing basic research, but they were trying to (eventually) field something operationally — not just develop basic technology and hand it off elsewhere for maturation. Also, the reciprocal visit of the uniformed general [2]. The presence of a general and entourage, probably also in uniform, makes a strong impression on civilian computer technologists. In the DoD world, there is a big difference between basic research (TRL 1-3, say [1]) versus TRL 4-5 (component demos) versus TRL 6-8 (operational tests). Basic research might be funded by DARPA and is pretty abstract, typically publishable in the open literature. As you work up toward operations, you start to deal directly with uniformed “warfighters” and realistic test environments. It’s a very different set of expectations. I think I understand better the reactions of many Googlers to the work described. (Note: not passing personal judgement - clearly I’ve spent some time in this milieu — just observing that this work and its trappings is not what many Googlers I have known would go for.) [1] https://en.m.wikipedia.org/wiki/Technology_readiness_level https://en.m.wikipedia.org/wiki/Technology_readiness_level [2] http://www.af.mil/About-Us/Biographies/Display/Article/108830/major-general-john-nt-jack-shanahan/ http://www.af.mil/About-Us/Biographies/Display/Article/10883...
strooper 8y ago
I don't see anything wrong for a company of a particular country helping their military. GE, Siemens, Mitsubishi and all tech giants of that era did that with pride during WWII. Then why can't Google, Facebook for USA? Perhaps the illusion of globalization is making us perceive Google, Facebook and other international giants as global companies. Although, geniuses from around the world have significant contributions in the success of these technology companies, after all, they are loyal only to profit and the USA. When a war breaks loose, these data monsters are one of the best weapons against the opponents.
- pera 8y ago
  During WWII Siemens owned a factory in Auschwitz and manufactured electrical parts for concentration camps and death camps.
- oliveshell 8y ago
  Hell of a comparison. “Why can’t they?,” you ask? I’m not saying they can’t, but for a start, how about the fact that we are no longer fighting a world war?
  - non_sequitur 8y ago
    And do you know what it is that really upholds the peace we all love?
    - oliveshell 8y ago
      What in God’s name are you asking? You seem confused as to the nature of my comment, which was merely pointing out the massive difference in context between WWII and today.
      - non_sequitur 8y ago
        So your contention is that you have to be fighting a world war in order to help your military? (The post you responded to) There's no other reason to help your military? Really? Edit: this came out a bit snide so let me elaborate. There are plenty of reasons you may want to help your military when there isn't a world war going on. You can do it for patriotism. You can do it for the economics. You can do it because military applications of technology are actually pretty cool and can lead to more commercial inventions. Or, to the point in my original response, you can do it because you understand deep down that the threat of military force is what underpins the overall fairly secure world that we live in. Look at the military maneuvers and carrier group movements every time two countries get nervous around each other. Despite (or maybe because of) evolution were still just a step away from clubbing our neighbor (or being clubbed) for resources. So yes, a strong military helps support the country Google was founded in and thrived in, and helps Google and every other company in the US be successful, and makes possible the salaries we all make. So there are definitely incentives for those companies to want to make sure their military is properly supported. And no, I'm not a military man.
reacharavindh 8y ago
It isn't wrong of Google to work their country's military. If not Google, some other company will take those contracts. They need to own up the responsibility for their decisions and of course be willing to put on with the media theme that "Google - the company that supplies AI tech to the military with possible usage in war". Denying that is what makes them look even fishier. I already distrust the company for anything related to privacy. Just one more reason to hate them more.
- opportune 8y ago
  Nothing wrong with IBM helping the Nazis count prisoners. If not them, some other company would have taken the contract This absurdly cowardly thinking is how people justify atrocities for money
  - Goronmon 8y ago
    You're exactly right. Working with the military is exactly the same as helping the Nazis.
  - reacharavindh 8y ago
    Treating companies as social personalities is just stupid. Google is a business, and it is there to make money (whilst not damaging itself too much that it goes out of existence). My point is, that Google as a company(their leadership) could choose to work with the military, and even openly build weapons for a country, as long as that motive is public, and bear the hate from public that they are a piece of shit company that whored out to the military. But, cat calling a company on forums whilst there are other companies that openly build weapons is just useless in my opinion.
    - opportune 8y ago
      Essentially everything we do on forums is useless, I don’t know why calling a company immoral is any different. All businesses exist to make money but if the pushback to a publicly perceived immoral act is great enough then they won’t do the immoral things. For example, I can say as a software developer that I would prefer not to work at google over my current employer because I don’t want to do immoral things. I do have a perceived moral outlook of the company’s leadership as a whole and that’s a decision that I can make. Not sure why you seem to think that’s stupid Also at the end of the day, nobody is absolved of guilt from doing something just because it made them money. That’s possibly the dumbest way to justify doing something as good
Proven 8y ago
Business’s good!
49bc 8y ago
If a company is going to build drone AI in use by the military, I feel MUCH better knowing that company is google and not some poor-quality contracting company.
xab9 8y ago
Now this world starting to feel more and more like a Gibson novel - while we had been promised it's going to be an Asimov one. I feel cheated somehow.
deleted 8y ago
[deleted]
dayaz36 8y ago
One of the contractual requirements for Google's acquisition of Deep Mind was that their technology will never be used for military purposes. I wonder if Google did this behind their back or Deep Mind is fully aware and is doing this willingly? "One constraint we do have— that wasn’t part of a committee but part of the acquisition terms—is that no technology coming out of Deep Mind will be used for military or intelligence purposes." -Demis Hassabis
- repolfx 8y ago
  Feels good but is useless. Google has its own AI research operation that's not the same as DeepMind. But even if it didn't, all they'd have to do is wait for some third party academics to publish a paper that improved slightly on a DeepMind paper, and then they could use it and say the "technology" didn't come from DeepMind. After all, where does tech come from? Standing on the shoulders of giants and all that.
LawnDart1 8y ago
This is what T800-101 really looks like https://en.wikipedia.org/wiki/General_Atomics_MQ-9_Reaper https://en.wikipedia.org/wiki/General_Atomics_MQ-9_Reaper
ctack 8y ago
Sounds a lot like Google is going to start killing people.
savanta 8y ago
Maybe I'm wrong but based on my experience if you are a company who is depending on end customers then you need to have a good reputation and a very strong and positive brand. Except your clients are totally dependent on your product with absolute zero alternative choice. Most military suppliers don't depend on private end customers so they can be successful without worrying about public opinion. I think Google is becoming more and more like companies Exxon Mobile or former giants like Malboro. People absolutely need their products so they can do whatever they want.
manigandham 8y ago
The world runs on violence. It's the fundamental force that keeps things in order. Modern society with all its rules and laws is nothing without enforcement, which is ultimately delivered via a monopoly license granted to the government with its military and police to keep everyone else safe and free. Being the best at deploying violence is how you defend against the chaos elsewhere. This isn't about "evil corporations" or misguided intentions, it's about human nature and how there will always be someone somewhere willing to cause harm. The only way to defeat them is by being better at it, but with the appropriate judgement to make sure its in proper defense. AI has already been a part of warfare for decades and the value of anything other than being the best is rapidly diminishing because of the exponential advantage that modern technology produces. You either stay ahead or lose the war before it's ever begun. Given that choice, I would much rather have the strongest military with all available resources and talent rather than worry and hope the rest of the world stays friendly, until the day it isn't.
- Sharlin 8y ago
  > You either stay ahead or lose the war before it's ever begun. We must be able to do better than that. http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ http://slatestarcodex.com/2014/07/30/meditations-on-moloch/
  - manigandham 8y ago
    It's been a long time since I've read this, but I believe it is stating what I said in much more eloquent terms. I'm unsure what you mean by "do better than that"?
    - Sharlin 8y ago
      I mean that what you posed is a false dilemma: we must be able to solve our coordination problems or face extinction.
      - manigandham 8y ago
eddd 8y ago
I wouldn't blame Google for this. It's sole purpose is to make money. So they make money wherever they can. People voted for a government that bombs other people, that's a consequence.
bronzeage 8y ago
Why US citizens hate their army so much? All i know is that if google announced it's working with our army on some AI projects, the response would be overwhelmingly positive, and people would praise Google for that. (I'm from Israel if you're wondering) And I think US are a minority in that the citizens despise their own government and army this much. Which really makes you wonder what's the whole point of democracy if you trust your own elected officials less than Russians trust their corrupted monarch.
- kome 8y ago
  well, why do Israelis love their army so much? I think to know the answer, but be aware Israel is a very special case for so many reasons. Edit: and the contrary is true, Americans loves their army, and they are all about that "thank you for your service" rhetoric, so perhaps just Israel top that.
  - myth_drannon 8y ago
    Most of Israelis serve in army that gives them an inside perspective of what is Army, how it functions and its goals. Americans see it from the mass media perspective only, which creates the same feelings in people like when non-Israelis see the news on Gaza fighting - very superficial and inducing very strong emotions. IMHO.
    - kome 8y ago
      Indeed, that's could be an explanation. Also Switzerland has conscription / compulsory draft, and among a lot of Swiss, the army is seen with sympathy. That said, I met a couple of Israelis that left the country for good because their experience with the army was so bad. I guess that there is some "voting with legs" going on.
    - aninhumer 8y ago
      >very superficial and inducing very strong emotions. IMHO. I think spending part of your young adult life participating in the actions and socialisation of an institution, and then living in a society where everyone around you shares that experience, creates far stronger emotional influence on people's judgements than seeing some people you don't know being shot in a far away country.
jxi 8y ago
Sundar's Google.
musicalentropy 8y ago
For people who still don't see any problem in this, I have to remind you that ultimately the guy on top of the military hierarchy in charge for selecting the drones with AI targets is... Donald Trump. If you're not scared now, I don't know what to do anymore. At some point, I thought that the main goal of these big startups with tons of claims and wishful thinking was to make the world a better place and help humanity to become a better version of itself. Instead now they create tools and weapons to "reduce collateral damages" in governemental induced wars (which have not solved anything over the last 20 years in Middle East but only increased cash flow in West pockets, made tons of death, both there and in West countries), and they have access to information related with most of the citizens at the same time. OK cool. In 10 years we will talk about whistleblowers killed by drones using AI as well.
- azangru 8y ago
  > I have to remind you that ultimately the guy on top of the military hierarchy in charge for selecting the drones with AI targets is... Donald Trump How can we be sure that this is so? - Can Trump make military decisions by himself (i.e. not choose among the ones that are offered to him by the military)? - Hasn't it been established recently that the military have the right to refuse his orders if they are particularly crazy (see e.g. https://www.independent.co.uk/news/world/americas/donald-trump-north-korea-crisis-nuclear-weapons-us-military-duty-refuse-illegal-instructions-war-a8055991.html https://www.independent.co.uk/news/world/americas/donald-tru...)?
deleted 8y ago
[deleted]
fdsak 8y ago
probably this is the right time to introduce an oath sth like this one https://github.com/Widdershin/programmers-oath https://github.com/Widdershin/programmers-oath
lucaspottersky 8y ago
it sucks that companies have this tendency to become nasty business
indescions_2018 8y ago
There must be some hard data. Looking back over the last 40+ years. Were research institutions that divested from defense funding during the Vietnam era adversely impacted. Perhaps short term. But on long scales there has been more than adequate consumer and private sector demand to compensate. Politically, the company could face a backlash in the form of antitrust scrutiny over search monopoly. Which Alphabet seems to be anticipating as historical inevitability. With the foresight even that perhaps breaking up large trusts is great for startup innovation ;) But it raises the general question. If given the option of taking $ to do basic science. Regardless of the funding entities motives. Do you do it anyway? An argument can be made to always take the money and do the science. You don't know what the world will look like in 10 years. Drone AI used for surgical strikes today will probably be civilian transferred to AgTech and planting seeds tomorrow. And there is always the possibility of a breakthrough. Among the many great examples are the Human Genome Project, the Manhattan Project and DARPA Grand Challenge. Terrestrial plutonium enrichment may turn out to be the key to deep space exploration, or space based defense. Early DOE-funded genetics research was specifically interested in answering the question: can we survive radiation induced mutations from the fallout of a nuclear war? And less than two decades after a Humvee from CMU completed 8 miles on its own in the California desert we will see fleets of semi-autonomous vehicles rolling out in major cities.
jdowner 8y ago
Do no evil* *conditions apply
deleted 8y ago
[deleted]
josefresco 8y ago
Dear young, super smart Googlers: If your knowledge of the Middle East and terrorism goes back to ISIS/Al Qaeda - stop coding (or selling ads) and visit your local library. The history section is chalk full of wonderfully informative books that will inform your opinions. You might find that Internet tropes about terrorism to be less than accurate. For example, I just finished "Rise and Kill First" [1] a rather enlightening book about Israel's use of targeted assassinations going back almost 100 years. When I started reading Israel wasn't in the news, but then the whole Iran/US nuclear deal fiasco hit, and it made Netanyahu's actions all the more clear. Note, I'm not saying justified but rather that I now understand why Israel thinks and acts the way it does, and how sometimes this clashes with their normally close allies like the US. 1: https://www.amazon.com/Rise-Kill-First-Targeted-Assassinations/dp/1400069718 https://www.amazon.com/Rise-Kill-First-Targeted-Assassinatio... Before that, I read "Ghost Wars" [2] another beast of a book that covers US involvement in the Middle East going back to the late 1970's up to 9/11. Fascinating book about the struggle to dominate Afghanistan by regional and world powers. [2] https://www.amazon.com/Ghost-Wars-Afghanistan-Invasion-September/dp/0143034669 https://www.amazon.com/Ghost-Wars-Afghanistan-Invasion-Septe... Before that, it was "The Brothers: John Foster Dulles, Allen Dulles, and Their Secret World War" [3] about the history of the CIA and the US's foreign policy in the last 100+ years. [3] https://www.amazon.com/Brothers-Foster-Dulles-Allen-Secret/dp/0805094970 https://www.amazon.com/Brothers-Foster-Dulles-Allen-Secret/d... Key point - These books look past recent events to provide an mostly complete understanding of these conflicts that you can't get with 20 minute Google searches and Wikipedia snippets.
- AndrewKemendo 8y ago
  Well beyond that, Military and Global Geopolitical research is a massive discipline that is studied most intensely by the Military itself. Just like most disciplines, it's not possible to get even a basic understanding of this issue with only a year or two of study.
new_age_garbage 8y ago
Why wouldn't you just spin off your military drone AI division. That seems like a no-brainer, even just for PR purposes. I'm not sure I'd want a Google rifle, based on the longevity of my first Pixel.
nevir 8y ago
The military is going to deploy deep learning on their drones, with or without Google's help. I'd rather have more people working on the problem than fewer. More scrutiny should, at least, improve the work and reduce errors in the models (and, hopefully, that leads to fewer casualties)
IanDrake 8y ago
Google developed a far left employee culture and then expected their employees to help the military. They don’t seem to understand how those two things are mostly incompatible.
beenBoutIT 8y ago
If Google refused to work with our military, Microsoft's subpar AI and ethics would power the drones and the project would move ahead.
- blindwatchmaker 8y ago
  Having to deal with UI developed by Microsoft is the only thing that could ever make me feel sorry for drone operators.
amaccuish 8y ago
People love to talk about Huawei's links to the People's Liberation Army, but when Google starts working with the US military, it's somehow different. Hmm...
- kaonashi 8y ago
  China doesn’t have the same (recent) history of ‘military adventurism’.
randcraw 8y ago
Apparently Google has decided NOT to pursue this line of work after all: https://gizmodo.com/google-plans-not-to-renew-its-contract-for-project-mave-1826488620 https://gizmodo.com/google-plans-not-to-renew-its-contract-f...
ggm 8y ago
Norbert Weiner
carapace 8y ago
"I Have No Mouth, and I Must Scream" by Harlan Ellison > The story takes place 109 years after the complete destruction of human civilization. The Cold War had escalated into a world war, fought mainly between China, Russia, and the United States. As the war progressed, the three warring nations each created a super-computer (with AI) capable of running the war more efficiently than humans. > The machines are each referred to as "AM", which originally stood for "Allied Mastercomputer", and then was later called "Adaptive Manipulator". Finally, "AM" stands for "Aggressive Menace". One day, one of the three computers becomes self-aware, and promptly absorbs the other two, thus taking control of the entire war. It carries out campaigns of mass genocide, killing off all but four men and one woman. > The survivors live together underground in an endless complex, the only habitable place left. The master computer harbors an immeasurable hatred for the group and spends every available moment torturing them. AM has not only managed to keep the humans from taking their own lives, but has made them virtually immortal. https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Scream https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Sc...

535 points by raleighm 8y ago

deleted 8y ago

[deleted]

calibas 8y ago

Is my moral compass off, or does war profiteering completely violate the whole "don't be evil" thing?

AmericanChopper 8y ago
I guess that depends on whether you view either profit or war as intrinsically evil.
- kibwen 8y ago
  It's whether one views the intersection of profit and war as intrinsically evil, which for most people is a resounding yes. Humans are already good enough at killing and causing misery for each other without adding additional economic incentives into the mix.
  - jonnybgood 8y ago
    But what if the US military wants to minimize civilian casualties and collateral damage as much as possible and wants technology developed to do so?
    - wavefunction 8y ago
      We don't know that Google will accomplish those goals.
      - jonnybgood 8y ago
        Well I think that's what the Air Force wants to find out.
    - SiempreViernes 8y ago
      The obvious way to reach the goal of minimize civilian casualities is to abstain from risky strikes and use smaller yield weapons. The us military wants technology that justifies them not making more conservative choices, and I don’t see any moral justification for that. Drones are useful for cheaply eliminating risk to one actor, thus allowing more risky behaviour to the side that has them. But being a cheap risk mitigator is also all they are, they bring no new strategic capability, as evidenced by how easily their use was ramped up and down, so their main effect, the one that could provide some moral justification, is making war cheaper.
    - randcraw 8y ago
      Unfortunately I'm not aware that minimization of civilian casualties is a US military objective. Until I hear that it is, I will assume it is not, since it's much easier to achieve your mission of killing bad guys if you're also willing to kill good guys. You get more rewards for killing more bad guys, not saving more good guys. Unfortunately that's how militaries and governments work.
      - AndrewKemendo 8y ago
        Unfortunately I'm not aware that minimization of civilian casualties is a US military objective. Not to put too fine of a point on it but even a basic search on the topic would yield a ton of official policy doing exactly that. There is explicit guidance and long history of minimizing civilian casualties as a goal within the DoD, and such guidance is codified in multiple legal documents under the Uniform Code of Military Justice, Rules of Engagement and international treaties. To wit: https://obamawhitehouse.archives.gov/the-press-office/2016/07/01/executive-order-united-states-policy-pre-and-post-strike-measures https://obamawhitehouse.archives.gov/the-press-office/2016/0... https://www.globalsecurity.org/military/library/policy/army/attp/attp3-37-31.pdf https://www.globalsecurity.org/military/library/policy/army/... This is also taught as a basic tenet in introductory courses for new Military Officers. One example here: https://www.trngcmd.marines.mil/Portals/207/Docs/TBS/B130936%20Law%20of%20War%20and%20Rules%20Of%20Engagement.pdf https://www.trngcmd.marines.mil/Portals/207/Docs/TBS/B130936...
        randcraw 8y ago
        A large set of such references are to be expected, of course. When a civilized people undertake killing on a large scale, you don't want others to think you act without justification. So you write serious administrative documents claiming your government actions to be justified and your military claims to minimize collateral damage. But are such claims really true? Does the part of the military chain of command that actually does the killing take those claims to heart, as much as pentagon spokesmen do? Do the superiors of lieutenants and privates hold them to that standard, even when the laws of conduct become unenforceable by collateral victims, as in drone strikes? In several US wars, such claim were not taken to heart. Vietnam became infamous in US history as lacking a sound casus belli, just as the 2003 Iraq War has. In messy wars like these, one of the principal casualties is often the enforcement of the rules of war, especially when it comes to laws of conduct that are still in the course of being written, such as the proper use of drones or AI. This lack of enforcement becomes especially fraught when novel weaponry and its use is governed not by the military under its code of conduct, but by the CIA (who now runs most drone snipery), which AFAIK lacks an equivalent martial code of conduct.
  - AmericanChopper 8y ago
    But you’re describing some commercial inventive to go to war, which is very different from selling goods and services to the military.
    - rickycook 8y ago
      those 2 things are pretty heavily correlated. guns and ammo, military rations manufacturers, and yes creators of technology that support military operations have a huge economic incentive for war the more war you create, the larger the military, and get larger their ability to spent on “protection” and “defence” *edit: which is certainly offset pretty substantially by the hit to the countries GDP, because yknow... war is expensive!
      - AmericanChopper 8y ago
        There is no logic to your argument here at all. You’re saying that every company that supplies anything to the military is automatically wielding an unethical influence to create war on false pretences. I think you can certainly find examples where commercial entities have played a suspicious role in that respect, but to say that simply by supplying goods or services to the military you are war mongering is completely devoid of logic.
        krageon 8y ago
        He is saying that any company that supplies to a military has an economic incentive to make sure war happens. Seeing as they profit from the size of the military, that's not exactly far-fetched.
        AmericanChopper 8y ago
        He’s making an enormous and baseless jump that any company that supplies the military must also be involved corruption and unethical war mongering. Just because an economic insentive exists, doesn’t mean people will exploit it. That’s just as senseless as saying that I’m probably a drug dealer because there’s an economic insentive for me to be. Aside from the complete lack of logic in this claim, it also doesn’t even make sense, since the military gets a budget every year, regardless of whether there’s a war to fight.
- swampthinker 8y ago
  War will always be evil, it's usually just about whether it is a necessary evil.
  - AmericanChopper 8y ago
    I think that’s a very oversimplified opinion. Depending on the alternative, war can certainly be the moral decision. I don’t think you’d find many would who would describe The Unions war on slavery as Evil.
    - wavefunction 8y ago
      It wasn't a war on Slavery though, as Lincoln only took the extraordinary step of releasing the Emancipation Proclamation late in the war.
      - AmericanChopper 8y ago
        That’s a semantic argument. The point is that any of us could come up with an example where the decision to go to war, either offensively or defensively, is clearly not evil.
      - will4274 8y ago
        Yes, it was. South Carolina was the first state to secede; South Carolina cited slavery in their declaration of secession; the war began with secession or immediately following and because of secession. The civil war was fought over slavery.
    - dragonwriter 8y ago
      The Union didn't fight a war on slavery (in fact, it included slave states). The Confederacy fought for slavery, the Union fought for, well, union.
- mtgx 8y ago
  Was IBM's providing of tech to the Nazis to kill Jewish people more effectively neutral or intrinsically evil? I try to avoid comparisons to the Nazis (even though many more people seem to be doing it these days with Trump in power), but isn't Google basically providing the government the technology to more effectively kill people in the Middle East, too? That's what it is at the end of the day, isn't it? History will remember Google for this - especially when its technology will be used for some horrifying stuff in the future (and it will be). Right now we haven't even seen its true consequences yet, so it's easy for some to dismiss it as no big deal.
  - AmericanChopper 8y ago
    This isn’t a well reasoned position at all. You can’t summarily declare that all wars or actions taken by the military are morally equivalent to an ethnoreligios genocide.
    - John_KZ 8y ago
      Is there another use for these drones right now? Because all I've seen is pretty close to "ethnoreligious genocide". Many people straightforwardly advocate for it. Can you really pretend that you're developing "neutral" weapons technology in this climate?
      - AmericanChopper 8y ago
        Appealing to mindless outrage is not a rational argument. Please list all of the genocides that the USA’s drone program has participated in. If anything, it seems this google technology would reduce the number of civilian casualties.
vanake 8y ago
that is why they removed it from Google’s Code of Conduct
- cornstalks 8y ago
  It wasn’t removed. It was moved towards the end. You’re welcome to have an opinion on that if you want, but let’s be accurate and honest about it and stop spreading the incorrect claim it was removed.
  - emptybits 8y ago
    Let's be accurate, then. It's disingenuous to simply say it was moved towards the end. It was always at the end but until approximately a month ago, Google actually led its Code of Conduct with "don't be evil" in the very first sentence. That was removed. Then again "don't be evil" was in the second paragraph. That was removed. Sources: April 21, 2018: https://web.archive.org/web/20180421105327/https://abc.xyz/investor/other/google-code-of-conduct.html https://web.archive.org/web/20180421105327/https://abc.xyz/i... May 4, 2018: https://web.archive.org/web/20180504211806/https://abc.xyz/investor/other/google-code-of-conduct.html https://web.archive.org/web/20180504211806/https://abc.xyz/i...
hazz99 8y ago
They removed that clause recently, I believe.
- kibwen 8y ago
  "Google Removes 'Don't Be Evil' Clause From Its Code of Conduct": https://gizmodo.com/google-removes-nearly-all-mentions-of-dont-be-evil-from-1826153393 https://gizmodo.com/google-removes-nearly-all-mentions-of-do... "“Don’t be evil” has been part of the company’s corporate code of conduct since 2000. When Google was reorganized under a new parent company, Alphabet, in 2015, Alphabet assumed a slightly adjusted version of the motto, “do the right thing.” However, Google retained its original “don’t be evil” language until [late April or early May 2018]. [...] The updated version of Google’s code of conduct still retains one reference to the company’s unofficial motto—the final line of the document is still: “And remember… don’t be evil, and if you see something that you think isn’t right – speak up!”"
  - cornstalks 8y ago
    That last sentence directly contradicts the title...
    - mtgx 8y ago
      It doesn't. The code of conduct used to start and center around the Don't be evil mantra. Now it seems to be in there just one last time before they completely eliminate, and mentioned in passing at the very end of a 3-page document: https://web.archive.org/web/20180421105327/https://abc.xyz/investor/other/google-code-of-conduct.html https://web.archive.org/web/20180421105327/https://abc.xyz/i...
- deleted 8y ago
  [deleted]
killjoywashere 8y ago
[deleted]
- deleted 8y ago
  [deleted]
- TAForObvReasons 8y ago
  You are talking besides the point. War != "War Profiteering". You can believe war is unavoidable and still find excess profiting from war morally repugnant
  - killjoywashere 8y ago
    The company with the single best resources to execute on this mission is trying to help make sure the kinetic energy doesn't get off target. How is that evil, exactly?
    - Iv 8y ago
      It is evil when the target choice is immoral. Simple as that.
    - rdiddly 8y ago
      By putting it on the target, duh. If I shoot you I don't get hailed as a hero for saving your family's lives and neither does the gun sight manufacturer. Especially since they did it for the money.
- Waterluvian 8y ago
  If you're trying to make a point, it is entirely lost in the rambling.
  - killjoywashere 8y ago
    Read The Dictator's Handbook. Then come back and tell me if I'm rambling. https://www.amazon.com/Dictators-Handbook-Behavior-Almost-Politics/dp/1610391845 https://www.amazon.com/Dictators-Handbook-Behavior-Almost-Po...
mtgx 8y ago
Good thing they removed all mentions of that from their workers' code of conduct then, no? They really dodged a bullet there (get it?) https://www.digitaltrends.com/computing/google-dont-be-evil/ https://www.digitaltrends.com/computing/google-dont-be-evil/ Moral compass out of sight, out of mind, is probably what the leadership must have been thinking, especially after the recent protest against Maven from a few thousand employees (which seems to have resulted only in the leadership trying to appease them by promising some secret bullshit "ethics rules")
- joshuamorton 8y ago
  Except they didn't, it's still in the code of conduct ;)
  - mtgx 8y ago
    It used to be a central piece. Now it barely gets a "Oh, and don't be evil guys, k?" mention in the very last line.
    - joshuamorton 8y ago
      "removed all mention" is entirely false though. And while I don't particularly care for this change, calling the concluding sentence of a document "a passing mention" feels forced. I would agree it's been deemphasized and should be mentioned more, but a concluding sentence is pretty central to a document. It's meant to sum up the entire document.
ChuckMcM 8y ago
When you are dying perhaps your definitions change. I hold the controversial (and often dismissed) opinion that Google is dying. I assert they are dying in part because they built a company around search advertising before they knew how to build a company, their initial success gave them the confidence and agency to create structural impediments to their ability to do anything but search advertising. And the search advertising market is not only saturated it is getting more competitive as the easy gains to be made there have already been milked for what they are worth. I support that opinion on the meager evidence of how they more and more cravenly monetize properties that they previously kept free of such things, their repeated failed attempts at building any other business outside of search that brings anything close to the margins they need for their business model to work, and the lack of 'barriers' to other well funded and durable companies from taking away everything that is special from them. (Bing, Uber, Cruise, Facebook, Apple, Etc.) They get less and less per click and they spend more and more to buy more traffic for their search boxes as documented in their financial statements for the last decade. They have repeatedly come under attack from outside agencies which see their use of, and exploitation of data about people as either anti-competitive or a violation of basic human rights. Those efforts have resulted in fines and legislation that has been cutting off other ways to monetize the traffic of their place in search. From that misguided perspective, military contracts make total sense. What they have is a tremendous infrastructure that can be wielded by a relatively small number of engineers to do amazing things. For now, that capability is unique. By turning it from a cost center into a profit center (or even a less cost center) it can help them make the profits they need to keep things rolling. When I was working there I got to see what spending 4 billion dollars a year on infrastructure buys you. It rivals what nation states can field, it rivals what the US government can field. Renting that out to a price insensitive buyer for a unique asset? Sounds like saving yourself from dying to me.
- thewarrior 8y ago
  Define dying ? I don't see how Google can go out of business in the next 10 years given their cash reserves. Eg: If they lost 2 billion dollars every year they could still keep doing that for decades. I'm pretty confident that they'll still be around in some form 20 years from now as well.
deleted 8y ago
[deleted]
cwkoss 8y ago
https://gizmodo.com/google-removes-nearly-all-mentions-of-dont-be-evil-from-1826153393 https://gizmodo.com/google-removes-nearly-all-mentions-of-do... Google is allowing their employees to be evil now

narrator 8y ago

Meanwhile, in Syria: https://www.youtube.com/watch?v=B5LdagBBo-k https://www.youtube.com/watch?v=B5LdagBBo-k

wavefunction 8y ago

That's exactly what I thought when the point was brought up that the contract between Google and the DoD "was only for $9,000,000." That's called getting your foot in the door.

aetherson 8y ago

I think that the leaked emails show that one or two execs within Google expressed the optimistic case for the eventual revenue leading from the project. That's how projects happen in companies. Some employee gets excited about the potential of the project and writes emails talking about all the huge numbers that project will post. Most of those emails are wrong. Maven may be wrong at any price or size. But Google's characterization of it as a minor project doesn't seem contradicted by these emails.

aunty_helen 8y ago
If it was only meant to be a $9 millon project, why take the risk? Google aren't that badly cash strapped.
- bb88 8y ago
  Because the military contracts are really lucrative. Many are worth billions on their face. Performing well on a 9 million dollar contract opens the door to larger contracts.
  - ocdtrekkie 8y ago
    Isn't this, in itself, a huge problem? Shouldn't public service be a low margin project? Could the fact that military contracts are so needlessly "lucrative" (despite the government having a literal monopoly on who you can sell advanced weaponry to) be why such a huge margin of your taxes go to defense?
    - mrep 8y ago
      Eh, government contracts come with all sorts of red tape that make that "lucrative" contract not so good once all the hidden costs are considered.
      - bb88 8y ago
        Yeah, there is red tape, but government contractors make this a core part of their business, so the relative risk is low. For a new company, the risk is relatively high, because there's a lot of up front work that has to be done (auditing, compliance, security, etc). But once that is ironed out, larger contracts become easier and easier. And then the only real risk is on a "firm fixed price" style of contract where the contractor assumes all the risk of product delivery. But many other are "time and materials" where the risk is shifted onto the government. If you fail to deliver a product, you're not on the hook because the government was paying for "time and materials" not the end product itself. So yes, google may actually lose money on a $9MM contract, but probably not on the next $100MM contract.

fakescience 8y ago

To any Google Employees reading this. Don't worry! There are SEVERAL ways you can reduce any cognitive dissonance you might be experiencing as you read more about the evils your company is committing : + rationalization of your own behavior : "I'm not doing this work...I'm working on projects that help people. And ads. But ads are pretty neutral compared to bombing people." + rationalization of others' behavior : "Bombing people isn't evil, since it's helping the military kill MORE _bad guys_ and LESS good guys. And gals. and children." + change your thoughts about yourself : "Maybe I'm not as good of a person as I thought I was..." [note : this one might be hardest to do!] + change your own behavior : Quit Google, or even better organize your fellow workers into some form of collective action (strike!) to remake the company into something that better fits your values and ethics.

flatline 8y ago
Or accept the reality that we live in a world that feels the need to kill people. I think it would be swell to not need a military but the reality is that we do. If you have qualms about working for Google as a defense contractor, consider the implications of living in the US (or in all likelihood wherever you happen to reside). Can you justify paying your taxes? Can you justify voting for one of a handful of candidates for federal government, all of whom are likely to support some form of military action or other? If a hostile foreign power was to invade your country, what would you want the response to be? To what ends and what efforts should we support the military? There are many ethically questionable choices we take for granted, especially those of us that live in the West. Everything we consume has a taint of human exploitation, environmental damage, contributes to oppressive regimes that murder people, etc. Where does it stop? On the other hand, it sounds like the work Google was doing here was to improve the relative safety of autonomous drone strikes. Oh, but if these strikes remained risky and unreliable, we wouldn’t do them! Believe me, there are a dozen other large defense contractors out there that will implement this if Google doesn’t, so it’s not like it’s a binary proposition. My point is, working for Google is not the right target on which to pin arguments for or against the ethics of autonomous drone strikes.
- fakescience 8y ago

itronitron 8y ago

Gee, if only Google was part of a larger umbrella organization they could have just transferred the project to a separate company and been done with it.

ocdtrekkie 8y ago
If none of us buy that nonsense, why would Google employees? Alphabet = Google. We all know that.

sqdbps 8y ago

The irony of this particular piece is that they are publishing emails expressing concern about how the media might frame and distort such an item and doing just that. The email discussion is mainly about potential revenue if they were to win the JEDI contract it's not about maven or 'expecting the drone ai work to grow'. Also worth noting that Google isn't the only contractor for maven yet they are made as such in most articles. As for the issue at large: Microsoft and Amazon have recently surpassed Google in market value and that is largely due to cloud, they also manage to not have their employees leak internal material to the press and protest publicly, and I think Google should look into achieving that.

repolfx 8y ago
Google probably can't change that, at least not overnight. The reason Google are experiencing leaks and taking flak for this is that it's only Google that ever claimed to be a superior moral actor. Amazon and Microsoft, for all their faults, never claimed to be much more than tech companies that want to make money - the extent of Microsoft's moral ambition was "computers are good, let's put a computer in every home", a goal which is hard to argue with given the attendant positive consequences of computing for society. Google, on the other hand, has frequently made the following claims: • It won't "be evil", whatever that means. • It has an ethics board, for AI specifically. • That it withdrew from the Chinese search market due to being unwilling to go along anymore with the government's censorship. • That it "believes in the empowering and democratizing effect of putting information in the hands of everyone, everywhere" • From its code of conduct, that "everything we do in connection with our work at Google will be, and should be, measured against the highest possible standards of ethical business conduct" And I would say in general, having once been a part of it, that parts of Google's workforce are quite convinced of their own moral superiority. Look at the relentless feminist virtue circus around the firing of Damore, a man whose crime was merely to point out that men and women have different interests and that's why so many jobs have unbalanced ratios. The company's top management went on TV and said they were convinced firing him was the "right" thing to do. Well you can't do all those things and then get upset when employees start leaking behaviour they feel to be unethical. Live by the sword, die by the sword. Summary: Google gets shit for working on the drone strike program for money, because (a) it has more money than it knows what to do with already and (b) it has constantly claimed to be a new kind of tech company that is ethically superior to the previous generation. Other firms lack either (a) or (b) or both.

mturmon 8y ago

This is actually very interesting, being the first detailed info on this work that I have seen. First, the scale (dollars/FTEs) of the work (and proposed work) seems larger than in some earlier reporting. Bizdev people will have incentive to boost projections, but even the current numbers seem higher. Second, the little fact that “...On October 27, 2017, a team from Google Cloud visited Beale Air Force Base...”. This indicates they were not just doing basic research, but they were trying to (eventually) field something operationally — not just develop basic technology and hand it off elsewhere for maturation. Also, the reciprocal visit of the uniformed general [2]. The presence of a general and entourage, probably also in uniform, makes a strong impression on civilian computer technologists. In the DoD world, there is a big difference between basic research (TRL 1-3, say [1]) versus TRL 4-5 (component demos) versus TRL 6-8 (operational tests). Basic research might be funded by DARPA and is pretty abstract, typically publishable in the open literature. As you work up toward operations, you start to deal directly with uniformed “warfighters” and realistic test environments. It’s a very different set of expectations. I think I understand better the reactions of many Googlers to the work described. (Note: not passing personal judgement - clearly I’ve spent some time in this milieu — just observing that this work and its trappings is not what many Googlers I have known would go for.) [1] https://en.m.wikipedia.org/wiki/Technology_readiness_level https://en.m.wikipedia.org/wiki/Technology_readiness_level [2] http://www.af.mil/About-Us/Biographies/Display/Article/108830/major-general-john-nt-jack-shanahan/ http://www.af.mil/About-Us/Biographies/Display/Article/10883...

strooper 8y ago

I don't see anything wrong for a company of a particular country helping their military. GE, Siemens, Mitsubishi and all tech giants of that era did that with pride during WWII. Then why can't Google, Facebook for USA? Perhaps the illusion of globalization is making us perceive Google, Facebook and other international giants as global companies. Although, geniuses from around the world have significant contributions in the success of these technology companies, after all, they are loyal only to profit and the USA. When a war breaks loose, these data monsters are one of the best weapons against the opponents.

pera 8y ago
During WWII Siemens owned a factory in Auschwitz and manufactured electrical parts for concentration camps and death camps.
oliveshell 8y ago
Hell of a comparison. “Why can’t they?,” you ask? I’m not saying they can’t, but for a start, how about the fact that we are no longer fighting a world war?
- non_sequitur 8y ago
  And do you know what it is that really upholds the peace we all love?
  - oliveshell 8y ago
    What in God’s name are you asking? You seem confused as to the nature of my comment, which was merely pointing out the massive difference in context between WWII and today.
    - non_sequitur 8y ago
      So your contention is that you have to be fighting a world war in order to help your military? (The post you responded to) There's no other reason to help your military? Really? Edit: this came out a bit snide so let me elaborate. There are plenty of reasons you may want to help your military when there isn't a world war going on. You can do it for patriotism. You can do it for the economics. You can do it because military applications of technology are actually pretty cool and can lead to more commercial inventions. Or, to the point in my original response, you can do it because you understand deep down that the threat of military force is what underpins the overall fairly secure world that we live in. Look at the military maneuvers and carrier group movements every time two countries get nervous around each other. Despite (or maybe because of) evolution were still just a step away from clubbing our neighbor (or being clubbed) for resources. So yes, a strong military helps support the country Google was founded in and thrived in, and helps Google and every other company in the US be successful, and makes possible the salaries we all make. So there are definitely incentives for those companies to want to make sure their military is properly supported. And no, I'm not a military man.

reacharavindh 8y ago

It isn't wrong of Google to work their country's military. If not Google, some other company will take those contracts. They need to own up the responsibility for their decisions and of course be willing to put on with the media theme that "Google - the company that supplies AI tech to the military with possible usage in war". Denying that is what makes them look even fishier. I already distrust the company for anything related to privacy. Just one more reason to hate them more.

opportune 8y ago
Nothing wrong with IBM helping the Nazis count prisoners. If not them, some other company would have taken the contract This absurdly cowardly thinking is how people justify atrocities for money
- Goronmon 8y ago
  You're exactly right. Working with the military is exactly the same as helping the Nazis.
- reacharavindh 8y ago
  Treating companies as social personalities is just stupid. Google is a business, and it is there to make money (whilst not damaging itself too much that it goes out of existence). My point is, that Google as a company(their leadership) could choose to work with the military, and even openly build weapons for a country, as long as that motive is public, and bear the hate from public that they are a piece of shit company that whored out to the military. But, cat calling a company on forums whilst there are other companies that openly build weapons is just useless in my opinion.
  - opportune 8y ago
    Essentially everything we do on forums is useless, I don’t know why calling a company immoral is any different. All businesses exist to make money but if the pushback to a publicly perceived immoral act is great enough then they won’t do the immoral things. For example, I can say as a software developer that I would prefer not to work at google over my current employer because I don’t want to do immoral things. I do have a perceived moral outlook of the company’s leadership as a whole and that’s a decision that I can make. Not sure why you seem to think that’s stupid Also at the end of the day, nobody is absolved of guilt from doing something just because it made them money. That’s possibly the dumbest way to justify doing something as good

Proven 8y ago

Business’s good!

49bc 8y ago

If a company is going to build drone AI in use by the military, I feel MUCH better knowing that company is google and not some poor-quality contracting company.

xab9 8y ago

Now this world starting to feel more and more like a Gibson novel - while we had been promised it's going to be an Asimov one. I feel cheated somehow.

deleted 8y ago

[deleted]

dayaz36 8y ago

One of the contractual requirements for Google's acquisition of Deep Mind was that their technology will never be used for military purposes. I wonder if Google did this behind their back or Deep Mind is fully aware and is doing this willingly? "One constraint we do have— that wasn’t part of a committee but part of the acquisition terms—is that no technology coming out of Deep Mind will be used for military or intelligence purposes." -Demis Hassabis

repolfx 8y ago
Feels good but is useless. Google has its own AI research operation that's not the same as DeepMind. But even if it didn't, all they'd have to do is wait for some third party academics to publish a paper that improved slightly on a DeepMind paper, and then they could use it and say the "technology" didn't come from DeepMind. After all, where does tech come from? Standing on the shoulders of giants and all that.

LawnDart1 8y ago

This is what T800-101 really looks like https://en.wikipedia.org/wiki/General_Atomics_MQ-9_Reaper https://en.wikipedia.org/wiki/General_Atomics_MQ-9_Reaper

ctack 8y ago

Sounds a lot like Google is going to start killing people.

savanta 8y ago

Maybe I'm wrong but based on my experience if you are a company who is depending on end customers then you need to have a good reputation and a very strong and positive brand. Except your clients are totally dependent on your product with absolute zero alternative choice. Most military suppliers don't depend on private end customers so they can be successful without worrying about public opinion. I think Google is becoming more and more like companies Exxon Mobile or former giants like Malboro. People absolutely need their products so they can do whatever they want.

manigandham 8y ago

The world runs on violence. It's the fundamental force that keeps things in order. Modern society with all its rules and laws is nothing without enforcement, which is ultimately delivered via a monopoly license granted to the government with its military and police to keep everyone else safe and free. Being the best at deploying violence is how you defend against the chaos elsewhere. This isn't about "evil corporations" or misguided intentions, it's about human nature and how there will always be someone somewhere willing to cause harm. The only way to defeat them is by being better at it, but with the appropriate judgement to make sure its in proper defense. AI has already been a part of warfare for decades and the value of anything other than being the best is rapidly diminishing because of the exponential advantage that modern technology produces. You either stay ahead or lose the war before it's ever begun. Given that choice, I would much rather have the strongest military with all available resources and talent rather than worry and hope the rest of the world stays friendly, until the day it isn't.

Sharlin 8y ago
> You either stay ahead or lose the war before it's ever begun. We must be able to do better than that. http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ http://slatestarcodex.com/2014/07/30/meditations-on-moloch/
- manigandham 8y ago
  It's been a long time since I've read this, but I believe it is stating what I said in much more eloquent terms. I'm unsure what you mean by "do better than that"?
  - Sharlin 8y ago
    I mean that what you posed is a false dilemma: we must be able to solve our coordination problems or face extinction.
    - manigandham 8y ago

eddd 8y ago

I wouldn't blame Google for this. It's sole purpose is to make money. So they make money wherever they can. People voted for a government that bombs other people, that's a consequence.

bronzeage 8y ago

Why US citizens hate their army so much? All i know is that if google announced it's working with our army on some AI projects, the response would be overwhelmingly positive, and people would praise Google for that. (I'm from Israel if you're wondering) And I think US are a minority in that the citizens despise their own government and army this much. Which really makes you wonder what's the whole point of democracy if you trust your own elected officials less than Russians trust their corrupted monarch.

kome 8y ago
well, why do Israelis love their army so much? I think to know the answer, but be aware Israel is a very special case for so many reasons. Edit: and the contrary is true, Americans loves their army, and they are all about that "thank you for your service" rhetoric, so perhaps just Israel top that.
- myth_drannon 8y ago
  Most of Israelis serve in army that gives them an inside perspective of what is Army, how it functions and its goals. Americans see it from the mass media perspective only, which creates the same feelings in people like when non-Israelis see the news on Gaza fighting - very superficial and inducing very strong emotions. IMHO.
  - kome 8y ago
    Indeed, that's could be an explanation. Also Switzerland has conscription / compulsory draft, and among a lot of Swiss, the army is seen with sympathy. That said, I met a couple of Israelis that left the country for good because their experience with the army was so bad. I guess that there is some "voting with legs" going on.
  - aninhumer 8y ago
    >very superficial and inducing very strong emotions. IMHO. I think spending part of your young adult life participating in the actions and socialisation of an institution, and then living in a society where everyone around you shares that experience, creates far stronger emotional influence on people's judgements than seeing some people you don't know being shot in a far away country.

jxi 8y ago

Sundar's Google.

musicalentropy 8y ago

For people who still don't see any problem in this, I have to remind you that ultimately the guy on top of the military hierarchy in charge for selecting the drones with AI targets is... Donald Trump. If you're not scared now, I don't know what to do anymore. At some point, I thought that the main goal of these big startups with tons of claims and wishful thinking was to make the world a better place and help humanity to become a better version of itself. Instead now they create tools and weapons to "reduce collateral damages" in governemental induced wars (which have not solved anything over the last 20 years in Middle East but only increased cash flow in West pockets, made tons of death, both there and in West countries), and they have access to information related with most of the citizens at the same time. OK cool. In 10 years we will talk about whistleblowers killed by drones using AI as well.

azangru 8y ago
> I have to remind you that ultimately the guy on top of the military hierarchy in charge for selecting the drones with AI targets is... Donald Trump How can we be sure that this is so? - Can Trump make military decisions by himself (i.e. not choose among the ones that are offered to him by the military)? - Hasn't it been established recently that the military have the right to refuse his orders if they are particularly crazy (see e.g. https://www.independent.co.uk/news/world/americas/donald-trump-north-korea-crisis-nuclear-weapons-us-military-duty-refuse-illegal-instructions-war-a8055991.html https://www.independent.co.uk/news/world/americas/donald-tru...)?

deleted 8y ago

[deleted]

fdsak 8y ago

probably this is the right time to introduce an oath sth like this one https://github.com/Widdershin/programmers-oath https://github.com/Widdershin/programmers-oath

lucaspottersky 8y ago

it sucks that companies have this tendency to become nasty business

indescions_2018 8y ago

There must be some hard data. Looking back over the last 40+ years. Were research institutions that divested from defense funding during the Vietnam era adversely impacted. Perhaps short term. But on long scales there has been more than adequate consumer and private sector demand to compensate. Politically, the company could face a backlash in the form of antitrust scrutiny over search monopoly. Which Alphabet seems to be anticipating as historical inevitability. With the foresight even that perhaps breaking up large trusts is great for startup innovation ;) But it raises the general question. If given the option of taking $ to do basic science. Regardless of the funding entities motives. Do you do it anyway? An argument can be made to always take the money and do the science. You don't know what the world will look like in 10 years. Drone AI used for surgical strikes today will probably be civilian transferred to AgTech and planting seeds tomorrow. And there is always the possibility of a breakthrough. Among the many great examples are the Human Genome Project, the Manhattan Project and DARPA Grand Challenge. Terrestrial plutonium enrichment may turn out to be the key to deep space exploration, or space based defense. Early DOE-funded genetics research was specifically interested in answering the question: can we survive radiation induced mutations from the fallout of a nuclear war? And less than two decades after a Humvee from CMU completed 8 miles on its own in the California desert we will see fleets of semi-autonomous vehicles rolling out in major cities.

jdowner 8y ago

Do no evil* *conditions apply

deleted 8y ago

[deleted]

josefresco 8y ago

Dear young, super smart Googlers: If your knowledge of the Middle East and terrorism goes back to ISIS/Al Qaeda - stop coding (or selling ads) and visit your local library. The history section is chalk full of wonderfully informative books that will inform your opinions. You might find that Internet tropes about terrorism to be less than accurate. For example, I just finished "Rise and Kill First" [1] a rather enlightening book about Israel's use of targeted assassinations going back almost 100 years. When I started reading Israel wasn't in the news, but then the whole Iran/US nuclear deal fiasco hit, and it made Netanyahu's actions all the more clear. Note, I'm not saying justified but rather that I now understand why Israel thinks and acts the way it does, and how sometimes this clashes with their normally close allies like the US. 1: https://www.amazon.com/Rise-Kill-First-Targeted-Assassinations/dp/1400069718 https://www.amazon.com/Rise-Kill-First-Targeted-Assassinatio... Before that, I read "Ghost Wars" [2] another beast of a book that covers US involvement in the Middle East going back to the late 1970's up to 9/11. Fascinating book about the struggle to dominate Afghanistan by regional and world powers. [2] https://www.amazon.com/Ghost-Wars-Afghanistan-Invasion-September/dp/0143034669 https://www.amazon.com/Ghost-Wars-Afghanistan-Invasion-Septe... Before that, it was "The Brothers: John Foster Dulles, Allen Dulles, and Their Secret World War" [3] about the history of the CIA and the US's foreign policy in the last 100+ years. [3] https://www.amazon.com/Brothers-Foster-Dulles-Allen-Secret/dp/0805094970 https://www.amazon.com/Brothers-Foster-Dulles-Allen-Secret/d... Key point - These books look past recent events to provide an mostly complete understanding of these conflicts that you can't get with 20 minute Google searches and Wikipedia snippets.

AndrewKemendo 8y ago
Well beyond that, Military and Global Geopolitical research is a massive discipline that is studied most intensely by the Military itself. Just like most disciplines, it's not possible to get even a basic understanding of this issue with only a year or two of study.

new_age_garbage 8y ago

Why wouldn't you just spin off your military drone AI division. That seems like a no-brainer, even just for PR purposes. I'm not sure I'd want a Google rifle, based on the longevity of my first Pixel.

nevir 8y ago

The military is going to deploy deep learning on their drones, with or without Google's help. I'd rather have more people working on the problem than fewer. More scrutiny should, at least, improve the work and reduce errors in the models (and, hopefully, that leads to fewer casualties)

IanDrake 8y ago

Google developed a far left employee culture and then expected their employees to help the military. They don’t seem to understand how those two things are mostly incompatible.

beenBoutIT 8y ago

If Google refused to work with our military, Microsoft's subpar AI and ethics would power the drones and the project would move ahead.

blindwatchmaker 8y ago
Having to deal with UI developed by Microsoft is the only thing that could ever make me feel sorry for drone operators.

amaccuish 8y ago

People love to talk about Huawei's links to the People's Liberation Army, but when Google starts working with the US military, it's somehow different. Hmm...

kaonashi 8y ago
China doesn’t have the same (recent) history of ‘military adventurism’.

randcraw 8y ago

Apparently Google has decided NOT to pursue this line of work after all: https://gizmodo.com/google-plans-not-to-renew-its-contract-for-project-mave-1826488620 https://gizmodo.com/google-plans-not-to-renew-its-contract-f...

ggm 8y ago

Norbert Weiner

carapace 8y ago

"I Have No Mouth, and I Must Scream" by Harlan Ellison > The story takes place 109 years after the complete destruction of human civilization. The Cold War had escalated into a world war, fought mainly between China, Russia, and the United States. As the war progressed, the three warring nations each created a super-computer (with AI) capable of running the war more efficiently than humans. > The machines are each referred to as "AM", which originally stood for "Allied Mastercomputer", and then was later called "Adaptive Manipulator". Finally, "AM" stands for "Aggressive Menace". One day, one of the three computers becomes self-aware, and promptly absorbs the other two, thus taking control of the entire war. It carries out campaigns of mass genocide, killing off all but four men and one woman. > The survivors live together underground in an endless complex, the only habitable place left. The master computer harbors an immeasurable hatred for the group and spends every available moment torturing them. AM has not only managed to keep the humans from taking their own lives, but has made them virtually immortal. https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Scream https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Sc...