8 ms·

Google Renounces AI for Weapons, but Will Still Sell to Military

93 points by Jerry2 8y ago

0xfffff 8y ago
Was discussed already. Google is trying to do damage control and entertains that "do no evil" was not a bad idea but thats a thing of the past.
- jkaplowitz 8y ago
  Google's "Don't be evil" motto never vanished - it's still in their code of conduct. It got deemphasized from a wordy mention at the top, but its new right-at-the-end placement is still prominent for people who read the whole document (as employees should) or who skip to the end. Their motto was never "do no evil" - that's a far harder standard given that undesired side effects happen for most actions.
  - wu-ikkyu 8y ago
    >Their motto was never "do no evil" - that's a far harder standard given that undesired side effects happen for most actions. One's capacity for evil indeed grows greatly when one is given too much power. Likewise, as Google's power has grown inordinately, their public emphasis on "doing no evil" has diminished
    - jkaplowitz 8y ago
      Yup. Though I still don't think it needs a change in intention - i.e. more evil can still result from more power, more "machine learning with more data to assist and suggest to delight users", without any specific evil intent. They definitely aren't better than most companies in certain corporate matters related to the employee experience (especially for those who are in one or more marginalized demographics), or in their willingness to be a defense contractor at all (back on topic for this news item). They also make plenty of product and PR mistakes. Still, many tech companies are notably worse on many important issues: more willing to directly facilitate military weaponry and intelligence for lethal purposes, less top-notch at security, fewer and weaker privacy promises, bad at rigorously implementing and adhering to the privacy promises they do make, more in the habit of voluntarily sharing direct access to your personally identifiable data with third parties, less willing to tolerate employees complaining about privacy/security/harassment/workplace issues, etc. Overall I think Google needs to be watched closely, forced to do the right thing by some mixture of internal external pressure, possibly split into 2-4 smaller companies, and needs to fix certain culture problems .... but I still think it's among the better of the large trendy US tech companies, alongside Microsoft. And GCP is awesome. Disclosure: I used to work at Google, including the GCP team; my mention of Microsoft is based on second-hand information from people I know who do or did work there.
  - Scea91 8y ago
    > its new right-at-the-end placement is still prominent for people who read the whole document (as employees should) or who skip to the end. I really wonder what is the percentage of employees that read their company's code of conducts. I wouldn't be surprised if it was 5 %.
    - dhimes 8y ago
      Yeah. They need to make it a youtube video.
    - jkaplowitz 8y ago
      Overall? You may very well be right. At Google? Based on my memories of the culture there, definitely higher than that. Certainly not 100% though, agreed.
  - amelius 8y ago
    It's a stupid motto. Imagine that Disney used that motto, would you still send your children to DisneyLand? Or a restaurant saying "don't poison the guests", would you still eat there?
    - jkaplowitz 8y ago
      It's a motto meant as internal guidance, as befits something a code of conduct. Not something meant for marketing purposes. And yes, I'd like a restaurant to encourage its workers to take steps to ensure the food is safe.
      - amelius 8y ago
        > And yes, I'd like a restaurant to encourage its workers to take steps to ensure the food is safe. By telling them to not poison the guests? Do I really have to explain what's wrong here?
        jkaplowitz 8y ago
        "Don't poison the guests" would be analogous to "don't do [a specific evil thing]." "Don't be evil" is more analogus to "don't get sloppy with hygiene." The second message is a good one for a restaurant to give its workers.
        amelius 8y ago
        The problem with Google's motto is that it's about intentional harm, whereas your example (being sloppy) is about unintentional harm.
- bertil 8y ago
  “Don’t be evil” was, from the second it was suggested by (I believe Paul Buchheit) a acknowledgement of failure. The story goes that the, then very young, company was trying to define their values and every suggestion, no matter how broad, was being shot down by a counter-example. Even “be good” was not enough because too vague, or not neutral enough. A young engineer tried to summarise that failure by a double-negative, which left all the moral undefined. I still think that this problem isn’t resolved: is the problem the existence of violence, its legitimate monopoly by a democratically elected temporary dictator, or that violence is being misused by jingoistic non-sense, and that the US armed forced are controlled by corporate interests? Is evading that question the most moral stance? I feel like it’s mainly an easier one to defend, rather than the better one.
frozenport 8y ago
Any component senior executive would simply devolve their military oriented business to the employees that want to execute it. This failure is acutely felt in Boston Dynamics which was peopled by jingoistic engineers, who were limited in their ability to produce value to Google by the inability of executive to disassociate themselves from the body of that companies work. Alphabet should have had a second, and majority owned company, take on that drone contract.
vezycash 8y ago
>Google’s... won’t design or deploy AI for "weapons or other technologies whose PRINCIPAL purpose or implementation is to cause or DIRECTLY facilitate injury to people." This policy is so broad it's meaningless. PRIMARY purpose - promise us that its primary purpose is something else and we're game. Direct Injury - AI which determines / designates hostiles and isn't installed directly on the weapon can't cause direct harm.
- jopsen 8y ago
  If they choose to make a very narrow interpretation, they'll be publicly shamed as violating their own policy. It sounds like the want to keep selling cloud services, which is pretty much a commodity at this point. Even if weak, any policy is better than none. Anyways, given Googles consumer brand I suspect any form of involvement with weapons systems would be unwise :) I'm surprised they even considered it.
  - vezycash 8y ago
    >I'm surprised they even considered it. Reason and morality falls apart when confronted with shareholder value.
    - saalweachter 8y ago
      I'm honestly curious if it was ever about money or just that - if you, you know, ignore the whole "killing people" aspect - killbots are an incredibly difficult and complex and cool piece of technology to work on. You go from problems like "can we draw a box around the faces in this artificial dataset" or "can we keep a car between these white lines" to problems like "can we identity a person from a km away while traveling at 100mph" and "how do you take evasive maneuvers" and "can you do all this with hardware mounted on a drone with an acceptable latency". The whole Jurassic Park "you got so wrapped up in asking if you could you didn't ask if you should" thing.
    - dfxm12 8y ago
      Which affects shareholders more? The potential negative PR or the potential government contracts? If it were so cut and dry, I don't think they'd have a policy at all, and I think this policy is the output of a calculated risk assessment where Google determined that the effect of the PR would be greater than the cash. Plus, there was a chunk of their staff that threatened to leave the company if they went through with their pentagon drone project [0]. I imagine it would be hard to explain that to Wall st. 0 - https://www.bloomberg.com/news/articles/2018-06-01/google-won-t-renew-pentagon-ai-drone-deal-after-staff-backlash https://www.bloomberg.com/news/articles/2018-06-01/google-wo...
adventured 8y ago
It's an interesting problem, when the most talented companies in China and Russia will have no problem developing AI weapons for the state to benefit the motherland (and even if they do have a problem with it, they'll have little choice). My guess is the way the deep state and military industrial complex will deal with this, is to seed new Googles - In-Q-Tel, DARPA, etc - that are more cooperative in this area, and to punish Google in various ways going forward for non-alignment (overtly or otherwise). What use to the military complex is Google if they won't fully cooperate on something like this; time to get themselves a new Google.
riskneutral 8y ago
Kind of interesting to contrast this with the attitude of earlier generations of technological innovators. Archimedes developed catapults and canons to fight invading Romans. Leonardo Da Vinci designed war machines, including a giant crossbow. I don’t recall the details, but many European mathematicians in the 18th and 19th centuries rushed to help their countries develop weaponry, which they saw as a sort of patriotic duty. Alan Turing helped invent the computer as part of a military effort. John von Neumann, besides also helping to invent the computer in a military effort, was also an expert in nuclear warheads and work in the Manhattan Project. Richard Feynman worked on ballistics calculations during WWII and then after Pearl Harbir he worked on enriching Uranium for the Manhattan Project. In the 1950s, Gordon Moore (cofounder of Intel) and Robert Noyce created Fairchild Semiconductor, which practically created Silicon Valley and the transistor. The first transistors sold by Fairchild were used in the B70 bomber. Needless to say, the list goes on.
- bertil 8y ago
  I think that Google is developing weapons, but in the form of spam detectors, face recognition and Geo IP technology. They might not sell them (they don’t need to) but the open-source nature makes using those in an information war far easier. A lot of Facebook’s effort to track whether Russian agents purchased ads space, how they fostered dissent with controversial groups, etc. all that false under a new military “space” (neither ground, sea or air). Friends of mine who work in security hate the idea of calling it “cyber”, but “information” seems appropriate: cultivating journalists and influencers is key for instance. It’s a war against democratic institutions more than population, but it affect power quite directly. What Google refused to do was to attach military technology from a previous era (explosive, kinetic weapons) to their own work (face recognition) because they find that abhorrent. That’s actually not the first time: the use of wild animals, poisons, children and later radioactive material and nerve agents in war has been decried by military technologists since Archimedes — generally around the idea of a cleaner violence.
  - forgot-my-pw 8y ago
    By that description, any technology can be used as a weapon of crime or war.
assblaster 8y ago
I don't have a popular opinion on this. I wish they did more to help the US military with AI technology. It reduces costs of human interpretation and doesn't have the same issues with classified information in the minds of those humans. There could be great time savings and the potential for higher reliability of image interpretation if unsure AI assessments are passed onto humans for interpretation. I don't believe that the core of the US military is evil, and as an American I am happy to have it defending my country.
- 0x4f3759df 8y ago
  China is a leader in AI and they will develop AI weapons (does anyone doubt this?) Isn't the logical conclusion the country with AI weapons will become militarily superior to one without?
  - gustavmarwin 8y ago
    I speak for many people when I say the world doesn't want the U.S. to keep its military superiority. We're tired of their shit. China or U.S. having it, same difference. Although we are way way off that, a better, more organised, more democratic, more powerful U.N. should have that role.
    - 0x4f3759df 8y ago
      >China or U.S. having it, same difference. I think there's a big difference here.
      - gustavmarwin 8y ago
        Enlighten us.
        sbarker 8y ago
        https://en.wikipedia.org/wiki/World_War_II https://en.wikipedia.org/wiki/World_War_II
Khaine 8y ago
I'm sure all those people who at Google who forced this decision will feel great. When the tech industry in China helps China become a pre-immenient force using AI for military purposes, I'm sure they will regret their decision.
- mattnewton 8y ago
  Is there a competition between China and the US to drone the Middle East I don’t know about? I certainly won’t feel badly when the technology is instead developed by another company and used to bomb non-combatants in Pakistan. It’s not miliatary contracts I have a problem with, it’s this particular program, this particular method.
mankash666 8y ago
>>> AI applications for weapons and technologies that "gather or use information for surveillance," Couldn't be more ironic. Google's core business is surveillance and serving ads - all aided by ai. This whole thing is a joke - a bunch of know-it-all overrated and overpayed jerks pretending to know what's best for the world/country. Why and how is it assumed that they know better than lawmakers, government officials and security agencies? At the very least, this is a vote of no-confidence against the very institutions and people instilled to defend the country and further it's interests!! When Motorola was acquired by Google, plenty of googlers commented on how much more smarter the average Googler was. And how the new genius Google upper brass world turn Motorola into a money making machine. However, they failed miserably at building and selling good hardware, and that struggle continues to this day through their pixel, Nexus and HTC devices. So - a bunch of cocky software engineers couldn't even build and sell better phones, why'd they be right on national security?
wyld_one 8y ago
Maybe we should put AI in all the weapons. Then have them play Tic Tac Toe. Call it the Joshua Project. :)

93 points by Jerry2 8y ago

Was discussed already. Google is trying to do damage control and entertains that "do no evil" was not a bad idea but thats a thing of the past.

jkaplowitz 8y ago
Google's "Don't be evil" motto never vanished - it's still in their code of conduct. It got deemphasized from a wordy mention at the top, but its new right-at-the-end placement is still prominent for people who read the whole document (as employees should) or who skip to the end. Their motto was never "do no evil" - that's a far harder standard given that undesired side effects happen for most actions.
- wu-ikkyu 8y ago
  >Their motto was never "do no evil" - that's a far harder standard given that undesired side effects happen for most actions. One's capacity for evil indeed grows greatly when one is given too much power. Likewise, as Google's power has grown inordinately, their public emphasis on "doing no evil" has diminished
  - jkaplowitz 8y ago
    Yup. Though I still don't think it needs a change in intention - i.e. more evil can still result from more power, more "machine learning with more data to assist and suggest to delight users", without any specific evil intent. They definitely aren't better than most companies in certain corporate matters related to the employee experience (especially for those who are in one or more marginalized demographics), or in their willingness to be a defense contractor at all (back on topic for this news item). They also make plenty of product and PR mistakes. Still, many tech companies are notably worse on many important issues: more willing to directly facilitate military weaponry and intelligence for lethal purposes, less top-notch at security, fewer and weaker privacy promises, bad at rigorously implementing and adhering to the privacy promises they do make, more in the habit of voluntarily sharing direct access to your personally identifiable data with third parties, less willing to tolerate employees complaining about privacy/security/harassment/workplace issues, etc. Overall I think Google needs to be watched closely, forced to do the right thing by some mixture of internal external pressure, possibly split into 2-4 smaller companies, and needs to fix certain culture problems .... but I still think it's among the better of the large trendy US tech companies, alongside Microsoft. And GCP is awesome. Disclosure: I used to work at Google, including the GCP team; my mention of Microsoft is based on second-hand information from people I know who do or did work there.
- Scea91 8y ago
  > its new right-at-the-end placement is still prominent for people who read the whole document (as employees should) or who skip to the end. I really wonder what is the percentage of employees that read their company's code of conducts. I wouldn't be surprised if it was 5 %.
  - dhimes 8y ago
    Yeah. They need to make it a youtube video.
  - jkaplowitz 8y ago
    Overall? You may very well be right. At Google? Based on my memories of the culture there, definitely higher than that. Certainly not 100% though, agreed.
- amelius 8y ago
  It's a stupid motto. Imagine that Disney used that motto, would you still send your children to DisneyLand? Or a restaurant saying "don't poison the guests", would you still eat there?
  - jkaplowitz 8y ago
    It's a motto meant as internal guidance, as befits something a code of conduct. Not something meant for marketing purposes. And yes, I'd like a restaurant to encourage its workers to take steps to ensure the food is safe.
    - amelius 8y ago
      > And yes, I'd like a restaurant to encourage its workers to take steps to ensure the food is safe. By telling them to not poison the guests? Do I really have to explain what's wrong here?
      - jkaplowitz 8y ago
        "Don't poison the guests" would be analogous to "don't do [a specific evil thing]." "Don't be evil" is more analogus to "don't get sloppy with hygiene." The second message is a good one for a restaurant to give its workers.
        amelius 8y ago
        The problem with Google's motto is that it's about intentional harm, whereas your example (being sloppy) is about unintentional harm.
bertil 8y ago
“Don’t be evil” was, from the second it was suggested by (I believe Paul Buchheit) a acknowledgement of failure. The story goes that the, then very young, company was trying to define their values and every suggestion, no matter how broad, was being shot down by a counter-example. Even “be good” was not enough because too vague, or not neutral enough. A young engineer tried to summarise that failure by a double-negative, which left all the moral undefined. I still think that this problem isn’t resolved: is the problem the existence of violence, its legitimate monopoly by a democratically elected temporary dictator, or that violence is being misused by jingoistic non-sense, and that the US armed forced are controlled by corporate interests? Is evading that question the most moral stance? I feel like it’s mainly an easier one to defend, rather than the better one.

frozenport 8y ago

Any component senior executive would simply devolve their military oriented business to the employees that want to execute it. This failure is acutely felt in Boston Dynamics which was peopled by jingoistic engineers, who were limited in their ability to produce value to Google by the inability of executive to disassociate themselves from the body of that companies work. Alphabet should have had a second, and majority owned company, take on that drone contract.

vezycash 8y ago

>Google’s... won’t design or deploy AI for "weapons or other technologies whose PRINCIPAL purpose or implementation is to cause or DIRECTLY facilitate injury to people." This policy is so broad it's meaningless. PRIMARY purpose - promise us that its primary purpose is something else and we're game. Direct Injury - AI which determines / designates hostiles and isn't installed directly on the weapon can't cause direct harm.

jopsen 8y ago
If they choose to make a very narrow interpretation, they'll be publicly shamed as violating their own policy. It sounds like the want to keep selling cloud services, which is pretty much a commodity at this point. Even if weak, any policy is better than none. Anyways, given Googles consumer brand I suspect any form of involvement with weapons systems would be unwise :) I'm surprised they even considered it.
- vezycash 8y ago
  >I'm surprised they even considered it. Reason and morality falls apart when confronted with shareholder value.
  - saalweachter 8y ago
    I'm honestly curious if it was ever about money or just that - if you, you know, ignore the whole "killing people" aspect - killbots are an incredibly difficult and complex and cool piece of technology to work on. You go from problems like "can we draw a box around the faces in this artificial dataset" or "can we keep a car between these white lines" to problems like "can we identity a person from a km away while traveling at 100mph" and "how do you take evasive maneuvers" and "can you do all this with hardware mounted on a drone with an acceptable latency". The whole Jurassic Park "you got so wrapped up in asking if you could you didn't ask if you should" thing.
  - dfxm12 8y ago
    Which affects shareholders more? The potential negative PR or the potential government contracts? If it were so cut and dry, I don't think they'd have a policy at all, and I think this policy is the output of a calculated risk assessment where Google determined that the effect of the PR would be greater than the cash. Plus, there was a chunk of their staff that threatened to leave the company if they went through with their pentagon drone project [0]. I imagine it would be hard to explain that to Wall st. 0 - https://www.bloomberg.com/news/articles/2018-06-01/google-won-t-renew-pentagon-ai-drone-deal-after-staff-backlash https://www.bloomberg.com/news/articles/2018-06-01/google-wo...

adventured 8y ago

It's an interesting problem, when the most talented companies in China and Russia will have no problem developing AI weapons for the state to benefit the motherland (and even if they do have a problem with it, they'll have little choice). My guess is the way the deep state and military industrial complex will deal with this, is to seed new Googles - In-Q-Tel, DARPA, etc - that are more cooperative in this area, and to punish Google in various ways going forward for non-alignment (overtly or otherwise). What use to the military complex is Google if they won't fully cooperate on something like this; time to get themselves a new Google.

riskneutral 8y ago

Kind of interesting to contrast this with the attitude of earlier generations of technological innovators. Archimedes developed catapults and canons to fight invading Romans. Leonardo Da Vinci designed war machines, including a giant crossbow. I don’t recall the details, but many European mathematicians in the 18th and 19th centuries rushed to help their countries develop weaponry, which they saw as a sort of patriotic duty. Alan Turing helped invent the computer as part of a military effort. John von Neumann, besides also helping to invent the computer in a military effort, was also an expert in nuclear warheads and work in the Manhattan Project. Richard Feynman worked on ballistics calculations during WWII and then after Pearl Harbir he worked on enriching Uranium for the Manhattan Project. In the 1950s, Gordon Moore (cofounder of Intel) and Robert Noyce created Fairchild Semiconductor, which practically created Silicon Valley and the transistor. The first transistors sold by Fairchild were used in the B70 bomber. Needless to say, the list goes on.

bertil 8y ago
I think that Google is developing weapons, but in the form of spam detectors, face recognition and Geo IP technology. They might not sell them (they don’t need to) but the open-source nature makes using those in an information war far easier. A lot of Facebook’s effort to track whether Russian agents purchased ads space, how they fostered dissent with controversial groups, etc. all that false under a new military “space” (neither ground, sea or air). Friends of mine who work in security hate the idea of calling it “cyber”, but “information” seems appropriate: cultivating journalists and influencers is key for instance. It’s a war against democratic institutions more than population, but it affect power quite directly. What Google refused to do was to attach military technology from a previous era (explosive, kinetic weapons) to their own work (face recognition) because they find that abhorrent. That’s actually not the first time: the use of wild animals, poisons, children and later radioactive material and nerve agents in war has been decried by military technologists since Archimedes — generally around the idea of a cleaner violence.
- forgot-my-pw 8y ago
  By that description, any technology can be used as a weapon of crime or war.

assblaster 8y ago

I don't have a popular opinion on this. I wish they did more to help the US military with AI technology. It reduces costs of human interpretation and doesn't have the same issues with classified information in the minds of those humans. There could be great time savings and the potential for higher reliability of image interpretation if unsure AI assessments are passed onto humans for interpretation. I don't believe that the core of the US military is evil, and as an American I am happy to have it defending my country.

0x4f3759df 8y ago
China is a leader in AI and they will develop AI weapons (does anyone doubt this?) Isn't the logical conclusion the country with AI weapons will become militarily superior to one without?
- gustavmarwin 8y ago
  I speak for many people when I say the world doesn't want the U.S. to keep its military superiority. We're tired of their shit. China or U.S. having it, same difference. Although we are way way off that, a better, more organised, more democratic, more powerful U.N. should have that role.
  - 0x4f3759df 8y ago
    >China or U.S. having it, same difference. I think there's a big difference here.
    - gustavmarwin 8y ago
      Enlighten us.
      - sbarker 8y ago
        https://en.wikipedia.org/wiki/World_War_II https://en.wikipedia.org/wiki/World_War_II

Khaine 8y ago

I'm sure all those people who at Google who forced this decision will feel great. When the tech industry in China helps China become a pre-immenient force using AI for military purposes, I'm sure they will regret their decision.

mattnewton 8y ago
Is there a competition between China and the US to drone the Middle East I don’t know about? I certainly won’t feel badly when the technology is instead developed by another company and used to bomb non-combatants in Pakistan. It’s not miliatary contracts I have a problem with, it’s this particular program, this particular method.

mankash666 8y ago

>>> AI applications for weapons and technologies that "gather or use information for surveillance," Couldn't be more ironic. Google's core business is surveillance and serving ads - all aided by ai. This whole thing is a joke - a bunch of know-it-all overrated and overpayed jerks pretending to know what's best for the world/country. Why and how is it assumed that they know better than lawmakers, government officials and security agencies? At the very least, this is a vote of no-confidence against the very institutions and people instilled to defend the country and further it's interests!! When Motorola was acquired by Google, plenty of googlers commented on how much more smarter the average Googler was. And how the new genius Google upper brass world turn Motorola into a money making machine. However, they failed miserably at building and selling good hardware, and that struggle continues to this day through their pixel, Nexus and HTC devices. So - a bunch of cocky software engineers couldn't even build and sell better phones, why'd they be right on national security?

wyld_one 8y ago

Maybe we should put AI in all the weapons. Then have them play Tic Tac Toe. Call it the Joshua Project. :)