19 ms·

I don't want to anyone to build autonomous weapons, but I don't want anyone to build nuclear weapons or any other weapons of war either; I don't see how to avoi

I don't want to anyone to build autonomous weapons, but I don't want anyone to build nuclear weapons or any other weapons of war either; I don't see how to avoid it. If the choice is to either develop and deploy autonomous weapons or to risk having your population conquered and murdered by enemies that use them, then there is no choice. Possibly, autonomous weapons like chemical weapons won't be important to victory, or like most biological weapons (AFAIK) they won't be cost-effective. But it's hard to imagine a human defeating a bot in a shootout; consider human stock market traders who try to compete with flash trading computers, for example. In fact, I wonder if some of the tech is the same for optimizing decision speed and accuracy. Perhaps the best response by governments is to use their resources to develop autonomous weapons countermeasures, especially those [EDIT: i.e., those countermeasures] that can be acquired and utilized by those with few resources: Towns, governments in poor countries, and even individuals. Also, my guess is that it's an area ripe for effective international standareds, treaties and law. All governments can agree that they don't want the chaos of proliferating, unregulated autonomous weapons and would work to enforce the rules.

guelo 11y ago
It is easy to prevent it, have the UN ban it and provide incentives for countries to sign a treaty. This has worked for things like chemical and biological warfare. The key is to star the process now before generals get their hands on the technology so there won't be any pushback.
- jackreichert 11y ago
  This is a good idea, in theory. I just wonder how controllable it would be, once AI is more of a ubiquitous technology. With nuclear and bio warfare you can ban certain substances. Development of safe nuclear energy has suffered from this, perhaps justifiably. But once there are APIs, Open Source libraries, etc. out there, how will we contain it?
  - hoprocker 11y ago
    Right, but for it to be successful, there's still a substantial amount of R&D to get the systems cooperating in a manner effective on the battlefield. I agree, however, that if this were to go forward with a military bankroll, the result would be much easier to replicate. I'm particularly struck by this sentence from the letter: "If any major military power pushes ahead with AI weapon development...autonomous weapons will become the Kalashnikovs of tomorrow." That's terrifying.
- lordnacho 11y ago
  There's a question of whether one day you will be able to just "git pull 'terminator' " and install it on a cheap drone with an Arduino to pull the trigger on a mounted AK47. I reckon the software will get more and more ubiquitous. You can already download image recognition software, maps, and all the other code you need. How far are we actually from being able to send a drone to do what contract killers used to do?
- zo1 11y ago
  I'd argue that it's not that easy to "ban" something by getting the UN to say so, or even getting conventional treaties signed. First, the nations have to ratify it. And even if they do, they have to abide by it. Here are some random examples that I googled-for off the top of my head where these treaties, laws and the UN have failed. Torture: http://www.cfr.org/international-law/united-states-geneva-conventions/p11485#p7 http://www.cfr.org/international-law/united-states-geneva-co... Tear Gas: http://www.washingtonpost.com/news/morning-mix/wp/2014/08/14/tear-gas-is-a-chemical-weapon-banned-in-war-but-ferguson-police-shoot-it-at-protesters/ http://www.washingtonpost.com/news/morning-mix/wp/2014/08/14... Kyoto Protocol: https://en.wikipedia.org/wiki/Kyoto_Protocol https://en.wikipedia.org/wiki/Kyoto_Protocol Extrajudicial Killings in Drone Strikes: http://www.theguardian.com/world/2012/jun/21/drone-strikes-international-law-un http://www.theguardian.com/world/2012/jun/21/drone-strikes-i...
  - ectoplasm 11y ago
    Yeah, you'd need a fully functional global judicial system that included a police force to make UN regulations stick.
    - nahumfarchi 11y ago
      hmm... maybe a fully functional autonomous weapons police force?
      - ectoplasm 11y ago
        Is that you Skynet?
  - vonmoltke 11y ago
    > Tear Gas: Your linked article specifically stated tear gas was still legal for police use under the CWC. An expert rightly points out that this is illogical (and I agree), but it is not a failure. Also, I should point out that the mace and pepper spray canisters that many people around the world carry for personal protection are also illegal chemical weapons under the CWC if used in combat. There is a whole lot more illogic and inconsistency to be found between what is legal in something arbitrarily defined as "war" and otherwise, if one delves deep enough into the various treaties and conventions.
- run4yourlives2 11y ago
  This has worked for things like chemical and biological warfare I think the Kurds and some Syrians would beg to differ.
  - ethanbond 11y ago
    And they face(d) legal retaliation. Just like any other law. Sure, people can break it. That opens the door to justified retaliation. Should we legalize theft just because people still get pickpocketed?
    - run4yourlives2 11y ago
      No, that's not my argument. My argument is that you shouldn't leave your doors wide open simply because theft is illegal. It still happens, and so will actions with "illegal" weapons. I'm not suggesting we should ignore it, I'm suggesting that aside from making us feel nice, it's not an effective "solution".
      - ethanbond 11y ago
        And what would be the analog of leaving your doors wide open in this scenario? Dismantling a standing army? Not doing any cyberwarfare R&D? I don't think either of those are included in the proposal of "ban offensive AI weapons."
        run4yourlives2 11y ago
        You may as well do the first two if you aren't going to do the latter. Fighting an opponent using AI is like fighting a modern army with spears because you think guns are evil. This is such a technology leap it's not even funny. I appreciate the signatories' intent here and applaud them for higher level thinking but banning offensive AI first strike or counter strike ability is not going to happen because it puts people that ignore said ban far ahead of you in the ability to deal death department. The US isn't going to dismantle it's nuclear arms, and neither is any other major power. Same story here, only these weapons are even scarier because of the fact that they are much more flexible. It's nuclear arms strength without all that gooey radiation mess. AI could do everything from surgical strikes to full on massed combat without losing a countryman and dominating other nations on the battlefield. Nobody is going to get caught flat footed on that one.
click170 11y ago
I agree. But I wonder how much resistance you would get from the military, veterans, military families, and so on who make the argument that for every robot we make a human soldier doesn't have to be put at risk. I don't agree with that line of thinking but it would be quite a debate to have.
- kohanz 11y ago
  > But I wonder how much resistance you would get from the military, veterans, military families, and so on who make the argument that for every robot we make a human soldier doesn't have to be put at risk. Or it could go the other way with those people and families worried about losing their livelihoods.
- sbarre 11y ago
  You make the counter-argument that their enemies will make the same argument to their populations, which lowers the bar for armed conflict on every side, and increases the odds of war coming to your homefront.
  - sitkack 11y ago
    The only safe answer is for every country to invade the rest of the world immediately.
  - ethanbond 11y ago
    This is exactly why I'm afraid of this future. If the richest nation can send their robots to rape and pillage other countries with no threat to their own population, why wouldn't they? It took public outrage over lost lives for us (the US) to pull out of a war that we were already losing (Vietnam). I can't imagine what it'd take if we were winning and not dying.
    - sbarre 11y ago
      > If the richest nation can send their robots to rape and pillage other countries with no threat to their own population, why wouldn't they? You're basically describing the US's drone war in the Middle East today. And think of how little resistance there is to that.
toomuchtodo 11y ago
> Possibly, autonomous weapons like chemical weapons won't be important to victory, or like most biological weapons (AFAIK) they won't be cost-effective. But it's hard to imagine a human defeating a bot in a shootout; consider human stock market traders who try to compete with flash trading computers, for example. In fact, I wonder if some of the tech is the same for optimizing decision speed and accuracy. The only way for human adversaries to fight autonomous weapons would be with brute, lethal force (nuclear/neutron weapons). It ends poorly for all involved.
- Miner49er 11y ago
  Wouldn't it be feasible to build an autonomous weapon that doesn't target people, to fight the autonomous weapons that do target people? I would assume at that point whichever AI has better hardware and better algorithms would win, right?
  - toomuchtodo 11y ago
    Possibly? Anything past a few years out in tech is hard to predict. I wouldn't outright dismiss the idea, but its a crapshoot what machine intelligence is/isn't going to be able to do. I'm an educated, practical tech professional and machine intelligence worries me more than any other technology out there (except possibly a virus paired with CRISPR CAS 9 for targeted, precise genome modification driven through a species' population).
    - dharma1 11y ago
      biological weapons using CRISPR to target genetic markers is much more worrying to me because once released to the wild, there is zero control over it
      - run4yourlives2 11y ago
        Yee-ouch. That's a worrying thought. It's the holocaust on steroids. I'd rather be facing a nuclear armed foe than one that can eradicate my race.
knodi123 11y ago
> But it's hard to imagine a human defeating a bot in a shootout In a fair fight? Sure. But until these bots have strong human-strength AI, enemies will always be able to come up with dirty tricks.
- ethanbond 11y ago
  Maybe if it was Bot vs Human. In reality, it will be Bot + Human vs Human.
  - lifeformed 11y ago
    Yes, I think Bot-assisted humans would be far more effective than either one alone. Imagine the cunning of a human brain, enhanced with the senses and reflexes of robotics.
  - knodi123 11y ago
    Bot+Human? You're describing a drone, and you're right, it works great. Might work better if they could upgrade the optics a few notches, but I'm sure that's already in the works.
    - ethanbond 11y ago
      A drone is one possible implementation, but I was thinking more of a standard infantryman accompanied by two or three bots. Leave the strategic decisions to the commanders, the tactical to the infantryman, and the split-second firefighting to the robots.
usrusr 11y ago
Biological and chemical weapons are considerably easier to exclude by mutual agreement than AI, because the line is pretty clearly drawn. There is not much of an incremental path from dropping explosives do dropping gas containers. The line suggested here seems much more arbitrary and would be more like a wide grey area that would be pushed wider and wider into AI territory by including some form of alibi human interaction just as a formality. As described in the open letter, the "forbidden technology" would sit firmly sandwiched between the well established technology of "seek to kill" missiles that are fully autonomus once fired (as opposed to "seek, then autonomously decide to kill or not", which would be forbidden) and teleoperated equipment which is also explicitly allowed. The latter wont be stopped from getting better and better autonomous capabilities by mandating an operator to sign off kill decisions, which will eventually become a meaningless formality. If we want to avoid autonomous weapons, we need a more robust line than the one suggested.
deleted 11y ago
[deleted]
justizin 11y ago
> I don't want to anyone to build autonomous weapons, but I don't want anyone to build nuclear weapons or any other weapons of war either FWIW, all of the scientists involved in creating the first nuclear weapons immediately after the first detonation began pushing for a ban on further nuclear armament, and since then all wars have been fought with conventional weapons. I've been reading about the nuclear arms race and it is terrifying how often we came to destroying ourselves. I have possibly never seen greater evidence that there may be a god.
- akiselev 11y ago
  》I have possibly never seen greater evidence that there may be a god. Or that human beings are predominantly good people who, when given the big red button, refuse to become destroyers of worlds?
  - run4yourlives2 11y ago
    It's worth noting here that even Hitler and Stalin opted against deploying chemical weapons. Well, at least on the battlefield. Of course both of them had direct experience of being a victim to those weapons. The same cannot be said for nuclear weapons I'm afraid. People forget how bad things can be given enough time.
run4yourlives2 11y ago
The problem with rules is that someone always has it in their best interest to break them. Unlike dropping a nuclear bomb, you could break the rules here for years without even being caught. It's more like Germany in the 1930's than the cold war.
presidentender 11y ago
I've had a shootout of sorts against a robot. The robot was armed with an airsoft gun, and I with a Glock pistol. The goal was not to kill the robot (since it was expensive and the owner and I had spent a long time getting the machine vision software workin) but to avoid being hit by the robot while engaging some other targets. The course had to be carefully constructed to avoid an immediate robot victory, and the robot wasn't mobile. I wouldn't take the human side in a confrontation with an armed robot driven by a defense budget. The disadvantage of a robot is limited mobility and difficulty distinguishing friends from foes, the same disadvantages which plague landmines. The advantage is that a robotic force could provide the same area denial as landmines without the long-term consequences: set 20% of the robots to come home and recharge every day, with a week-long battery life, and you've got a very short period during which problems can happen.
- sangnoir 11y ago
  Both problems can be solved with today's technnology: make the robot airbourne to improve mobility, then tag friendly forces with some IFF broadcast. Declare a curfew and boom, everyone who's not a friendly is an enemy combatant. I hope I don't get to live to see such a day.
pjc50 11y ago
All governments can agree that they don't want the chaos of proliferating, unregulated autonomous weapons The US is not going to give up this capability. It's still not quite fully signed up to the landmine treaty. We know how this will go: automated colonial "antiterrorism" enforcement. Like drone strikes today, only lower cost. Entire populations kept in line by the robots that hunt in the night. Objecting to the death robots and organising against it will be considered evidence of terrorism and result in your death, along with anyone who phoned you recently enough. Deployed from Turkey to Tripoli.

guelo 11y ago

It is easy to prevent it, have the UN ban it and provide incentives for countries to sign a treaty. This has worked for things like chemical and biological warfare. The key is to star the process now before generals get their hands on the technology so there won't be any pushback.

jackreichert 11y ago
This is a good idea, in theory. I just wonder how controllable it would be, once AI is more of a ubiquitous technology. With nuclear and bio warfare you can ban certain substances. Development of safe nuclear energy has suffered from this, perhaps justifiably. But once there are APIs, Open Source libraries, etc. out there, how will we contain it?
- hoprocker 11y ago
  Right, but for it to be successful, there's still a substantial amount of R&D to get the systems cooperating in a manner effective on the battlefield. I agree, however, that if this were to go forward with a military bankroll, the result would be much easier to replicate. I'm particularly struck by this sentence from the letter: "If any major military power pushes ahead with AI weapon development...autonomous weapons will become the Kalashnikovs of tomorrow." That's terrifying.
lordnacho 11y ago
There's a question of whether one day you will be able to just "git pull 'terminator' " and install it on a cheap drone with an Arduino to pull the trigger on a mounted AK47. I reckon the software will get more and more ubiquitous. You can already download image recognition software, maps, and all the other code you need. How far are we actually from being able to send a drone to do what contract killers used to do?
zo1 11y ago
I'd argue that it's not that easy to "ban" something by getting the UN to say so, or even getting conventional treaties signed. First, the nations have to ratify it. And even if they do, they have to abide by it. Here are some random examples that I googled-for off the top of my head where these treaties, laws and the UN have failed. Torture: http://www.cfr.org/international-law/united-states-geneva-conventions/p11485#p7 http://www.cfr.org/international-law/united-states-geneva-co... Tear Gas: http://www.washingtonpost.com/news/morning-mix/wp/2014/08/14/tear-gas-is-a-chemical-weapon-banned-in-war-but-ferguson-police-shoot-it-at-protesters/ http://www.washingtonpost.com/news/morning-mix/wp/2014/08/14... Kyoto Protocol: https://en.wikipedia.org/wiki/Kyoto_Protocol https://en.wikipedia.org/wiki/Kyoto_Protocol Extrajudicial Killings in Drone Strikes: http://www.theguardian.com/world/2012/jun/21/drone-strikes-international-law-un http://www.theguardian.com/world/2012/jun/21/drone-strikes-i...
- ectoplasm 11y ago
  Yeah, you'd need a fully functional global judicial system that included a police force to make UN regulations stick.
  - nahumfarchi 11y ago
    hmm... maybe a fully functional autonomous weapons police force?
    - ectoplasm 11y ago
      Is that you Skynet?
- vonmoltke 11y ago
  > Tear Gas: Your linked article specifically stated tear gas was still legal for police use under the CWC. An expert rightly points out that this is illogical (and I agree), but it is not a failure. Also, I should point out that the mace and pepper spray canisters that many people around the world carry for personal protection are also illegal chemical weapons under the CWC if used in combat. There is a whole lot more illogic and inconsistency to be found between what is legal in something arbitrarily defined as "war" and otherwise, if one delves deep enough into the various treaties and conventions.
run4yourlives2 11y ago
This has worked for things like chemical and biological warfare I think the Kurds and some Syrians would beg to differ.
- ethanbond 11y ago
  And they face(d) legal retaliation. Just like any other law. Sure, people can break it. That opens the door to justified retaliation. Should we legalize theft just because people still get pickpocketed?
  - run4yourlives2 11y ago
    No, that's not my argument. My argument is that you shouldn't leave your doors wide open simply because theft is illegal. It still happens, and so will actions with "illegal" weapons. I'm not suggesting we should ignore it, I'm suggesting that aside from making us feel nice, it's not an effective "solution".
    - ethanbond 11y ago
      And what would be the analog of leaving your doors wide open in this scenario? Dismantling a standing army? Not doing any cyberwarfare R&D? I don't think either of those are included in the proposal of "ban offensive AI weapons."
      - run4yourlives2 11y ago
        You may as well do the first two if you aren't going to do the latter. Fighting an opponent using AI is like fighting a modern army with spears because you think guns are evil. This is such a technology leap it's not even funny. I appreciate the signatories' intent here and applaud them for higher level thinking but banning offensive AI first strike or counter strike ability is not going to happen because it puts people that ignore said ban far ahead of you in the ability to deal death department. The US isn't going to dismantle it's nuclear arms, and neither is any other major power. Same story here, only these weapons are even scarier because of the fact that they are much more flexible. It's nuclear arms strength without all that gooey radiation mess. AI could do everything from surgical strikes to full on massed combat without losing a countryman and dominating other nations on the battlefield. Nobody is going to get caught flat footed on that one.

click170 11y ago

I agree. But I wonder how much resistance you would get from the military, veterans, military families, and so on who make the argument that for every robot we make a human soldier doesn't have to be put at risk. I don't agree with that line of thinking but it would be quite a debate to have.

kohanz 11y ago
> But I wonder how much resistance you would get from the military, veterans, military families, and so on who make the argument that for every robot we make a human soldier doesn't have to be put at risk. Or it could go the other way with those people and families worried about losing their livelihoods.
sbarre 11y ago
You make the counter-argument that their enemies will make the same argument to their populations, which lowers the bar for armed conflict on every side, and increases the odds of war coming to your homefront.
- sitkack 11y ago
  The only safe answer is for every country to invade the rest of the world immediately.
- ethanbond 11y ago
  This is exactly why I'm afraid of this future. If the richest nation can send their robots to rape and pillage other countries with no threat to their own population, why wouldn't they? It took public outrage over lost lives for us (the US) to pull out of a war that we were already losing (Vietnam). I can't imagine what it'd take if we were winning and not dying.
  - sbarre 11y ago
    > If the richest nation can send their robots to rape and pillage other countries with no threat to their own population, why wouldn't they? You're basically describing the US's drone war in the Middle East today. And think of how little resistance there is to that.

toomuchtodo 11y ago

> Possibly, autonomous weapons like chemical weapons won't be important to victory, or like most biological weapons (AFAIK) they won't be cost-effective. But it's hard to imagine a human defeating a bot in a shootout; consider human stock market traders who try to compete with flash trading computers, for example. In fact, I wonder if some of the tech is the same for optimizing decision speed and accuracy. The only way for human adversaries to fight autonomous weapons would be with brute, lethal force (nuclear/neutron weapons). It ends poorly for all involved.

Miner49er 11y ago
Wouldn't it be feasible to build an autonomous weapon that doesn't target people, to fight the autonomous weapons that do target people? I would assume at that point whichever AI has better hardware and better algorithms would win, right?
- toomuchtodo 11y ago
  Possibly? Anything past a few years out in tech is hard to predict. I wouldn't outright dismiss the idea, but its a crapshoot what machine intelligence is/isn't going to be able to do. I'm an educated, practical tech professional and machine intelligence worries me more than any other technology out there (except possibly a virus paired with CRISPR CAS 9 for targeted, precise genome modification driven through a species' population).
  - dharma1 11y ago
    biological weapons using CRISPR to target genetic markers is much more worrying to me because once released to the wild, there is zero control over it
    - run4yourlives2 11y ago
      Yee-ouch. That's a worrying thought. It's the holocaust on steroids. I'd rather be facing a nuclear armed foe than one that can eradicate my race.

knodi123 11y ago

> But it's hard to imagine a human defeating a bot in a shootout In a fair fight? Sure. But until these bots have strong human-strength AI, enemies will always be able to come up with dirty tricks.

ethanbond 11y ago
Maybe if it was Bot vs Human. In reality, it will be Bot + Human vs Human.
- lifeformed 11y ago
  Yes, I think Bot-assisted humans would be far more effective than either one alone. Imagine the cunning of a human brain, enhanced with the senses and reflexes of robotics.
- knodi123 11y ago
  Bot+Human? You're describing a drone, and you're right, it works great. Might work better if they could upgrade the optics a few notches, but I'm sure that's already in the works.
  - ethanbond 11y ago
    A drone is one possible implementation, but I was thinking more of a standard infantryman accompanied by two or three bots. Leave the strategic decisions to the commanders, the tactical to the infantryman, and the split-second firefighting to the robots.

usrusr 11y ago

Biological and chemical weapons are considerably easier to exclude by mutual agreement than AI, because the line is pretty clearly drawn. There is not much of an incremental path from dropping explosives do dropping gas containers. The line suggested here seems much more arbitrary and would be more like a wide grey area that would be pushed wider and wider into AI territory by including some form of alibi human interaction just as a formality. As described in the open letter, the "forbidden technology" would sit firmly sandwiched between the well established technology of "seek to kill" missiles that are fully autonomus once fired (as opposed to "seek, then autonomously decide to kill or not", which would be forbidden) and teleoperated equipment which is also explicitly allowed. The latter wont be stopped from getting better and better autonomous capabilities by mandating an operator to sign off kill decisions, which will eventually become a meaningless formality. If we want to avoid autonomous weapons, we need a more robust line than the one suggested.

deleted 11y ago

[deleted]

justizin 11y ago

> I don't want to anyone to build autonomous weapons, but I don't want anyone to build nuclear weapons or any other weapons of war either FWIW, all of the scientists involved in creating the first nuclear weapons immediately after the first detonation began pushing for a ban on further nuclear armament, and since then all wars have been fought with conventional weapons. I've been reading about the nuclear arms race and it is terrifying how often we came to destroying ourselves. I have possibly never seen greater evidence that there may be a god.

akiselev 11y ago
》I have possibly never seen greater evidence that there may be a god. Or that human beings are predominantly good people who, when given the big red button, refuse to become destroyers of worlds?
- run4yourlives2 11y ago
  It's worth noting here that even Hitler and Stalin opted against deploying chemical weapons. Well, at least on the battlefield. Of course both of them had direct experience of being a victim to those weapons. The same cannot be said for nuclear weapons I'm afraid. People forget how bad things can be given enough time.

run4yourlives2 11y ago

The problem with rules is that someone always has it in their best interest to break them. Unlike dropping a nuclear bomb, you could break the rules here for years without even being caught. It's more like Germany in the 1930's than the cold war.

presidentender 11y ago

I've had a shootout of sorts against a robot. The robot was armed with an airsoft gun, and I with a Glock pistol. The goal was not to kill the robot (since it was expensive and the owner and I had spent a long time getting the machine vision software workin) but to avoid being hit by the robot while engaging some other targets. The course had to be carefully constructed to avoid an immediate robot victory, and the robot wasn't mobile. I wouldn't take the human side in a confrontation with an armed robot driven by a defense budget. The disadvantage of a robot is limited mobility and difficulty distinguishing friends from foes, the same disadvantages which plague landmines. The advantage is that a robotic force could provide the same area denial as landmines without the long-term consequences: set 20% of the robots to come home and recharge every day, with a week-long battery life, and you've got a very short period during which problems can happen.

sangnoir 11y ago
Both problems can be solved with today's technnology: make the robot airbourne to improve mobility, then tag friendly forces with some IFF broadcast. Declare a curfew and boom, everyone who's not a friendly is an enemy combatant. I hope I don't get to live to see such a day.

pjc50 11y ago

All governments can agree that they don't want the chaos of proliferating, unregulated autonomous weapons The US is not going to give up this capability. It's still not quite fully signed up to the landmine treaty. We know how this will go: automated colonial "antiterrorism" enforcement. Like drone strikes today, only lower cost. Entire populations kept in line by the robots that hunt in the night. Objecting to the death robots and organising against it will be considered evidence of terrorism and result in your death, along with anyone who phoned you recently enough. Deployed from Turkey to Tripoli.