10 ms·

The Case Against an Autonomous Military

22 points by dnetesn 9y ago

ashleyn 9y ago
I would think James Cameron articulated quite a compelling case against one 35 years ago.
ataturk 9y ago
The Framers of the US Constitution were rightly concerned about standing armies. Not that anyone cares about any of it at this point--we're all going to go through hell again, probably nuclear hell, then people will understand. Seriously, how much death and destruction does the US need to mete out on an annual basis, regardless of drones or what have you? All this tech has done is empower the worst, most brutal people.
Eire_Banshee 9y ago
We can argue about the implications all we want... but given the choice between sending an American soldier overseas to die or a robot to get destroyed... the choice is pretty obvious to me.
- decebalus1 9y ago
  how about not sending anyone overseas? Why isn't this choice being presented? Let's draw up the stakeholders of 'sending entities abroad to kill' and find out why this choice is not considered in the first place instead of fighting over the philosophy of who or what kills to push forward a sick foreign policy. I see this every day in American politics. Couple of days ago people were fighting over what constitutes an 'assault riffle'. Mission accomplished: the actual original debate (banning guns) was already abandoned.
  - pc86 9y ago
    "Banning guns" violates the USC and simply is not going to happen without a Constitutional amendment, which politically will never happen to 2A.
    - Retric 9y ago
      Banning gun clips is viable, even cartridges seem to be optional as ball and powder is clearly a gun. Single shot weapons are good enough for home defense and vastly reduce the impact of gun violence.
      - pc86 9y ago
        Banning magazines based on what? Any magazine ban that effectively bans a class of weapon (e.g. nearly every type of handgun currently manufactured) is no different than banning handguns.
        Retric 9y ago
        You can't buy a belt-fed machine gun, so banning types of guns and ammunition are both constitutional. They are fairly simple to manufacture a single shot hand gun, and they are still plenty useful to hunt wildlife with just look at history. The clip is a weapon of war, just look at the impact repeating rifles had on the civil war for example. As to the reasoning, a single bullet is a threat, but far less useful for mass shootings. So banning them would save thousands of american lives over time and it's also constitutional.
        megaman22 9y ago
        Actually, you can buy a machine gun. It just requires a fuckload of money, time, and the willingness to go under the microscope by the ATF in order to obtain the necessary licenses.
        Retric 9y ago
        Not a new one. My next step is to change bullet diameters.
        vlehto 9y ago
        Did you know there are lots of hobbyists who cast their own bullets? Any diameter is doable to them.
  - Clubber 9y ago
    >the actual original debate (banning guns) was already abandoned. I don't think an outright ban was ever really up for debate. A minority wants guns banned outright. Most people just want stricter laws. Also, "assault rifles," constitute a minority of homicides with firearms. It's mostly hand guns. Either way, gun violence is way down from it's historical peak during the early 90s. No one really knows why.
  - GCU-Empiricist 9y ago
    I haven't kept the situational awareness bubble on the whys of today's wars, but generally it is or is supposed to be a case of 1. We need to beat up A to protect B or 2. we are eventually going to fight so lets do it in their yard and not mess up ours. These are generally morally good reasons to spend blood and treasure
    - Tyrek 9y ago
      But the counterpoint is that it's too easy to fall into a 'We MUST intervene to protect democracy/progress/freedom/whatever' if the only political cost is money (which is fairly easy to find for the military), as opposed to blood. Would the Vietnam withdrawal have happened if young Americans weren't dying? It's far too easy to grandstand politically at home while causing immense suffering to foreign populations if there's no blood on the line.
  - Eire_Banshee 9y ago
    Well, believe it or not, not everyone wants to ban guns. Some of us really enjoy them. And Im not advocating sending more people overseas. Im saying when we do, Id rather it be a robot than a person... why is this about gun control now?
- NicoJuicy 9y ago
  What if a country hasn't got robots, but has nukes. Then the choice is not that obvious anymore.
- darkerside 9y ago
  The path to Hell is paved with good intentions. You can make nothing but good decisions when considered individually, and yet, end up exactly where you don't want to be. It's no smarter than your typical greedy algorithm leading to a local maximum, or worse, once you get forced down a path you'd prefer not to be on.
- MarkMMullin 9y ago
  To me as well - first it starts with "I don't want to lose any of my soldiers but I must be able to kill theirs" and then more and more questions about machine autonomy keep popping up and finally we've got machines capable of making the decision on their own, and then we're pretty much doomed - not the snazzy terminator robots, just a lot of dumb plastic machines lower on the intelligence level than a flatworm, however that won't make us any less dead as a species. If you aren't willing to risk lives you absolutely shouldn't be killing people, and we're getting to the point where we really need to give it up as a strategy before we do ourselves in.
greedo 9y ago
Has there been any technology that hasn't been used in a military conflict?
- rdl 9y ago
  We walked back from chem/bio to a great degree, but only really because we had something bigger/better (nuclear). A bunch of relatively useless things (blinding lasers, stuff like that) have been eliminated/banned, but again mainly because they're not militarily particularly useful.
  - chopin 9y ago
    I believe that, unfortunately, only weapons with little military use are banned. Take, for instance, chemical weapons. These are pretty much useless in modern (highly mobile) warfare. Take another example, mines. There is a treaty on banning them but it has not been ratified by the worst actors [1]. [1] https://en.wikipedia.org/wiki/List_of_parties_to_the_Ottawa_Treaty https://en.wikipedia.org/wiki/List_of_parties_to_the_Ottawa_...
    - greedo 9y ago
      Mines are extremely useful in modern warfare. The ability to shape an opponents maneuverability is critical. Mines (especially stuff like DPICM) allow defenses to attrit attackers.
  - cavanasm 9y ago
    They were banned for being inhumane, not not militarily useful. The international community doesn't like the idea of military actors running around with weapons that permanently cripple but don't kill; landmines are probably a better example of a banned weapon than blinding lasers. They're highly effective military weapons, but frequently leave victims alive and in need of critical medical care, and then support for the rest of their lives; incredibly terrifying and demoralizing from a psychological perspective. Countries that care about their soldiers don't want their soldiers to worry about that. The blinding lasers are similar: how many volunteers could the US get if there was a high risk of "merely" being permanently blinded, and then discharged?
breakpointalpha 9y ago
The arguments presented are fairly weak. The two main suppositions are that "AI can be used to target racial groups." and "AI weapons could end up in the hands of ISIS, et. al." While both of these are true, and certainly cause for concern, the author greatly ignores the game theory aspect of AI powered weapons. Namely, if our enemies are building them, we must too. I'm not trying to fear monger, but US and China are already engaged in an AI arms race. It's illogical for either party at this point to stop development. https://www.nytimes.com/2017/02/03/technology/artificial-intelligence-china-united-states.html https://www.nytimes.com/2017/02/03/technology/artificial-int...
- pc86 9y ago
  So do you think the best solution is to return to a Cold War-era MAD environment?
  - dantillberg 9y ago
    Does MAD apply to AI-centered weapons? I thought the Cold War's MAD environment had more to do with the advantages that a nation would have if it were to preemptively attack, because the weapons act so fast over such long distances.
    - Tyrek 9y ago
      MAD is predicated on the fact that ICBMs take time to arrive at their target, and are effectively unstoppable. The idea is that if nuclear state A was to target nuclear state B, B would be able to launch their own arsenal at A before B was obliterated. Additional methods of delivery such as sub nukes, strategic bombers, Carriers with nukes, etc. further reinforce the concept as B would be able to retaliate even after its destruction. Now, the modern context has weakened MAD somewhat, because it's strictly an interpretation of nation-state level nuclear conflict. With the proliferation of (plausibly deniable) non-state actors in the 21st century, the concept breaks down as it makes it much harder to 'attribute' a target to blanket nuke in response.
  - kerkeslager 9y ago
astine 9y ago
The best way to avid killing people in warfare is to avoid going to war, I think. Though, to some degree conflict is always going to be inevitable so we've got to prepare for it. I don't really see a situation in which autonomous weapons would be much worse that humans. Humans can (and do) conduct genocides. If the concern is some rogue actor like North Korea getting new more powerful weapons, I don't see the benefit for the West to avoid their research and development as other countries could easily pursue autonomous weapons independently by applying civilian AI techniques to warfare.
- maxxxxx 9y ago
  Autonomous weapons are much worse. If in 2003 the US had had autonomous weapons they probably would still be occupying Iraq because it would just cost money and no (American) lives. Unnecessary wars will be much easier if they only cost money.
vinceguidry 9y ago
I don't see 'autonomous' weapons as being developed by the US military as ever being banned under international law. Let us examine weapons that have been banned. Chemical and biological weapons were banned under international law because they aren't all that much more effective than conventional weapons yet cause an undue amount of suffering in comparison. You get no benefits, only downsides. The other example, anti-personnel mines, is more instructive. International law bans them because historically, belligerents who have employed them for area denial purposes never stick around and de-mine the area after the conflict is over, leaving the civilian population to suffer. The United States refuses to sign a blanket ban on landmines, as her position is that her landmines are different, they self-destruct after a set period of time, therefore they don't threaten civilians long after the war is over. There are two takeaways here. First, weapons are not banned before they are tested in a conflict. Second, effectiveness trumps morals, if a weapon of war is effective, that can tip it's acceptability status even in the face of "collateral" damage. This is the case with nuclear weaponry, which imposes costs on civilians but is considered worth it for their sheer power to end a conflict in minutes. This is, after all, the entire premise upon which war is waged, for the good of the many over the few who are harmed. AI controlled weapons, unless they're left in a battlefield for civilians to contend with, something that appears fairly unlikely, at least at the moment, does not change this basic equation.
- skybrian 9y ago
  Calling this an "equation" seems overly fatalistic, as if it were inevitable. I'm pessimistic about this as well, but there certainly are attempts to prevent nuclear weapons from spreading, and efforts to reduce the arsenals of countries that have them. These attempts often don't work, but they do matter because sometimes they do.
  - vinceguidry 9y ago
    People that I know in the military wish strong states with strong weaponry to act as a deterrent to conflict. They trust in the professionalism of other states' armed forces and the diplomatic process, which evolved over centuries of war to prevent accidents and human failings. This was greatly tested during the Cold War and the policies and procedures held conflicts down to a relative minimum, the unfortunate proxy war in Vietnam aside. Once it became clear to planners that MAD wasn't a real scenario and that the most effective strategy for nuclear weaponry involves smaller, more numerous weapons, targeted at enemy military installations, focus shifted on ensuring that enough weapons are available to fully end a conflict. Disarmament agreements unfortunately ensure that there is strategic space for a nuclear exchange to not fully end a conflict. This raises the probabilities of major conflict rather than reduces them. Great in theory, not in practice.
3pt14159 9y ago
We already have autonomous militaries. We have for a long time. Nuclear Weapons don't have pilots, after all. I'm in favour autonomous militaries amongst non-pariah states. At least insomuch as I'm in favour of militaries in general: Temporary, necessary evils. What is different is the plummeting costs of killbots. We should do everything in our power to stop these from proliferating to lesser states or terrorist groups. Just as we do with nuclear weapons. But at the end of the day autonomous devices are going to kill people and terrorists are going to do some of the killing. We need to accept that fact and design around it, not plead that the reality be different.
- crankylinuxuser 9y ago
  What crazier is I could (but WONT) whip up a pretty effective geneva convention style device. A blinder. It's one of the most horrific styles of weaponry - you don't even have the decency to kill them. All you do is monitor motion via a webcam. Once you see motion, you turn on an array of high power IR lasers with lenses to make an arc of laser light. There's no blink response, and the damage is already done. It would cost what, $500 total for the platform, including batteries for lengthy operation off grid. To be honest, I'd rather build stuff that bolsters all of humanity, and not just 1 tribe other another. I wish we'd get out of that "us vs them" mindset. Maybe someday.
  - 3pt14159 9y ago
    Yeah, I've quietly / privately worried about this as well. One watt lasers are easy to get. Though I think you're looking at way more than $500 for anything reliable. Thankfully there are international agreements against using these weapons on the battlefield.
- mtgx 9y ago
  There are two main functions of let's call them "modern" autonomous weapons, if you're going to expand the definition of autonomous weapons that much: 1) the decision to kill is not made by a human 2) the cost plummets so much that in the leaders' minds it starts to make sense to send one after almost everyone they want gone for whatever reason (just like it's already happening with drone strikes. It used to be that air strikes were mainly used in active war/conflict. Now it's more of a pervasive "they are always there" situation).
bhouston 9y ago
Arguing against an autonomous military is like arguing against machine guns or any other major technological innovation that makes it easier to kill people. The only things we have generally outlawed is mass killing devices such as gases and nuclear weapons and I guess firebombing in retrospect (e.g. Dresden.)
- robotresearcher 9y ago
  Nuclear weapons are not outlawed. The US has thousands. https://en.wikipedia.org/wiki/Nuclear_weapons_of_the_United_States https://en.wikipedia.org/wiki/Nuclear_weapons_of_the_United_...
  - bhouston 9y ago
    They are at least controlled and not practically used at the moment. And most people see them as lines not to be crossed.
    - robotresearcher 9y ago
      This is the relevant voluntary treaty: https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferation_of_Nuclear_Weapons https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferatio... The treaty admits the US, Russia, UK, France and China to maintain nuclear weapons. Israel, India and Pakistan never agreed, and North Korea acceded. Some of those countries can deliver a nuke to anywhere on the planet in an hour. edit: added Israel to the list of non-signatories.
- paganel 9y ago
  > firebombing in retrospect (e.g. Dresden.) Cluster-bombing of mainly civilian targets (with some opposing forces using those civilians as a shield) is still actively used by both the US (in Raqqa) and Russia (in Aleppo). Cluster-bombing is probably not as bad as firebombing but it's nonetheless pretty damn scary. Here's a YT video of the Russians doing it in Aleppo (https://www.youtube.com/watch?v=wnd3IYH7x1o https://www.youtube.com/watch?v=wnd3IYH7x1o) and the effects of the US bombardment on Raqqa: https://www.youtube.com/watch?v=Fe9qS0QKFSU https://www.youtube.com/watch?v=Fe9qS0QKFSU
throwaway2016a 9y ago
Unfortunately I don't know if it the the government we need to worry about. We assume that the military will be the one to implement this... drone technology is very cheap now (relatively) and so are image recognition systems. A rouge / non-state organization could easily and relatively cheaply create automated turrets and automated drones. Heck, Youtube has tons of videos of Americans doing it in their backyard just for "fun". Would they be as good as a state military's $10MM drone? Probably not. But when you can build 10k of them from the price of one, who cares? Really we should be worried about both.
eaxitect 9y ago
Arguments are fairly weak. I think saying "It’s certainly not far-fetched to suppose that a group like ISIS, for example, could acquire A.I. capabilities." is almost equal to saying they could acquire F-22 fighters or B-2 bombers as well. Developing sophisticated AI-based integrated weapon system is not that easy.
dfsegoat 9y ago
Relevant side analysis of the Rule of Law / International legal precedents RE: autonomous weapons systems, for those who are interested: https://breakingdefense.com/2018/04/a-treaty-to-ban-autonomous-intelligence-weapons/ https://breakingdefense.com/2018/04/a-treaty-to-ban-autonomo...
tim333 9y ago
On the other hand a case for autonomy is you could potentially kill less people. Say the enemy have a bunch of tanks doing bad stuff, humans fighting them are probably going to blow them up killing the crews before they can get you. With AI you could maybe use smaller drone like things designed to disable the tanks without killing the crews. And if the crews kill the drones who cares - they are only drones, send some more. I quite like the idea of AI that could take out military hardware and perhaps injure a few fighters without killing anyone. Maybe like T2 https://www.youtube.com/watch?v=kEztkckjRtE https://www.youtube.com/watch?v=kEztkckjRtE
mtgx 9y ago
Relevant recent story. Now imagine if these were autonomous: https://www.nbcnews.com/news/military/russia-has-figured-out-how-jam-u-s-drones-syria-n863931 https://www.nbcnews.com/news/military/russia-has-figured-out...
pjc50 9y ago
There's nowhere near enough discussion of the point that ""AI"" or other weapons with autonomous fire control are a huge target for "cyberwarfare", including outside of a conventional war. While ISIS aren't going to develop their own AI they could concievably develop some zero-days in US systems. Turn missile defence systems against civilian airliners, for example.
vlehto 9y ago
>“autonomous weapons are ideal for tasks such as assassinations, destabilizing nations, subduing populations and selectively killing a particular ethnic group.” This is not the first time this has come up. Even remotely controlled weapons are incredibly dangerous. I think the solution is not to ban autonomous weapons, they will be then used by non-state actors or "non-state actors". Governments need to enable better protection of citizens privacy. Currently any time I get mail, there is my physical address combined with my name. My phone number, current location and name are also bundled and probably visible to all kinds of people for all kinds of purposes. It would be terribly easy to find my home address, tie small bomb to cheap drone and crash that to my bedroom window while I'm asleep. It's a perfect crime. All this could be done because I have said "I don't like nazies" or "I oppose Sharia in my country" in facebook post.

22 points by dnetesn 9y ago

I would think James Cameron articulated quite a compelling case against one 35 years ago.

The Framers of the US Constitution were rightly concerned about standing armies. Not that anyone cares about any of it at this point--we're all going to go through hell again, probably nuclear hell, then people will understand. Seriously, how much death and destruction does the US need to mete out on an annual basis, regardless of drones or what have you? All this tech has done is empower the worst, most brutal people.

Eire_Banshee 9y ago

We can argue about the implications all we want... but given the choice between sending an American soldier overseas to die or a robot to get destroyed... the choice is pretty obvious to me.

decebalus1 9y ago
how about not sending anyone overseas? Why isn't this choice being presented? Let's draw up the stakeholders of 'sending entities abroad to kill' and find out why this choice is not considered in the first place instead of fighting over the philosophy of who or what kills to push forward a sick foreign policy. I see this every day in American politics. Couple of days ago people were fighting over what constitutes an 'assault riffle'. Mission accomplished: the actual original debate (banning guns) was already abandoned.
- pc86 9y ago
  "Banning guns" violates the USC and simply is not going to happen without a Constitutional amendment, which politically will never happen to 2A.
  - Retric 9y ago
    Banning gun clips is viable, even cartridges seem to be optional as ball and powder is clearly a gun. Single shot weapons are good enough for home defense and vastly reduce the impact of gun violence.
    - pc86 9y ago
      Banning magazines based on what? Any magazine ban that effectively bans a class of weapon (e.g. nearly every type of handgun currently manufactured) is no different than banning handguns.
      - Retric 9y ago
        You can't buy a belt-fed machine gun, so banning types of guns and ammunition are both constitutional. They are fairly simple to manufacture a single shot hand gun, and they are still plenty useful to hunt wildlife with just look at history. The clip is a weapon of war, just look at the impact repeating rifles had on the civil war for example. As to the reasoning, a single bullet is a threat, but far less useful for mass shootings. So banning them would save thousands of american lives over time and it's also constitutional.
        megaman22 9y ago
        Actually, you can buy a machine gun. It just requires a fuckload of money, time, and the willingness to go under the microscope by the ATF in order to obtain the necessary licenses.
        Retric 9y ago
        Not a new one. My next step is to change bullet diameters.
        vlehto 9y ago
        Did you know there are lots of hobbyists who cast their own bullets? Any diameter is doable to them.
- Clubber 9y ago
  >the actual original debate (banning guns) was already abandoned. I don't think an outright ban was ever really up for debate. A minority wants guns banned outright. Most people just want stricter laws. Also, "assault rifles," constitute a minority of homicides with firearms. It's mostly hand guns. Either way, gun violence is way down from it's historical peak during the early 90s. No one really knows why.
- GCU-Empiricist 9y ago
  I haven't kept the situational awareness bubble on the whys of today's wars, but generally it is or is supposed to be a case of 1. We need to beat up A to protect B or 2. we are eventually going to fight so lets do it in their yard and not mess up ours. These are generally morally good reasons to spend blood and treasure
  - Tyrek 9y ago
    But the counterpoint is that it's too easy to fall into a 'We MUST intervene to protect democracy/progress/freedom/whatever' if the only political cost is money (which is fairly easy to find for the military), as opposed to blood. Would the Vietnam withdrawal have happened if young Americans weren't dying? It's far too easy to grandstand politically at home while causing immense suffering to foreign populations if there's no blood on the line.
- Eire_Banshee 9y ago
  Well, believe it or not, not everyone wants to ban guns. Some of us really enjoy them. And Im not advocating sending more people overseas. Im saying when we do, Id rather it be a robot than a person... why is this about gun control now?
NicoJuicy 9y ago
What if a country hasn't got robots, but has nukes. Then the choice is not that obvious anymore.
darkerside 9y ago
The path to Hell is paved with good intentions. You can make nothing but good decisions when considered individually, and yet, end up exactly where you don't want to be. It's no smarter than your typical greedy algorithm leading to a local maximum, or worse, once you get forced down a path you'd prefer not to be on.
MarkMMullin 9y ago
To me as well - first it starts with "I don't want to lose any of my soldiers but I must be able to kill theirs" and then more and more questions about machine autonomy keep popping up and finally we've got machines capable of making the decision on their own, and then we're pretty much doomed - not the snazzy terminator robots, just a lot of dumb plastic machines lower on the intelligence level than a flatworm, however that won't make us any less dead as a species. If you aren't willing to risk lives you absolutely shouldn't be killing people, and we're getting to the point where we really need to give it up as a strategy before we do ourselves in.

greedo 9y ago

Has there been any technology that hasn't been used in a military conflict?

rdl 9y ago
We walked back from chem/bio to a great degree, but only really because we had something bigger/better (nuclear). A bunch of relatively useless things (blinding lasers, stuff like that) have been eliminated/banned, but again mainly because they're not militarily particularly useful.
- chopin 9y ago
  I believe that, unfortunately, only weapons with little military use are banned. Take, for instance, chemical weapons. These are pretty much useless in modern (highly mobile) warfare. Take another example, mines. There is a treaty on banning them but it has not been ratified by the worst actors [1]. [1] https://en.wikipedia.org/wiki/List_of_parties_to_the_Ottawa_Treaty https://en.wikipedia.org/wiki/List_of_parties_to_the_Ottawa_...
  - greedo 9y ago
    Mines are extremely useful in modern warfare. The ability to shape an opponents maneuverability is critical. Mines (especially stuff like DPICM) allow defenses to attrit attackers.
- cavanasm 9y ago
  They were banned for being inhumane, not not militarily useful. The international community doesn't like the idea of military actors running around with weapons that permanently cripple but don't kill; landmines are probably a better example of a banned weapon than blinding lasers. They're highly effective military weapons, but frequently leave victims alive and in need of critical medical care, and then support for the rest of their lives; incredibly terrifying and demoralizing from a psychological perspective. Countries that care about their soldiers don't want their soldiers to worry about that. The blinding lasers are similar: how many volunteers could the US get if there was a high risk of "merely" being permanently blinded, and then discharged?

breakpointalpha 9y ago

The arguments presented are fairly weak. The two main suppositions are that "AI can be used to target racial groups." and "AI weapons could end up in the hands of ISIS, et. al." While both of these are true, and certainly cause for concern, the author greatly ignores the game theory aspect of AI powered weapons. Namely, if our enemies are building them, we must too. I'm not trying to fear monger, but US and China are already engaged in an AI arms race. It's illogical for either party at this point to stop development. https://www.nytimes.com/2017/02/03/technology/artificial-intelligence-china-united-states.html https://www.nytimes.com/2017/02/03/technology/artificial-int...

pc86 9y ago
So do you think the best solution is to return to a Cold War-era MAD environment?
- dantillberg 9y ago
  Does MAD apply to AI-centered weapons? I thought the Cold War's MAD environment had more to do with the advantages that a nation would have if it were to preemptively attack, because the weapons act so fast over such long distances.
  - Tyrek 9y ago
    MAD is predicated on the fact that ICBMs take time to arrive at their target, and are effectively unstoppable. The idea is that if nuclear state A was to target nuclear state B, B would be able to launch their own arsenal at A before B was obliterated. Additional methods of delivery such as sub nukes, strategic bombers, Carriers with nukes, etc. further reinforce the concept as B would be able to retaliate even after its destruction. Now, the modern context has weakened MAD somewhat, because it's strictly an interpretation of nation-state level nuclear conflict. With the proliferation of (plausibly deniable) non-state actors in the 21st century, the concept breaks down as it makes it much harder to 'attribute' a target to blanket nuke in response.
- kerkeslager 9y ago

astine 9y ago

The best way to avid killing people in warfare is to avoid going to war, I think. Though, to some degree conflict is always going to be inevitable so we've got to prepare for it. I don't really see a situation in which autonomous weapons would be much worse that humans. Humans can (and do) conduct genocides. If the concern is some rogue actor like North Korea getting new more powerful weapons, I don't see the benefit for the West to avoid their research and development as other countries could easily pursue autonomous weapons independently by applying civilian AI techniques to warfare.

maxxxxx 9y ago
Autonomous weapons are much worse. If in 2003 the US had had autonomous weapons they probably would still be occupying Iraq because it would just cost money and no (American) lives. Unnecessary wars will be much easier if they only cost money.

vinceguidry 9y ago

I don't see 'autonomous' weapons as being developed by the US military as ever being banned under international law. Let us examine weapons that have been banned. Chemical and biological weapons were banned under international law because they aren't all that much more effective than conventional weapons yet cause an undue amount of suffering in comparison. You get no benefits, only downsides. The other example, anti-personnel mines, is more instructive. International law bans them because historically, belligerents who have employed them for area denial purposes never stick around and de-mine the area after the conflict is over, leaving the civilian population to suffer. The United States refuses to sign a blanket ban on landmines, as her position is that her landmines are different, they self-destruct after a set period of time, therefore they don't threaten civilians long after the war is over. There are two takeaways here. First, weapons are not banned before they are tested in a conflict. Second, effectiveness trumps morals, if a weapon of war is effective, that can tip it's acceptability status even in the face of "collateral" damage. This is the case with nuclear weaponry, which imposes costs on civilians but is considered worth it for their sheer power to end a conflict in minutes. This is, after all, the entire premise upon which war is waged, for the good of the many over the few who are harmed. AI controlled weapons, unless they're left in a battlefield for civilians to contend with, something that appears fairly unlikely, at least at the moment, does not change this basic equation.

skybrian 9y ago
Calling this an "equation" seems overly fatalistic, as if it were inevitable. I'm pessimistic about this as well, but there certainly are attempts to prevent nuclear weapons from spreading, and efforts to reduce the arsenals of countries that have them. These attempts often don't work, but they do matter because sometimes they do.
- vinceguidry 9y ago
  People that I know in the military wish strong states with strong weaponry to act as a deterrent to conflict. They trust in the professionalism of other states' armed forces and the diplomatic process, which evolved over centuries of war to prevent accidents and human failings. This was greatly tested during the Cold War and the policies and procedures held conflicts down to a relative minimum, the unfortunate proxy war in Vietnam aside. Once it became clear to planners that MAD wasn't a real scenario and that the most effective strategy for nuclear weaponry involves smaller, more numerous weapons, targeted at enemy military installations, focus shifted on ensuring that enough weapons are available to fully end a conflict. Disarmament agreements unfortunately ensure that there is strategic space for a nuclear exchange to not fully end a conflict. This raises the probabilities of major conflict rather than reduces them. Great in theory, not in practice.

3pt14159 9y ago

We already have autonomous militaries. We have for a long time. Nuclear Weapons don't have pilots, after all. I'm in favour autonomous militaries amongst non-pariah states. At least insomuch as I'm in favour of militaries in general: Temporary, necessary evils. What is different is the plummeting costs of killbots. We should do everything in our power to stop these from proliferating to lesser states or terrorist groups. Just as we do with nuclear weapons. But at the end of the day autonomous devices are going to kill people and terrorists are going to do some of the killing. We need to accept that fact and design around it, not plead that the reality be different.

crankylinuxuser 9y ago
What crazier is I could (but WONT) whip up a pretty effective geneva convention style device. A blinder. It's one of the most horrific styles of weaponry - you don't even have the decency to kill them. All you do is monitor motion via a webcam. Once you see motion, you turn on an array of high power IR lasers with lenses to make an arc of laser light. There's no blink response, and the damage is already done. It would cost what, $500 total for the platform, including batteries for lengthy operation off grid. To be honest, I'd rather build stuff that bolsters all of humanity, and not just 1 tribe other another. I wish we'd get out of that "us vs them" mindset. Maybe someday.
- 3pt14159 9y ago
  Yeah, I've quietly / privately worried about this as well. One watt lasers are easy to get. Though I think you're looking at way more than $500 for anything reliable. Thankfully there are international agreements against using these weapons on the battlefield.
mtgx 9y ago
There are two main functions of let's call them "modern" autonomous weapons, if you're going to expand the definition of autonomous weapons that much: 1) the decision to kill is not made by a human 2) the cost plummets so much that in the leaders' minds it starts to make sense to send one after almost everyone they want gone for whatever reason (just like it's already happening with drone strikes. It used to be that air strikes were mainly used in active war/conflict. Now it's more of a pervasive "they are always there" situation).

bhouston 9y ago

Arguing against an autonomous military is like arguing against machine guns or any other major technological innovation that makes it easier to kill people. The only things we have generally outlawed is mass killing devices such as gases and nuclear weapons and I guess firebombing in retrospect (e.g. Dresden.)

robotresearcher 9y ago
Nuclear weapons are not outlawed. The US has thousands. https://en.wikipedia.org/wiki/Nuclear_weapons_of_the_United_States https://en.wikipedia.org/wiki/Nuclear_weapons_of_the_United_...
- bhouston 9y ago
  They are at least controlled and not practically used at the moment. And most people see them as lines not to be crossed.
  - robotresearcher 9y ago
    This is the relevant voluntary treaty: https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferation_of_Nuclear_Weapons https://en.wikipedia.org/wiki/Treaty_on_the_Non-Proliferatio... The treaty admits the US, Russia, UK, France and China to maintain nuclear weapons. Israel, India and Pakistan never agreed, and North Korea acceded. Some of those countries can deliver a nuke to anywhere on the planet in an hour. edit: added Israel to the list of non-signatories.
paganel 9y ago
> firebombing in retrospect (e.g. Dresden.) Cluster-bombing of mainly civilian targets (with some opposing forces using those civilians as a shield) is still actively used by both the US (in Raqqa) and Russia (in Aleppo). Cluster-bombing is probably not as bad as firebombing but it's nonetheless pretty damn scary. Here's a YT video of the Russians doing it in Aleppo (https://www.youtube.com/watch?v=wnd3IYH7x1o https://www.youtube.com/watch?v=wnd3IYH7x1o) and the effects of the US bombardment on Raqqa: https://www.youtube.com/watch?v=Fe9qS0QKFSU https://www.youtube.com/watch?v=Fe9qS0QKFSU

throwaway2016a 9y ago

Unfortunately I don't know if it the the government we need to worry about. We assume that the military will be the one to implement this... drone technology is very cheap now (relatively) and so are image recognition systems. A rouge / non-state organization could easily and relatively cheaply create automated turrets and automated drones. Heck, Youtube has tons of videos of Americans doing it in their backyard just for "fun". Would they be as good as a state military's $10MM drone? Probably not. But when you can build 10k of them from the price of one, who cares? Really we should be worried about both.

eaxitect 9y ago

Arguments are fairly weak. I think saying "It’s certainly not far-fetched to suppose that a group like ISIS, for example, could acquire A.I. capabilities." is almost equal to saying they could acquire F-22 fighters or B-2 bombers as well. Developing sophisticated AI-based integrated weapon system is not that easy.

dfsegoat 9y ago

Relevant side analysis of the Rule of Law / International legal precedents RE: autonomous weapons systems, for those who are interested: https://breakingdefense.com/2018/04/a-treaty-to-ban-autonomous-intelligence-weapons/ https://breakingdefense.com/2018/04/a-treaty-to-ban-autonomo...

tim333 9y ago

On the other hand a case for autonomy is you could potentially kill less people. Say the enemy have a bunch of tanks doing bad stuff, humans fighting them are probably going to blow them up killing the crews before they can get you. With AI you could maybe use smaller drone like things designed to disable the tanks without killing the crews. And if the crews kill the drones who cares - they are only drones, send some more. I quite like the idea of AI that could take out military hardware and perhaps injure a few fighters without killing anyone. Maybe like T2 https://www.youtube.com/watch?v=kEztkckjRtE https://www.youtube.com/watch?v=kEztkckjRtE

mtgx 9y ago

Relevant recent story. Now imagine if these were autonomous: https://www.nbcnews.com/news/military/russia-has-figured-out-how-jam-u-s-drones-syria-n863931 https://www.nbcnews.com/news/military/russia-has-figured-out...

pjc50 9y ago

There's nowhere near enough discussion of the point that ""AI"" or other weapons with autonomous fire control are a huge target for "cyberwarfare", including outside of a conventional war. While ISIS aren't going to develop their own AI they could concievably develop some zero-days in US systems. Turn missile defence systems against civilian airliners, for example.

vlehto 9y ago

>“autonomous weapons are ideal for tasks such as assassinations, destabilizing nations, subduing populations and selectively killing a particular ethnic group.” This is not the first time this has come up. Even remotely controlled weapons are incredibly dangerous. I think the solution is not to ban autonomous weapons, they will be then used by non-state actors or "non-state actors". Governments need to enable better protection of citizens privacy. Currently any time I get mail, there is my physical address combined with my name. My phone number, current location and name are also bundled and probably visible to all kinds of people for all kinds of purposes. It would be terribly easy to find my home address, tie small bomb to cheap drone and crash that to my bedroom window while I'm asleep. It's a perfect crime. All this could be done because I have said "I don't like nazies" or "I oppose Sharia in my country" in facebook post.