15 ms·

Lethal Autonomous Weapons Exist; They Must Be Banned

152 points by doppioandante 5y ago

thesuperbigfrog 5y ago
The problem with arms races is that they are races: no one wants to be left behind. What is likely to happen is any major nation-state that signs a treaty agreeing to a ban will STILL secretly be researching and building autonomous (or nearly autonomous) weapon systems because they do not want to be left behind.
- omginternets 5y ago
  I would add that it's also very difficult to study and develop countermeasures without cultivating cutting-edge knowledge of how such weapons are designed, built and operated. Doing that means you eventually need field data. The slope is extremely slippery, but it's also dangerous not to traverse it.
  - falcolas 5y ago
    Agreed. We've already trod this ground with chemical weapons, so at least we have a template when we follow it once more with autonomous weapons.
- marcinzm 5y ago
  They also need to research them to learn how to counter them if an enemy does use them. Given that most of the conflicts of the last few decades were not with states but with groups the chance of someone using them is high. Especially as technology becomes more accessible.
- marvin 5y ago
  It would be a disaster to enter a military conflict with an adversary that has autonomous weapons if you don’t have at least effective countermeasures yourself. I suppose nuclear states could in principle rest on their laurels, but at least from a layman’s perspective it doesn’t give an impression of being strategically sound. But obviously it’s a scary rabbit hole to enter. It’s not fun when game-theoretical considerations or human nature seems to make dubious technological developments inevitable.
- xvilka 5y ago
  Winner will be the one who will design biological equivalents - genetically modified eagles or something similar.
  - treeman79 5y ago
    We just had a gain of function escape ordeal that we are still coping with.
5560675260 5y ago
I, for one, welcome our murderbot overlords. If we can't stop using drones to kill people, we should at least try to reduce collateral damage. With some advancements autonomous drones could be more precise and reliable than human operated ones, meaning that as an outcome fewer weddings will get blown up.
- StavrosK 5y ago
  And if fewer weddings get blown up, we'll use them more and more to kill people. I wonder whether the absolute number of killings will go up or down this way.
  - Matticus_Rex 5y ago
    Wait, why will we use them more and more to kill people if fewer weddings get blown up? I'm not seeing the causal link.
    - StavrosK 5y ago
      Because blowing up weddings leads to dissatisfied voters, so politicians are less likely to do that very often.
    - ahepp 5y ago
      I think they are suggesting current usage is limited by political fallout due to the misidentification rate. They hypothesize reducing the misidentification rate will increase the usage rate, ultimately killing more people (but fewer wedding parties)
    - Jtsummers 5y ago
      If they're accurate (or more accurate) then there's less resistance to using them more. They will be used to target an increasingly large number of people (or individuals) with a kind of specificity that present weapon systems cannot be used for and which impedes their use in some cases (because of the risk of collateral damage or the political fallout from a recent incident of bombing a wedding party).
marcus_holmes 5y ago
This feels like the same thing that happened with tanks. The end of WW1 saw tanks deployed effectively. WW2 was then fought using entirely new tactics and strategies enabled by tanks (not least "blitzkrieg"). Britain and France, notably, hadn't really worked out how to use them effectively and so were left fighting the last war, horribly disadvantaged until they worked out how to catch up (though air power being developed at the same time also had an influence). So now we have drones, and all those incredibly expensive fighter planes are useless. Aircraft carriers will be replaced with cheap drone swarm carriers, so naval strategy will need to change. Anti-tank drone swarms will make conventional armour more or less ineffective. And so on. The good news is that there will be anti-drone swarms too. And so we'll end up fighting battles mostly by destroying drones instead of people. Once you've knocked out all the enemy drones, they'll surrender because humans can't fight drones. Interesting times.
- nickthemagicman 5y ago
  > And so we'll end up fighting battles mostly by destroying drones instead of people. This is one piece of possibly good news that it seems like everyone is overlooking. Automation/Remote control applies not just to drones, but tanks, boats, other aerial vehicles, and maybe troops(after seeing that Boston Dynamics dog robot) etc. Seems like it will be a small squad of highly advanced human troops to move in after the Remotes to secure and maintain control of the area. So many lives will be spared during wars. However, this may be unbeatable tech by humans, so any future revolutions will be easily quelled by a gov't with a few drone pilots piloting a drone swarm in a room a thousand miles away.
  - marcus_holmes 5y ago
    I agree. This isn't all bad :) Like the move from conscript peasant armies to volunteer professional armies, leave the rest of us out of it. It'll come down to "how many drones, and of what quality, can your manufacturing base produce?". But this is not that dissimilar from WW2's "how many tanks, and of what quality, can you manufacture?" equation. Insurgencies will be about getting a small swarm in exactly the right place at the right time.
  - 5y ago
dctoedt 5y ago
The "Slaughterbot" video: https://www.youtube.com/watch?v=9CO6M2HsoIA https://www.youtube.com/watch?v=9CO6M2HsoIA
crumbshot 5y ago
Not only should there should be no autonomous drones, the non-autonomous remote controller hardware should be wired up such that if they are shot out of the sky, this kills the human operator and anyone sitting nearby. This would be much fairer than the current system of gamified point-and-click murder.
- barbazoo 5y ago
  As horrible as this sounds, I like the "real world consequence" part of it. Maybe this would lead to people reconsidering their participation in those war games.
  - nick238 5y ago
    If you die in the game, you die for real!
- UncleEntity 5y ago
  Who, realistically, would enforce such a system? I'm pretty sure any artificial rules which make war "fairer" will just be ignored when real bullets start flying since "it's better to be judged by twelve than to be carried by six" is an actual unofficial rule in the military.
jcims 5y ago
Product page for the UAV in question: https://www.stm.com.tr/en/kargu-autonomous-tactical-multi-rotor-attack-uav https://www.stm.com.tr/en/kargu-autonomous-tactical-multi-ro...
- barbazoo 5y ago
  Does anyone know what "Indigenous National Embedded Hardware and Software" means? Domestically developed maybe?
  - jcims 5y ago
    You could ask them on twitter lol - https://twitter.com/STMDefence https://twitter.com/STMDefence Surreal.
gpm 5y ago
Autonomous weapons should be embraced, otherwise less scrupulous people will embrace them, and they will win. Non-autonomous weapons will simply not be able to compete long term. Actually we wouldn't even be able to compete today, things like missiles are already autonomous, and if they weren't we would simply be unable to compete in any battlefield where radio was jammed, and it would give a large force multiplier to adversaries even in battle fields where radio is not jammed.
CountDrewku 5y ago
They seem to be only presenting the negative aspects of "slaughterbots". Could they not potentially be better than human controlled devices since they should be able to more accurately target things and create less collateral damage? Of course, there's always the doomsday terminator scenario where they go haywire and decide to start targeting all humans....
- SonicScrub 5y ago
  It's worth noting that this line of reasoning has been used to justify the invention of machines that ended up causing many more deaths than before. Richard Gatling was inspired to invent the Gatling gun as a way to reduce the number of deaths by combat. His logic was that if could replace the lines of musket-wielding infantry with a small team that could fire just as many rounds, the number of soldiers an army would require would drop, and therefore the number of combat deaths would drop as well. This line of reasoning was proven wrong, as the deployment of Gatling guns simply caused more causalities in the American Civil War, and the gun's descendants enabled the mass slaughter in the First World War. Similar logic was applied to bombers in the Second World War. If bombers could fly into enemy territory, hit key targets behind enemy lines and fly out, then the need for armies would go away. Instead we got The Blitz, the firebombing of Tokyo and Dresden, and the deadliest war in history. Maybe this time is somehow different, but it's an important historical context to consider. Often the expansion of our destructive capabilities simply encourages more destruction.
  - CountDrewku 5y ago
    Yes I'm just playing devil's advocate. My initial response is that they should be banned but like most things someone will continue creating them. Do we really think China cares whether there's a ban? Typically the only thing that stops WMD type weapons (not sure if slaughterbots fall into this category) is the enemy having them as well. Assured mutual destruction seems to be the only viable option.
ccsnags 5y ago
In the book “The Diamond Age” by Neal Stephenson there are mist-like swarms of nanobots that act as an immune system for regions against malicious nanobots. I can picture there being something like this with drones. There are problems to this. The major one being that, as mentioned by the posts on this thread, you have to develop the weapon to really know how to counter it.
- _jal 5y ago
  That's the Extropian "Utility Fog" concept. A more direct science fiction forerunner of this is Bruce Stirling's _Islands in the Net_ (1988). There is a scene where cheap explosive drone swarms are used in terror attacks. And of course, there's the famous first line of _Count Zero_. "They sent a Slamhound on Turner's trail in New Delhi, slotted it to his pheromones and the color of his hair. It caught up with him on a street called Chandni Chauk and came scrambling for his rented BMW - Its core was a kilogram of recrystallized hexogene and flaked TNT."
  - deleted 5y ago
    [deleted]
ru552 5y ago
We have hand held weapons to combat drones today. See https://www.droneshield.com/dronegun-tactical https://www.droneshield.com/dronegun-tactical https://www.reddit.com/r/pics/comments/o0kzuq/the_security_on_the_biden_king_phillippe_meeting/ https://www.reddit.com/r/pics/comments/o0kzuq/the_security_o...
- mdorazio 5y ago
  These countermeasures generally only work against drones that are not fully autonomous since they rely on disrupting communication with a ground controller. If your drone is perfectly happy to loiter and kill people based on facial recognition and some basic parameters, dronegun won't work. The other point to keep in mind is that cheap lethal drones are often designed to operate in swarm mode rather than individually. It's pretty difficult to knock out an autonomous swarm with weapons designed to target one drone at a time.
superbaconman 5y ago
Is this the official position of ieee? It's seems both unrealistic and unconstitutional (at least within the US).
- Jtsummers 5y ago
  It's a piece in IEEE Spectrum, lots of people submit articles there. With respect to unconstitutionality, really? Are you also able to acquire and keep on your property a functioning ICBM or nuclear bomb? Arms controls exist in the US despite the 2nd amendment.
  - superbaconman 5y ago
    Considering that the technology to build such drones is so widely available there's no reason why small arm drones wouldn't be protected under the 2nd amendment. Drones with bombs are a different story, but will be developed regardless of any international agreement (just like bio weapons).
michaelt 5y ago
Are these drone-based 'lethal autonomous weapons' markedly different to cruise missiles? Or is the alarm here about the fact that every man and his dog will be able to get their hands on them, when they cost $1000 instead of $1000000?
- neurobashing 5y ago
  I guess the answer to me is "it depends". Modern cruise missiles are pretty much drones, but at the same time they're always what I guess I'd call "weapons of state". Big, expensive, and part of a large decision-making loop. You can't just lob one; someone signed for it. Attacks are planned in advance at high levels. whereas this "slaughterbot" idea is smaller, low-cost, and with far less top-down control. At least, as I understand it.
- nick238 5y ago
  I think the latter. For what it's worth, the 'arms' that are still protected by the 2nd Amendment are still approximately the same as in 1787 (line of sight, bang bang), as you're legally restricted from owning guided bombs, cruise missiles, nuclear weapons, etc, but even aside from that, you're practically restricted because of limited availability and/or extreme cost. On the flip side, the hardware for Slaughterbots is basically here: TinyWhoops cost ~$100, say ~$100 to upgrade the ARM processor to one that can process realtime video, ~$100 for a shaped charge. So the hardware is super cheap, and once developed, the software to tie it all together has nearly no incremental cost. Once it's out there, that's the genie leaving the bottle.
- zokier 5y ago
  That is my question too. Tomahawk missiles have been fielded since early 80s, and are very much fire and forget autonomous weapons that can track targets based on sensor data (i.e. not reliant on GPS or other external guidance). Maybe the small difference is that cruise missiles generally are not anti-personnel weapons; maybe the article is seeking a ban in the same sense that Ottawa treaty banned anti-personnel mines but allows anti-tank mines. But I think this is something that would need to be explicitly addressed instead of just spouting sky is falling rhetoric.
jadzdc 5y ago
STM, the Turkish company that makes the KARGU drone referenced in the article has a promotional video from 2018 showing the unit being demonstrated: https://www.youtube.com/watch?v=Oqv9yaPLhEk https://www.youtube.com/watch?v=Oqv9yaPLhEk
Invictus0 5y ago
Weapons that have been successfully banned in the past typically had an alternative technology that it could be substituted with. I'm not sure you can say the same of lethal drones. In the case of Azerbaijan vs Armenia, the sum total of the fighter aircraft of BOTH nations combined is just 5. For poorer nations, drones represent a democratization of the battlefield; an extraordinarily cheap technology with a lot of firepower per dollar. I don't think these countries will be willing to give this up easily.
nabla9 5y ago
I don't fully understand how there is real difference? If you give a swarm of drones a kill zone and time limit, then they try them to kill every human in the area, how is that so different from MLRS salvo? Cruise missiles are already slaughterbots. You just give them location and they kill everyone in that location.
- dgan 5y ago
  Location is wrong argument. The article specifically points out that those drones were said to follow retreating fighters, and keep shooting un-discriminately
  - stickfigure 5y ago
    Retreating fighters are still combatants. Surrendering fighters would be a different matter.
CoolGuySteve 5y ago
One of the reasons the attempted drone assassination of Maduro in 2018 failed was because police were using radio jammers: https://en.wikipedia.org/wiki/2018_Caracas_drone_attack https://en.wikipedia.org/wiki/2018_Caracas_drone_attack It's fairly easy and cheap to make an explosive drone and train a neural network to drive it using off the shelf components and open source software. It follows that it's inevitable that at least semi-autonomous drones will be made to counter radio jamming. So while the best outcome a treaty could hope for is to prevent mass manufacturing these things by some countries, any motivated small country/individual will be able to craft them.
- gonzo41 5y ago
  Drones can't beat nylon nets or other low tech countermeasures. I think anti drone water cannons will outpace drone assassinations. And more than both, regular bombs and guns will still do most of the killing.
  - gpm 5y ago
    A motorized sniper rifle with face detection to find targets and a high rate of fire seems like it could exist with current technology, require about the same effort to deploy as a sniper with a typical sniper rifle, require less training from the person in the field, and be a hell of a lot more deadly. Same with a machine gun for closer quarters. Regular bombs and guns probably still do most of the killing for quite awhile because of inertia, but I'm pretty firmly of the belief that they are outdated technology or will be soon, especially the regular guns part. Regular bombs are arguably already outdated by missiles with (frequently autonomous) guidance... but for a lot of applications dumb explosives do work pretty well so those probably stick around for a long time.
    - gonzo41 5y ago
      Sniping isn't that simple. The risk of counterfire is real. I think autonomous weapons will always be more successful in area denial and holding territory. Ie landmines.
alexf95 5y ago
It always has been scary to me to how weapons evolve over time. Even though they are (thank god) mostly not needed they still keep developing further to be more lethal and efficient just to be "prepared" to what might happen in the future.
some_random 5y ago
There's an interesting similarity between autonomous weapon systems and ransomware discussions. Both have just come to the attention of the mainstream audience (or audiences outside their usual niche) and discussion in those spaces is about where it was 5 years ago. Also I'd really expect HN of all places to understand why banning what is basically computer vision based targeting systems is absurd.
- ethbr0 5y ago
  Technologists tend to underappreciated relative utility as a driving popularity metric -- does it accomplish a job I need, better / cheaper / faster? My thought is that ransomware is seeing a resurgence because corporations are increasingly internetworked and hybridized with the global internet, because they have to be. So ransomware has begun to outweigh espionage as a method for extracting money from corporations. Drones (as used so far, in the remote suicide version) solve a key problem many states have: lack of effective tactical ISR. The US and major powers do not have this problem. Autonomous drones solve... what problem? My inclination is that they'll deliver the same value as current-gen ML systems: mass, automated intelligence collection & basic analysis. As well as continuous monitoring and triggering on a pre-configured event (e.g. truck leaves this building, launch missile). Killing something, once you know what and where it is, is not a problem most militaries have. Training and paying and allocating large numbers of soldiers to do basic intelligence trawling is absolutely a problem most militaries have. I think you see this in the US drone approach evolution. Shifting from a single vehicle approach to a survivable system with attritable assets communicating back through stealth communication hubs. Because persistence is the real value.
  - whatshisface 5y ago
    >Killing something, once you know what and where it is, is not a problem most militaries have. It would be if there was a real war with a lot of things at once. That statement is only true for peacetime "executive action" wars with a 100:1 force ratio. It is an auxiliary role that has little to do with how things would go if there was a war between somewhat equal powers. The lessons of enforcing foreign policy on ten people who live in a country with neutral diplomatic status will probably not apply if you change the number ten or the neutral diplomatic status.
jdmoreira 5y ago
Not with current tech but let’s say 5 years from now can’t a country or even a terrorist organization deploy hundreds of autonomous drones with charging stations that will effectively shoot everyone on sight? It’s like a school massacre but at a city scale or worse. Sure eventually everyone will go inside or underground but there is still incredible damage to be made in the meanwhile. You can equip these with heat vision and they will even hunt you at night. This is also incredible tech for hunting fugitives. In the US the population has enough handguns that they can probably fight these off to some extent but in Europe it would be impossible.
- barbazoo 5y ago
  They'd have to get their hands on "hundreds of autonomous drones" and deploy them on site. Seems unlikely for any of the current terrorist organisations.
  - jdmoreira 5y ago
    really? It’s not like this would cost more than 5K usd each. It doesn’t have to be military grade this is not much more than a DJI drone with better firmware and a micro railgun
    - barbazoo 5y ago
      sure, looking at how they usually conduct business though, it seems unlikely that would happen. but whatever, it could theoretically happen, what do I know.
credit_guy 5y ago
How silly. How can such a ban be enforced? Will Al Qaeda and ISIS sign the treaty? Will Assad sign it? Will he respect it any more than he respected the ban on chemical weapons?
- tclancy 5y ago
  How about a change to how we choose to live as a world and make them illegal to build, enforce those laws strictly, etc. The fact we live in a world where arms is a hyper-profitable amoral (at best) industry doesn't mean we should or have to. >‘It seems to be easier for us today to imagine the thoroughgoing deterioration of the earth and of nature than the breakdown of late capitalism; perhaps that is due to some weakness in our imaginations’ https://www.lrb.co.uk/the-paper/v41/n09/jenny-turner/not-no-longer-but-not-yet https://www.lrb.co.uk/the-paper/v41/n09/jenny-turner/not-no-...
  - deleted 5y ago
    [deleted]
  - mam3 5y ago
    World is amoral. Welcome to reality
Element_ 5y ago
How would they enforce a ban? Will UN Inspectors start analyzing source code?
yboris 5y ago
Relevant: https://www.stopkillerrobots.org/ https://www.stopkillerrobots.org/
deleted 5y ago
[deleted]
todd8 5y ago
Banning autonomous weapons is a good idea, but history indicates that the bad guys might very well continue to develop them while claiming that they are not. For example, see the biological weapons program pursued by the former Soviet Union in secret despite being signatories of the international ban on biological weapons[1]. [1] https://en.wikipedia.org/wiki/Soviet_biological_weapons_program https://en.wikipedia.org/wiki/Soviet_biological_weapons_prog...
openasocket 5y ago
Whenever these discussions are brought up I feel the need to remind everyone that autonomous weapons and drones are not new inventions. The first remotely controlled weapons were developed and tested in the 1910s: https://en.wikipedia.org/wiki/Kettering_Bug https://en.wikipedia.org/wiki/Kettering_Bug . You also saw the emergence of drones being used as targets, to train air crews and anti-aircraft gunners. WW2 saw numerous instances of autonomous weapons being used in combat. The obvious examples are the V1 and V2, which both used internal guidance systems. There's also the Fritz X, a radio-controlled air-launched anti-ship missile. There was also Operation Aphrodite (https://en.wikipedia.org/wiki/Operation_Aphrodite https://en.wikipedia.org/wiki/Operation_Aphrodite ) in which the Allies loaded up a b-17 bomber with explosives, re-worked to be radio controlled with TV guidance, and flown into hard targets. Similarly, you have the German Mistel (https://en.wikipedia.org/wiki/Mistel https://en.wikipedia.org/wiki/Mistel ) which involved a drone loaded to capacity with explosives and guided to a target, but in this case the drone is carried by an aircraft and dropped, making this the first air-launched cruise missile used in combat. During the Cold War you started to see drones being used for reconnaissance. The US had the Ryan Model 147 and D-21. Once computers became significantly advanced you started seeing an explosion of different applications. The tomahawk missile is extensively programmable, allowing the user to give it a flight course consisting of a number of waypoints, and digital images along its intended path. It would then compare those images to what it sees on the route and use that for course corrections, ensuring extreme accuracy against targets a thousand miles away in a time before GPS. (fun fact: during the Gulf War, the Navy had to plan their Tomahawk stikes to go through the Southeastern portion of Iraq instead of a more direct route because they could use the mountains and hills for course corrections, while the direct route went over relatively featureless desert). In the late 1980s Israel would employ loitering munitions. Essentially these were suicide drones intended to take out enemy radars. It would loiter above for hours until detecting an enemy radar and then dive. There were also decoy drones. These didn't have a warhead, they were intended to simply fly around acting as bait. Some decoys are fitted with radar reflectors to make them appear larger on radar than they really are, essentially the opposite of stealth aircraft. Autonomous weapons have been around for a long time and aren't going anywhere. Frankly, I think this is probably better in terms of reducing civilian casualties. I'd rather have soldiers use a precision guided weapon like this than calling in mortars or artillery.
mam3 5y ago
This is dumb and childish on so many levels. You will NEVER ever prevent bad guys from creating them by "banning" them. Look at drugs: all this war / bans / illegality, and the cocaine traffic never ever went down in decades (maybe during covid). Best course of action is to invest in them to not be left behind
- Wavelets 5y ago
  I agree 100%. So naive. Just think of a world where we (US, EU, etc.) stop investing in this technology while Russia, China, etc. continue. It really burns me to hear AI professors rail against “autonomous weapons” yet go work at Facebook, Google, etc. Information, especially manipulation of it, is a weapon too.
namlem 5y ago
Nope. We need autonomous underwater attack vehicles if we are to defend Taiwan from Chinese invasion.
Digit-Al 5y ago
I see lots of people offering elaborate solutions to take down a swarm of drones. I may be wildly off the mark here but would it not be feasible to fly over them with a jet fighter and let the supersonic shockwave take out a lot of them? I am thinking of some other more low tech solutions to whittle them down as well: firing a weighted net; sticky silly string type stuff to tangle and gum up the rotors.
arcanon 5y ago
Initiating swarms of slaughter bots could be a warcrime.

152 points by doppioandante 5y ago

The problem with arms races is that they are races: no one wants to be left behind. What is likely to happen is any major nation-state that signs a treaty agreeing to a ban will STILL secretly be researching and building autonomous (or nearly autonomous) weapon systems because they do not want to be left behind.

omginternets 5y ago
I would add that it's also very difficult to study and develop countermeasures without cultivating cutting-edge knowledge of how such weapons are designed, built and operated. Doing that means you eventually need field data. The slope is extremely slippery, but it's also dangerous not to traverse it.
- falcolas 5y ago
  Agreed. We've already trod this ground with chemical weapons, so at least we have a template when we follow it once more with autonomous weapons.
marcinzm 5y ago
They also need to research them to learn how to counter them if an enemy does use them. Given that most of the conflicts of the last few decades were not with states but with groups the chance of someone using them is high. Especially as technology becomes more accessible.
marvin 5y ago
It would be a disaster to enter a military conflict with an adversary that has autonomous weapons if you don’t have at least effective countermeasures yourself. I suppose nuclear states could in principle rest on their laurels, but at least from a layman’s perspective it doesn’t give an impression of being strategically sound. But obviously it’s a scary rabbit hole to enter. It’s not fun when game-theoretical considerations or human nature seems to make dubious technological developments inevitable.
xvilka 5y ago
Winner will be the one who will design biological equivalents - genetically modified eagles or something similar.
- treeman79 5y ago
  We just had a gain of function escape ordeal that we are still coping with.

5560675260 5y ago

I, for one, welcome our murderbot overlords. If we can't stop using drones to kill people, we should at least try to reduce collateral damage. With some advancements autonomous drones could be more precise and reliable than human operated ones, meaning that as an outcome fewer weddings will get blown up.

StavrosK 5y ago
And if fewer weddings get blown up, we'll use them more and more to kill people. I wonder whether the absolute number of killings will go up or down this way.
- Matticus_Rex 5y ago
  Wait, why will we use them more and more to kill people if fewer weddings get blown up? I'm not seeing the causal link.
  - StavrosK 5y ago
    Because blowing up weddings leads to dissatisfied voters, so politicians are less likely to do that very often.
  - ahepp 5y ago
    I think they are suggesting current usage is limited by political fallout due to the misidentification rate. They hypothesize reducing the misidentification rate will increase the usage rate, ultimately killing more people (but fewer wedding parties)
  - Jtsummers 5y ago
    If they're accurate (or more accurate) then there's less resistance to using them more. They will be used to target an increasingly large number of people (or individuals) with a kind of specificity that present weapon systems cannot be used for and which impedes their use in some cases (because of the risk of collateral damage or the political fallout from a recent incident of bombing a wedding party).

marcus_holmes 5y ago

This feels like the same thing that happened with tanks. The end of WW1 saw tanks deployed effectively. WW2 was then fought using entirely new tactics and strategies enabled by tanks (not least "blitzkrieg"). Britain and France, notably, hadn't really worked out how to use them effectively and so were left fighting the last war, horribly disadvantaged until they worked out how to catch up (though air power being developed at the same time also had an influence). So now we have drones, and all those incredibly expensive fighter planes are useless. Aircraft carriers will be replaced with cheap drone swarm carriers, so naval strategy will need to change. Anti-tank drone swarms will make conventional armour more or less ineffective. And so on. The good news is that there will be anti-drone swarms too. And so we'll end up fighting battles mostly by destroying drones instead of people. Once you've knocked out all the enemy drones, they'll surrender because humans can't fight drones. Interesting times.

nickthemagicman 5y ago
> And so we'll end up fighting battles mostly by destroying drones instead of people. This is one piece of possibly good news that it seems like everyone is overlooking. Automation/Remote control applies not just to drones, but tanks, boats, other aerial vehicles, and maybe troops(after seeing that Boston Dynamics dog robot) etc. Seems like it will be a small squad of highly advanced human troops to move in after the Remotes to secure and maintain control of the area. So many lives will be spared during wars. However, this may be unbeatable tech by humans, so any future revolutions will be easily quelled by a gov't with a few drone pilots piloting a drone swarm in a room a thousand miles away.
- marcus_holmes 5y ago
  I agree. This isn't all bad :) Like the move from conscript peasant armies to volunteer professional armies, leave the rest of us out of it. It'll come down to "how many drones, and of what quality, can your manufacturing base produce?". But this is not that dissimilar from WW2's "how many tanks, and of what quality, can you manufacture?" equation. Insurgencies will be about getting a small swarm in exactly the right place at the right time.
- 5y ago

dctoedt 5y ago

The "Slaughterbot" video: https://www.youtube.com/watch?v=9CO6M2HsoIA https://www.youtube.com/watch?v=9CO6M2HsoIA

crumbshot 5y ago

Not only should there should be no autonomous drones, the non-autonomous remote controller hardware should be wired up such that if they are shot out of the sky, this kills the human operator and anyone sitting nearby. This would be much fairer than the current system of gamified point-and-click murder.

barbazoo 5y ago
As horrible as this sounds, I like the "real world consequence" part of it. Maybe this would lead to people reconsidering their participation in those war games.
- nick238 5y ago
  If you die in the game, you die for real!
UncleEntity 5y ago
Who, realistically, would enforce such a system? I'm pretty sure any artificial rules which make war "fairer" will just be ignored when real bullets start flying since "it's better to be judged by twelve than to be carried by six" is an actual unofficial rule in the military.

jcims 5y ago

Product page for the UAV in question: https://www.stm.com.tr/en/kargu-autonomous-tactical-multi-rotor-attack-uav https://www.stm.com.tr/en/kargu-autonomous-tactical-multi-ro...

barbazoo 5y ago
Does anyone know what "Indigenous National Embedded Hardware and Software" means? Domestically developed maybe?
- jcims 5y ago
  You could ask them on twitter lol - https://twitter.com/STMDefence https://twitter.com/STMDefence Surreal.

gpm 5y ago

Autonomous weapons should be embraced, otherwise less scrupulous people will embrace them, and they will win. Non-autonomous weapons will simply not be able to compete long term. Actually we wouldn't even be able to compete today, things like missiles are already autonomous, and if they weren't we would simply be unable to compete in any battlefield where radio was jammed, and it would give a large force multiplier to adversaries even in battle fields where radio is not jammed.

CountDrewku 5y ago

They seem to be only presenting the negative aspects of "slaughterbots". Could they not potentially be better than human controlled devices since they should be able to more accurately target things and create less collateral damage? Of course, there's always the doomsday terminator scenario where they go haywire and decide to start targeting all humans....

SonicScrub 5y ago
It's worth noting that this line of reasoning has been used to justify the invention of machines that ended up causing many more deaths than before. Richard Gatling was inspired to invent the Gatling gun as a way to reduce the number of deaths by combat. His logic was that if could replace the lines of musket-wielding infantry with a small team that could fire just as many rounds, the number of soldiers an army would require would drop, and therefore the number of combat deaths would drop as well. This line of reasoning was proven wrong, as the deployment of Gatling guns simply caused more causalities in the American Civil War, and the gun's descendants enabled the mass slaughter in the First World War. Similar logic was applied to bombers in the Second World War. If bombers could fly into enemy territory, hit key targets behind enemy lines and fly out, then the need for armies would go away. Instead we got The Blitz, the firebombing of Tokyo and Dresden, and the deadliest war in history. Maybe this time is somehow different, but it's an important historical context to consider. Often the expansion of our destructive capabilities simply encourages more destruction.
- CountDrewku 5y ago
  Yes I'm just playing devil's advocate. My initial response is that they should be banned but like most things someone will continue creating them. Do we really think China cares whether there's a ban? Typically the only thing that stops WMD type weapons (not sure if slaughterbots fall into this category) is the enemy having them as well. Assured mutual destruction seems to be the only viable option.

ccsnags 5y ago

In the book “The Diamond Age” by Neal Stephenson there are mist-like swarms of nanobots that act as an immune system for regions against malicious nanobots. I can picture there being something like this with drones. There are problems to this. The major one being that, as mentioned by the posts on this thread, you have to develop the weapon to really know how to counter it.

_jal 5y ago
That's the Extropian "Utility Fog" concept. A more direct science fiction forerunner of this is Bruce Stirling's _Islands in the Net_ (1988). There is a scene where cheap explosive drone swarms are used in terror attacks. And of course, there's the famous first line of _Count Zero_. "They sent a Slamhound on Turner's trail in New Delhi, slotted it to his pheromones and the color of his hair. It caught up with him on a street called Chandni Chauk and came scrambling for his rented BMW - Its core was a kilogram of recrystallized hexogene and flaked TNT."
- deleted 5y ago
  [deleted]

ru552 5y ago

We have hand held weapons to combat drones today. See https://www.droneshield.com/dronegun-tactical https://www.droneshield.com/dronegun-tactical https://www.reddit.com/r/pics/comments/o0kzuq/the_security_on_the_biden_king_phillippe_meeting/ https://www.reddit.com/r/pics/comments/o0kzuq/the_security_o...

mdorazio 5y ago
These countermeasures generally only work against drones that are not fully autonomous since they rely on disrupting communication with a ground controller. If your drone is perfectly happy to loiter and kill people based on facial recognition and some basic parameters, dronegun won't work. The other point to keep in mind is that cheap lethal drones are often designed to operate in swarm mode rather than individually. It's pretty difficult to knock out an autonomous swarm with weapons designed to target one drone at a time.

superbaconman 5y ago

Is this the official position of ieee? It's seems both unrealistic and unconstitutional (at least within the US).

Jtsummers 5y ago
It's a piece in IEEE Spectrum, lots of people submit articles there. With respect to unconstitutionality, really? Are you also able to acquire and keep on your property a functioning ICBM or nuclear bomb? Arms controls exist in the US despite the 2nd amendment.
- superbaconman 5y ago
  Considering that the technology to build such drones is so widely available there's no reason why small arm drones wouldn't be protected under the 2nd amendment. Drones with bombs are a different story, but will be developed regardless of any international agreement (just like bio weapons).

michaelt 5y ago

Are these drone-based 'lethal autonomous weapons' markedly different to cruise missiles? Or is the alarm here about the fact that every man and his dog will be able to get their hands on them, when they cost $1000 instead of $1000000?

neurobashing 5y ago
I guess the answer to me is "it depends". Modern cruise missiles are pretty much drones, but at the same time they're always what I guess I'd call "weapons of state". Big, expensive, and part of a large decision-making loop. You can't just lob one; someone signed for it. Attacks are planned in advance at high levels. whereas this "slaughterbot" idea is smaller, low-cost, and with far less top-down control. At least, as I understand it.
nick238 5y ago
I think the latter. For what it's worth, the 'arms' that are still protected by the 2nd Amendment are still approximately the same as in 1787 (line of sight, bang bang), as you're legally restricted from owning guided bombs, cruise missiles, nuclear weapons, etc, but even aside from that, you're practically restricted because of limited availability and/or extreme cost. On the flip side, the hardware for Slaughterbots is basically here: TinyWhoops cost ~$100, say ~$100 to upgrade the ARM processor to one that can process realtime video, ~$100 for a shaped charge. So the hardware is super cheap, and once developed, the software to tie it all together has nearly no incremental cost. Once it's out there, that's the genie leaving the bottle.
zokier 5y ago
That is my question too. Tomahawk missiles have been fielded since early 80s, and are very much fire and forget autonomous weapons that can track targets based on sensor data (i.e. not reliant on GPS or other external guidance). Maybe the small difference is that cruise missiles generally are not anti-personnel weapons; maybe the article is seeking a ban in the same sense that Ottawa treaty banned anti-personnel mines but allows anti-tank mines. But I think this is something that would need to be explicitly addressed instead of just spouting sky is falling rhetoric.

jadzdc 5y ago

STM, the Turkish company that makes the KARGU drone referenced in the article has a promotional video from 2018 showing the unit being demonstrated: https://www.youtube.com/watch?v=Oqv9yaPLhEk https://www.youtube.com/watch?v=Oqv9yaPLhEk

Invictus0 5y ago

Weapons that have been successfully banned in the past typically had an alternative technology that it could be substituted with. I'm not sure you can say the same of lethal drones. In the case of Azerbaijan vs Armenia, the sum total of the fighter aircraft of BOTH nations combined is just 5. For poorer nations, drones represent a democratization of the battlefield; an extraordinarily cheap technology with a lot of firepower per dollar. I don't think these countries will be willing to give this up easily.

nabla9 5y ago

I don't fully understand how there is real difference? If you give a swarm of drones a kill zone and time limit, then they try them to kill every human in the area, how is that so different from MLRS salvo? Cruise missiles are already slaughterbots. You just give them location and they kill everyone in that location.

dgan 5y ago
Location is wrong argument. The article specifically points out that those drones were said to follow retreating fighters, and keep shooting un-discriminately
- stickfigure 5y ago
  Retreating fighters are still combatants. Surrendering fighters would be a different matter.

CoolGuySteve 5y ago

One of the reasons the attempted drone assassination of Maduro in 2018 failed was because police were using radio jammers: https://en.wikipedia.org/wiki/2018_Caracas_drone_attack https://en.wikipedia.org/wiki/2018_Caracas_drone_attack It's fairly easy and cheap to make an explosive drone and train a neural network to drive it using off the shelf components and open source software. It follows that it's inevitable that at least semi-autonomous drones will be made to counter radio jamming. So while the best outcome a treaty could hope for is to prevent mass manufacturing these things by some countries, any motivated small country/individual will be able to craft them.

gonzo41 5y ago
Drones can't beat nylon nets or other low tech countermeasures. I think anti drone water cannons will outpace drone assassinations. And more than both, regular bombs and guns will still do most of the killing.
- gpm 5y ago
  A motorized sniper rifle with face detection to find targets and a high rate of fire seems like it could exist with current technology, require about the same effort to deploy as a sniper with a typical sniper rifle, require less training from the person in the field, and be a hell of a lot more deadly. Same with a machine gun for closer quarters. Regular bombs and guns probably still do most of the killing for quite awhile because of inertia, but I'm pretty firmly of the belief that they are outdated technology or will be soon, especially the regular guns part. Regular bombs are arguably already outdated by missiles with (frequently autonomous) guidance... but for a lot of applications dumb explosives do work pretty well so those probably stick around for a long time.
  - gonzo41 5y ago
    Sniping isn't that simple. The risk of counterfire is real. I think autonomous weapons will always be more successful in area denial and holding territory. Ie landmines.

alexf95 5y ago

It always has been scary to me to how weapons evolve over time. Even though they are (thank god) mostly not needed they still keep developing further to be more lethal and efficient just to be "prepared" to what might happen in the future.

some_random 5y ago

There's an interesting similarity between autonomous weapon systems and ransomware discussions. Both have just come to the attention of the mainstream audience (or audiences outside their usual niche) and discussion in those spaces is about where it was 5 years ago. Also I'd really expect HN of all places to understand why banning what is basically computer vision based targeting systems is absurd.

ethbr0 5y ago
Technologists tend to underappreciated relative utility as a driving popularity metric -- does it accomplish a job I need, better / cheaper / faster? My thought is that ransomware is seeing a resurgence because corporations are increasingly internetworked and hybridized with the global internet, because they have to be. So ransomware has begun to outweigh espionage as a method for extracting money from corporations. Drones (as used so far, in the remote suicide version) solve a key problem many states have: lack of effective tactical ISR. The US and major powers do not have this problem. Autonomous drones solve... what problem? My inclination is that they'll deliver the same value as current-gen ML systems: mass, automated intelligence collection & basic analysis. As well as continuous monitoring and triggering on a pre-configured event (e.g. truck leaves this building, launch missile). Killing something, once you know what and where it is, is not a problem most militaries have. Training and paying and allocating large numbers of soldiers to do basic intelligence trawling is absolutely a problem most militaries have. I think you see this in the US drone approach evolution. Shifting from a single vehicle approach to a survivable system with attritable assets communicating back through stealth communication hubs. Because persistence is the real value.
- whatshisface 5y ago
  >Killing something, once you know what and where it is, is not a problem most militaries have. It would be if there was a real war with a lot of things at once. That statement is only true for peacetime "executive action" wars with a 100:1 force ratio. It is an auxiliary role that has little to do with how things would go if there was a war between somewhat equal powers. The lessons of enforcing foreign policy on ten people who live in a country with neutral diplomatic status will probably not apply if you change the number ten or the neutral diplomatic status.

jdmoreira 5y ago

Not with current tech but let’s say 5 years from now can’t a country or even a terrorist organization deploy hundreds of autonomous drones with charging stations that will effectively shoot everyone on sight? It’s like a school massacre but at a city scale or worse. Sure eventually everyone will go inside or underground but there is still incredible damage to be made in the meanwhile. You can equip these with heat vision and they will even hunt you at night. This is also incredible tech for hunting fugitives. In the US the population has enough handguns that they can probably fight these off to some extent but in Europe it would be impossible.

barbazoo 5y ago
They'd have to get their hands on "hundreds of autonomous drones" and deploy them on site. Seems unlikely for any of the current terrorist organisations.
- jdmoreira 5y ago
  really? It’s not like this would cost more than 5K usd each. It doesn’t have to be military grade this is not much more than a DJI drone with better firmware and a micro railgun
  - barbazoo 5y ago
    sure, looking at how they usually conduct business though, it seems unlikely that would happen. but whatever, it could theoretically happen, what do I know.

credit_guy 5y ago

How silly. How can such a ban be enforced? Will Al Qaeda and ISIS sign the treaty? Will Assad sign it? Will he respect it any more than he respected the ban on chemical weapons?

tclancy 5y ago
How about a change to how we choose to live as a world and make them illegal to build, enforce those laws strictly, etc. The fact we live in a world where arms is a hyper-profitable amoral (at best) industry doesn't mean we should or have to. >‘It seems to be easier for us today to imagine the thoroughgoing deterioration of the earth and of nature than the breakdown of late capitalism; perhaps that is due to some weakness in our imaginations’ https://www.lrb.co.uk/the-paper/v41/n09/jenny-turner/not-no-longer-but-not-yet https://www.lrb.co.uk/the-paper/v41/n09/jenny-turner/not-no-...
- deleted 5y ago
  [deleted]
- mam3 5y ago
  World is amoral. Welcome to reality

Element_ 5y ago

How would they enforce a ban? Will UN Inspectors start analyzing source code?

yboris 5y ago

Relevant: https://www.stopkillerrobots.org/ https://www.stopkillerrobots.org/

deleted 5y ago

[deleted]

todd8 5y ago

Banning autonomous weapons is a good idea, but history indicates that the bad guys might very well continue to develop them while claiming that they are not. For example, see the biological weapons program pursued by the former Soviet Union in secret despite being signatories of the international ban on biological weapons[1]. [1] https://en.wikipedia.org/wiki/Soviet_biological_weapons_program https://en.wikipedia.org/wiki/Soviet_biological_weapons_prog...

openasocket 5y ago

Whenever these discussions are brought up I feel the need to remind everyone that autonomous weapons and drones are not new inventions. The first remotely controlled weapons were developed and tested in the 1910s: https://en.wikipedia.org/wiki/Kettering_Bug https://en.wikipedia.org/wiki/Kettering_Bug . You also saw the emergence of drones being used as targets, to train air crews and anti-aircraft gunners. WW2 saw numerous instances of autonomous weapons being used in combat. The obvious examples are the V1 and V2, which both used internal guidance systems. There's also the Fritz X, a radio-controlled air-launched anti-ship missile. There was also Operation Aphrodite (https://en.wikipedia.org/wiki/Operation_Aphrodite https://en.wikipedia.org/wiki/Operation_Aphrodite ) in which the Allies loaded up a b-17 bomber with explosives, re-worked to be radio controlled with TV guidance, and flown into hard targets. Similarly, you have the German Mistel (https://en.wikipedia.org/wiki/Mistel https://en.wikipedia.org/wiki/Mistel ) which involved a drone loaded to capacity with explosives and guided to a target, but in this case the drone is carried by an aircraft and dropped, making this the first air-launched cruise missile used in combat. During the Cold War you started to see drones being used for reconnaissance. The US had the Ryan Model 147 and D-21. Once computers became significantly advanced you started seeing an explosion of different applications. The tomahawk missile is extensively programmable, allowing the user to give it a flight course consisting of a number of waypoints, and digital images along its intended path. It would then compare those images to what it sees on the route and use that for course corrections, ensuring extreme accuracy against targets a thousand miles away in a time before GPS. (fun fact: during the Gulf War, the Navy had to plan their Tomahawk stikes to go through the Southeastern portion of Iraq instead of a more direct route because they could use the mountains and hills for course corrections, while the direct route went over relatively featureless desert). In the late 1980s Israel would employ loitering munitions. Essentially these were suicide drones intended to take out enemy radars. It would loiter above for hours until detecting an enemy radar and then dive. There were also decoy drones. These didn't have a warhead, they were intended to simply fly around acting as bait. Some decoys are fitted with radar reflectors to make them appear larger on radar than they really are, essentially the opposite of stealth aircraft. Autonomous weapons have been around for a long time and aren't going anywhere. Frankly, I think this is probably better in terms of reducing civilian casualties. I'd rather have soldiers use a precision guided weapon like this than calling in mortars or artillery.

mam3 5y ago

This is dumb and childish on so many levels. You will NEVER ever prevent bad guys from creating them by "banning" them. Look at drugs: all this war / bans / illegality, and the cocaine traffic never ever went down in decades (maybe during covid). Best course of action is to invest in them to not be left behind

Wavelets 5y ago
I agree 100%. So naive. Just think of a world where we (US, EU, etc.) stop investing in this technology while Russia, China, etc. continue. It really burns me to hear AI professors rail against “autonomous weapons” yet go work at Facebook, Google, etc. Information, especially manipulation of it, is a weapon too.

namlem 5y ago

Nope. We need autonomous underwater attack vehicles if we are to defend Taiwan from Chinese invasion.

Digit-Al 5y ago

I see lots of people offering elaborate solutions to take down a swarm of drones. I may be wildly off the mark here but would it not be feasible to fly over them with a jet fighter and let the supersonic shockwave take out a lot of them? I am thinking of some other more low tech solutions to whittle them down as well: firing a weighted net; sticky silly string type stuff to tangle and gum up the rotors.

arcanon 5y ago

Initiating swarms of slaughter bots could be a warcrime.