5 ms·

The military always claims they will have a human 'in the loop' to make kill decisions. That promise will go out the window just as soon as other nations arm a

The military always claims they will have a human 'in the loop' to make kill decisions. That promise will go out the window just as soon as other nations arm ai enabled drones with anti air missiles. There's no time for 'human in the loop' when the first to act survives. Thus down the slippery slope we go. In 20 years time we'll probably have aircraft that do figure eights over enemy territory for hours before automatically engaging 'military aged males'. Slaughter bots is the ultimate conclusion to this: https://www.youtube.com/watch?v=O-2tpwW0kmU https://www.youtube.com/watch?v=O-2tpwW0kmU

_8j50 4y ago
Ok, so? Less humans die and how are they tactically or strategically different from a precise wmd? I.e.: their implication can deter conflict.
- twosdai 4y ago
  I think the main idea is ceding control to a system to make the kill, don't kill decision. This is the "so" part of the article and the comment. With wmds, there are humans who make the choice at the end of the day. The loss of life scale is not relevant as much since we could make ai powered wmds as well, the fundamental question is "can a computer kill?" And maybe more importantly "who is responsible for it making mistakes? L"
  - _8j50 4y ago
    It can make a decision to kill but humans who operate it will always be responsible, mistake or not.
2OEH8eoCRo0 4y ago
> There's no time for 'human in the loop' when the first to act survives. Of course there is. "AI" controlled hardware doesn't care about getting out alive. A human is more likely to have an itchy trigger finger because a mistake means death.
- ceejayoz 4y ago
  "AI" controlled hardware doesn't care about anything. A human is more likely to say "wait, this order doesn't make sense" or "what if those are just civilians?"
  - 2OEH8eoCRo0 4y ago
    You're missing my point. A robot can afford to wait and get it right because it has no concept of self preservation. The operator giving the kill/no-kill can afford to wait and get it right because they're not in immediate danger. How would a stressed human in mortal danger be preferable here?
    - ceejayoz 4y ago
      > How would a stressed human in mortal danger be preferable here? https://theintercept.com/2015/11/19/former-drone-operators-say-they-were-horrified-by-cruelty-of-assassination-program/ https://theintercept.com/2015/11/19/former-drone-operators-s... "U.S. drone operators are inflicting heavy civilian casualties and have developed an institutional culture callous to the death of children and other innocents, four former operators said at a press briefing today in New York." It's a lot easier to be callous towards pixels on a screen, IMO, let alone an AI model trained on past data that makes the shoot/don't shoot call. I recognize that combat stress can cause bad decisions to be made, but I don't think the cost/benefit calculus is as clear-cut as you're making it out to be.
      - 2OEH8eoCRo0 4y ago
        > It's a lot easier to be callous towards pixels on a screen What altitude do you think these payloads are dropped from? The targets are pixels on a screen regardless. I think the issue you raised is worth talking about but I think it's orthogonal to the AI issue. The AI kill/no-kill issue can be mitigated via process. Reviews, shorter "deployment", rotations, etc. We aren't creating AI systems to fight Taliban, it's for a future war where real tangible peer threats can touch you. How would you process away or mitigate the scared young grunt or pilot who are facing death? I'm advocating for humans remaining in the loop btw.
alkonaut 4y ago
> That promise will go out the window just as soon as other nations arm ai enabled drones with anti air missiles. Ethics don't go out the window because of autonomy. It's dark, but it's not somehow easy to shrug off as "ok no human then no ethics". We already have semi-autonomous weapons such as CIWS (Goalkeeper/Phalanx) which are autonomous because it's very tedious for a human to sit idle and wait for threats and then it's really hard to hit them. Perfect job for a robot. But the ethical job of the human in that case is ensuring that the weapon isn't armed if there is too high risk of collateral damage (These sit on ships, typically, so a human decides whether it's safe to arm the weapon given the situation around the vessel). The dumbest autonomous weapons are mines, they are hundreds of years old. The ethical decision is made when laying down the mine, not when the explosion is triggered. So yes it's a slippery slope, but the "human in the loop" isn't only ethical when it's the kill decision. There is a "human in the loop" when laying a mine too. The ethics involved consists of weighing the risk of killing the wrong target, properly documenting where they are planted and so on. There is no human "in the loop" at the time the mine kills someone. But that doesn't somehow throw ethics out the window. The person deploying the mine - or the slaughterbot - is the person in the loop. > In 20 years time we'll probably have aircraft that do figure eights over enemy territory for hours before automatically engaging 'military aged males'. I think that is unlikely, but possibly "armed" personnel. We already have craft that do figure eights for hours before automatically engaging specific signatures of vehicles (if we'd want to). And as I said, even artillery shells do this. But yes, finding a concentration of troops and deciding whether they are soldiers/civilian isn't even 20 years away.
- Groxx 4y ago
  I'm pretty sure they were claiming that ethics go out the window when threatened, and it will probably lead to using AI in a problematic way because it's easy to see it as a quick win (via improved autonomy), and once it's out there it tends to stay in use. Not that autonomy leads to ethics taking a break. The latter may also be true, but that wasn't what they were claiming.
- merely-unlikely 4y ago
  On your point about land mines, antipersonnel variants have actually been banned by the UN for a while now - "Anti-personnel landmines are prohibited under the Convention on the Prohibition of the Use, Stockpiling, Production and Transfer of Anti-Personnel Mines and on Their Destruction (or Mine Ban Convention), adopted in 1997."[1] As you say, the ethics debate isn't really new. [1] https://www.un.org/disarmament/convarms/landmines/ https://www.un.org/disarmament/convarms/landmines/
pmarreck 4y ago
so... the obvious response to this is anti-drone drones or lasers (lasers can't be dodged by "stochastic movements")... which means things evolve into a robot war... which means we then mutually decide to simulate the war online inside computers and accept the outcome to save money on both sides... which means we end up at PVP multiplayer war... which is where it should have gone to begin with ;)
- robertlagrant 4y ago
  We should also decommission all the nukes and get world leaders to play DEFCON[0]. [0] https://store.steampowered.com/app/1520/DEFCON https://store.steampowered.com/app/1520/DEFCON
jonnybgood 4y ago
The only time where there is any possibility of removing the human from the loop is at the advent of AGI. There are many factors on and off the battlefield, such as the domestic and geopolitical spheres, that go in to targeting. Only at the minimum level of an AGI that’s capable of understanding human intentions and situational nuances can potentially remove a human from the loop.
GartzenDeHaes 4y ago
This promise of "a human in the loop" can be met simply by adjusting the timing. For example, authorizing action prior to launch or remotely prior to engagement. This is what they do to allow single seat fighters to drop nuclear weapons, for example.
polishdude20 4y ago
If we've got AI bots able to do that kind of stuff, why would an enemy care anymore about military aged males?
deleted 4y ago
[deleted]
rtkwe 4y ago
> There's no time for 'human in the loop' when the first to act survives. That does depend a lot on the detection vs kill range vs time to kill. In a close air dog fight sure but most modern fighter doctrine is focused around long range kills and stealth already. Cannons are nearly vestigial on the F35, it only has ~4-5 seconds of ammo worth to fire.
LarryMullins 4y ago
> The military always claims they will have a human 'in the loop' to make kill decisions. That promise will go out the window just as soon as The promise is already a farce because autonomous weapons that sense the environment and make unilateral decisions to kill have already been in use for decades now. From low-tech landmines to high-tech ship-classifying CAPTOR torpedoes which fire themselves, a weapon sensing its surroundings and deciding to kill is nothing new.
- robertlagrant 4y ago
  It is new. Landmines can be reasonably clearly demarcated. Torpedoes can only be fired at certain things. AI-powered aircraft have the potential to be flying over civilian populations, or at least near enough to hit them with a missile. And with landmines - they're hideous and we shouldn't want more things like them anyway.
  - LarryMullins 4y ago
    > Torpedoes can only be fired at certain things. Yeah, the humans program the torpoedo to discriminate enemy ships from others, and then entrust that classifier to make autonomous kill decisions. How will autonomous drones and missiles work any differently? They're going to be precisely the same; classifiers engineered/trained by humans entrusted to make decisions to kill. They might shoot a city full of civilians, and a CAPTOR torpedo might shoot a ship full of civilians too. It's nothing new, it's more of the same brought to a wider scope than before.
    - robertlagrant 4y ago
      I think CAPTOR is a deep water anti-submarine device? I agree that a roving band of nearby civilians in a submarine would be at risk, but that's what a war zone is for. But more generally, I see what you're saying: it's the same problem, but at a different scale - in this case, the scale of "distance from the weapon". Minefields are very localised; underwater anti-submarine mines require a submarine; autonomous aircraft are just the same, but with a much bigger range. However, I think the range matters. Civilians avoiding a war zone in a ship or a minefield, while bad, is not the same thing as autonomous aeroplanes able to go to a static civilian population and fire at it.
deleted 4y ago
[deleted]
jareklupinski 4y ago
Don't forget the endgame https://www.youtube.com/watch?v=IjJmTeBSEzU https://www.youtube.com/watch?v=IjJmTeBSEzU > The last war broke out. All the people are dead, but the machines continue dutifully to follow orders.

_8j50 4y ago

Ok, so? Less humans die and how are they tactically or strategically different from a precise wmd? I.e.: their implication can deter conflict.

twosdai 4y ago
I think the main idea is ceding control to a system to make the kill, don't kill decision. This is the "so" part of the article and the comment. With wmds, there are humans who make the choice at the end of the day. The loss of life scale is not relevant as much since we could make ai powered wmds as well, the fundamental question is "can a computer kill?" And maybe more importantly "who is responsible for it making mistakes? L"
- _8j50 4y ago
  It can make a decision to kill but humans who operate it will always be responsible, mistake or not.

2OEH8eoCRo0 4y ago

> There's no time for 'human in the loop' when the first to act survives. Of course there is. "AI" controlled hardware doesn't care about getting out alive. A human is more likely to have an itchy trigger finger because a mistake means death.

ceejayoz 4y ago
"AI" controlled hardware doesn't care about anything. A human is more likely to say "wait, this order doesn't make sense" or "what if those are just civilians?"
- 2OEH8eoCRo0 4y ago
  You're missing my point. A robot can afford to wait and get it right because it has no concept of self preservation. The operator giving the kill/no-kill can afford to wait and get it right because they're not in immediate danger. How would a stressed human in mortal danger be preferable here?
  - ceejayoz 4y ago
    > How would a stressed human in mortal danger be preferable here? https://theintercept.com/2015/11/19/former-drone-operators-say-they-were-horrified-by-cruelty-of-assassination-program/ https://theintercept.com/2015/11/19/former-drone-operators-s... "U.S. drone operators are inflicting heavy civilian casualties and have developed an institutional culture callous to the death of children and other innocents, four former operators said at a press briefing today in New York." It's a lot easier to be callous towards pixels on a screen, IMO, let alone an AI model trained on past data that makes the shoot/don't shoot call. I recognize that combat stress can cause bad decisions to be made, but I don't think the cost/benefit calculus is as clear-cut as you're making it out to be.
    - 2OEH8eoCRo0 4y ago
      > It's a lot easier to be callous towards pixels on a screen What altitude do you think these payloads are dropped from? The targets are pixels on a screen regardless. I think the issue you raised is worth talking about but I think it's orthogonal to the AI issue. The AI kill/no-kill issue can be mitigated via process. Reviews, shorter "deployment", rotations, etc. We aren't creating AI systems to fight Taliban, it's for a future war where real tangible peer threats can touch you. How would you process away or mitigate the scared young grunt or pilot who are facing death? I'm advocating for humans remaining in the loop btw.

alkonaut 4y ago

> That promise will go out the window just as soon as other nations arm ai enabled drones with anti air missiles. Ethics don't go out the window because of autonomy. It's dark, but it's not somehow easy to shrug off as "ok no human then no ethics". We already have semi-autonomous weapons such as CIWS (Goalkeeper/Phalanx) which are autonomous because it's very tedious for a human to sit idle and wait for threats and then it's really hard to hit them. Perfect job for a robot. But the ethical job of the human in that case is ensuring that the weapon isn't armed if there is too high risk of collateral damage (These sit on ships, typically, so a human decides whether it's safe to arm the weapon given the situation around the vessel). The dumbest autonomous weapons are mines, they are hundreds of years old. The ethical decision is made when laying down the mine, not when the explosion is triggered. So yes it's a slippery slope, but the "human in the loop" isn't only ethical when it's the kill decision. There is a "human in the loop" when laying a mine too. The ethics involved consists of weighing the risk of killing the wrong target, properly documenting where they are planted and so on. There is no human "in the loop" at the time the mine kills someone. But that doesn't somehow throw ethics out the window. The person deploying the mine - or the slaughterbot - is the person in the loop. > In 20 years time we'll probably have aircraft that do figure eights over enemy territory for hours before automatically engaging 'military aged males'. I think that is unlikely, but possibly "armed" personnel. We already have craft that do figure eights for hours before automatically engaging specific signatures of vehicles (if we'd want to). And as I said, even artillery shells do this. But yes, finding a concentration of troops and deciding whether they are soldiers/civilian isn't even 20 years away.

Groxx 4y ago
I'm pretty sure they were claiming that ethics go out the window when threatened, and it will probably lead to using AI in a problematic way because it's easy to see it as a quick win (via improved autonomy), and once it's out there it tends to stay in use. Not that autonomy leads to ethics taking a break. The latter may also be true, but that wasn't what they were claiming.
merely-unlikely 4y ago
On your point about land mines, antipersonnel variants have actually been banned by the UN for a while now - "Anti-personnel landmines are prohibited under the Convention on the Prohibition of the Use, Stockpiling, Production and Transfer of Anti-Personnel Mines and on Their Destruction (or Mine Ban Convention), adopted in 1997."[1] As you say, the ethics debate isn't really new. [1] https://www.un.org/disarmament/convarms/landmines/ https://www.un.org/disarmament/convarms/landmines/

pmarreck 4y ago

so... the obvious response to this is anti-drone drones or lasers (lasers can't be dodged by "stochastic movements")... which means things evolve into a robot war... which means we then mutually decide to simulate the war online inside computers and accept the outcome to save money on both sides... which means we end up at PVP multiplayer war... which is where it should have gone to begin with ;)

robertlagrant 4y ago
We should also decommission all the nukes and get world leaders to play DEFCON[0]. [0] https://store.steampowered.com/app/1520/DEFCON https://store.steampowered.com/app/1520/DEFCON

jonnybgood 4y ago

The only time where there is any possibility of removing the human from the loop is at the advent of AGI. There are many factors on and off the battlefield, such as the domestic and geopolitical spheres, that go in to targeting. Only at the minimum level of an AGI that’s capable of understanding human intentions and situational nuances can potentially remove a human from the loop.

GartzenDeHaes 4y ago

This promise of "a human in the loop" can be met simply by adjusting the timing. For example, authorizing action prior to launch or remotely prior to engagement. This is what they do to allow single seat fighters to drop nuclear weapons, for example.

polishdude20 4y ago

If we've got AI bots able to do that kind of stuff, why would an enemy care anymore about military aged males?

deleted 4y ago

[deleted]

rtkwe 4y ago

> There's no time for 'human in the loop' when the first to act survives. That does depend a lot on the detection vs kill range vs time to kill. In a close air dog fight sure but most modern fighter doctrine is focused around long range kills and stealth already. Cannons are nearly vestigial on the F35, it only has ~4-5 seconds of ammo worth to fire.

LarryMullins 4y ago

> The military always claims they will have a human 'in the loop' to make kill decisions. That promise will go out the window just as soon as The promise is already a farce because autonomous weapons that sense the environment and make unilateral decisions to kill have already been in use for decades now. From low-tech landmines to high-tech ship-classifying CAPTOR torpedoes which fire themselves, a weapon sensing its surroundings and deciding to kill is nothing new.

robertlagrant 4y ago
It is new. Landmines can be reasonably clearly demarcated. Torpedoes can only be fired at certain things. AI-powered aircraft have the potential to be flying over civilian populations, or at least near enough to hit them with a missile. And with landmines - they're hideous and we shouldn't want more things like them anyway.
- LarryMullins 4y ago
  > Torpedoes can only be fired at certain things. Yeah, the humans program the torpoedo to discriminate enemy ships from others, and then entrust that classifier to make autonomous kill decisions. How will autonomous drones and missiles work any differently? They're going to be precisely the same; classifiers engineered/trained by humans entrusted to make decisions to kill. They might shoot a city full of civilians, and a CAPTOR torpedo might shoot a ship full of civilians too. It's nothing new, it's more of the same brought to a wider scope than before.
  - robertlagrant 4y ago
    I think CAPTOR is a deep water anti-submarine device? I agree that a roving band of nearby civilians in a submarine would be at risk, but that's what a war zone is for. But more generally, I see what you're saying: it's the same problem, but at a different scale - in this case, the scale of "distance from the weapon". Minefields are very localised; underwater anti-submarine mines require a submarine; autonomous aircraft are just the same, but with a much bigger range. However, I think the range matters. Civilians avoiding a war zone in a ship or a minefield, while bad, is not the same thing as autonomous aeroplanes able to go to a static civilian population and fire at it.

deleted 4y ago

[deleted]

jareklupinski 4y ago

Don't forget the endgame https://www.youtube.com/watch?v=IjJmTeBSEzU https://www.youtube.com/watch?v=IjJmTeBSEzU > The last war broke out. All the people are dead, but the machines continue dutifully to follow orders.