4 ms·

With a self-driving car, the desired outcome is to save lives. With an autonomous weapon, the desired outcome is to kill. Having a human inefficiency in the loo

With a self-driving car, the desired outcome is to save lives. With an autonomous weapon, the desired outcome is to kill. Having a human inefficiency in the loop means you can have mercy and refrain from killing. I don't think the problem is with AIs acurrately identifying friends and enemies. You can't solve that 100% anyway, but if it can aid humans and prevent mistakes, all the better. The problem is when you start to build absolutely relentless border drones that can one-shot the enemy, without any possibility for mercy or negotiation.

betwixthewires 6y ago
I don't think there's a single militarized border on earth where the sentry holding the gun has any say on whether he must shoot someone who violates the perimeter. Mercy and negotiation on these issues happen with diplomacy, not along the fenceline. With weapons, the desired outcome is not to kill, not in most circumstances anyway. In circumstances where it is, it is almost always the desire of an individual wielding a weapon. Normally, the desired outcome is deterrence or defense, and when used proactively, to lessen the enemy's ability to continue to wage the conflict. With deterrence, the threat of death is the deterrent, with defense, killing is the machanism by which the threat is neutralized, and with proactive aggression the goal may or may not include killing, and is more likely to include killing if the enemy's ability to wage conflict is dependent on manpower. In all three scenarios, the desired outcome of developing autonomous military capability is to save lives.
- bigfudge 6y ago
  I doubt that is true. In Iraq and Afghanistan there was wide variation in the number of civilians killed by different units. Some of that may be context-effects, but I doubt all of it. Also, having people in the loop reminds at least someone that this is 'real' and that we need to carry on thinking about it. If you can obliterate enemies' bodies and the only duty on people is to restock the ammo I think that reduces the moral burden of conflict in an unacceptable way.
- captainmuon 6y ago
  What I mean is, if your goal is to keep people out of a certain area, period, the drone will say "can do", and it will be able to do it with frightening precision and no doubts. Humans have natural stopping points. One maybe stupid example is when there were lots of Syrian refugees on the way through Europe in 2015. We learned the hard way that, if people are pushing against your borders, the only way to keep them out is to eventually use force. We were not willing to do so, so we had to accept that and grant people asylum (which I think was the right decision). Imagine (a bit scifi but for the sake of argument) "border patrol 2000", which will give a warning, then give a more stern warning, then raise a wall, then try rubber bullets, then try to keep you from climbing the wall with a grappling arm, and finally using live rounds. If you are really determined it will eventually have to shoot. Human solderies did shoot in the past but they did also sometimes not shoot, and it always had a political aftermath. But with automatic systems, it is too easy to psychologically detach, and to say it's e.g. the refugees own fault.

betwixthewires 6y ago

I don't think there's a single militarized border on earth where the sentry holding the gun has any say on whether he must shoot someone who violates the perimeter. Mercy and negotiation on these issues happen with diplomacy, not along the fenceline. With weapons, the desired outcome is not to kill, not in most circumstances anyway. In circumstances where it is, it is almost always the desire of an individual wielding a weapon. Normally, the desired outcome is deterrence or defense, and when used proactively, to lessen the enemy's ability to continue to wage the conflict. With deterrence, the threat of death is the deterrent, with defense, killing is the machanism by which the threat is neutralized, and with proactive aggression the goal may or may not include killing, and is more likely to include killing if the enemy's ability to wage conflict is dependent on manpower. In all three scenarios, the desired outcome of developing autonomous military capability is to save lives.

bigfudge 6y ago
I doubt that is true. In Iraq and Afghanistan there was wide variation in the number of civilians killed by different units. Some of that may be context-effects, but I doubt all of it. Also, having people in the loop reminds at least someone that this is 'real' and that we need to carry on thinking about it. If you can obliterate enemies' bodies and the only duty on people is to restock the ammo I think that reduces the moral burden of conflict in an unacceptable way.
captainmuon 6y ago
What I mean is, if your goal is to keep people out of a certain area, period, the drone will say "can do", and it will be able to do it with frightening precision and no doubts. Humans have natural stopping points. One maybe stupid example is when there were lots of Syrian refugees on the way through Europe in 2015. We learned the hard way that, if people are pushing against your borders, the only way to keep them out is to eventually use force. We were not willing to do so, so we had to accept that and grant people asylum (which I think was the right decision). Imagine (a bit scifi but for the sake of argument) "border patrol 2000", which will give a warning, then give a more stern warning, then raise a wall, then try rubber bullets, then try to keep you from climbing the wall with a grappling arm, and finally using live rounds. If you are really determined it will eventually have to shoot. Human solderies did shoot in the past but they did also sometimes not shoot, and it always had a political aftermath. But with automatic systems, it is too easy to psychologically detach, and to say it's e.g. the refugees own fault.