4 ms·

How would you curtail their use when any military that commits to using them will have a huge advantage ? This isn’t like nuclear weapons where any use is curt

How would you curtail their use when any military that commits to using them will have a huge advantage ? This isn’t like nuclear weapons where any use is curtailed by the apocalyptic outcomes. Killer robots are the way we will fight in the future and any military which refuses to deploy them will find themselves facing defeat.

throw_nbvc1234 4y ago
AI is not allowed to make the decision to kill, that must be done by a human. Having every other step done by an AI is still going to be a huge advantage over non-AI military forces.
- michaelmior 4y ago
  Even if the decision is made by a human, it really matters how that decision is presented. My understanding of the current state of warfare (which could be way off) is that were possible, targets are still largely selected manually based on verified intelligence. What if we allow AI to propose targets? Even if testing shows that the AI is highly effective at picking "good" targets, that changes the situation entirely. Suppose humans make the final decision of which target to strike. How much time is given to make that decision? What information is presented to them? If the answer is that humans make a decision in a short amount of time largely relying on an AI-generated confidence score, are humans really making the decision?
  - Keegs 4y ago
    Exactly. It’s too easy for these decisions to become a formality. ProPublica’s reporting on Cigna[1] is a great example: > Before health insurers reject claims for medical reasons, company doctors must review them, according to insurance laws > A Cigna algorithm flags mismatches between diagnoses and what the company considers acceptable tests and procedures for those ailments Sounds fine, except > “We literally click and submit,” one former Cigna doctor said. “It takes all of 10 seconds to do 50 at a time.” [1]: https://www.propublica.org/article/cigna-pxdx-medical-health-insurance-rejection-claims https://www.propublica.org/article/cigna-pxdx-medical-health...
  - hammyhavoc 4y ago
    Have you heard about https://www.theguardian.com/technology/2022/nov/20/part-of-the-kill-chain-how-can-we-control-weaponised-robots https://www.theguardian.com/technology/2022/nov/20/part-of-t... ? And also, how do you feel about already-existing heatseekers? A little different, but the killchain still leans on an automated system for targeting and engagement.
    - michaelmior 4y ago
      Thanks for the link! As far as existing heat-seekers, I would assume that they are generally used in cases where the only target which will likely be locked on to is the desired target. However, this is coming from a place of total ignorance of what the process of target acquisition might look like and what conditions they are used in.
      - hammyhavoc 4y ago
        Heatseekers can be very effective, but as with any automated system, it isn't infallible. Friendly-fire even happens by human beings using small arms, just like strikes on civilian schools and hospitals happen because the intelligence told a human being that it looked like it could be a "terrorist training ground". I'd be interested to see what the failure rate of an AI looked like in what actions it would have taken based on available data versus what actions human took over a several year sample. I have a feeling that the AI will either look terrible, or the human beings will look terrible, or they'll look pretty equal with strange fringe cases where the AI is better than the human, and vice versa. Judgement and authorization are interesting. Something else you might be interested in (which you may already know about) is PAL systems for nuclear weapons: https://en.wikipedia.org/wiki/Permissive_action_link https://en.wikipedia.org/wiki/Permissive_action_link You'll likely be interested in the "two-man rule": https://en.wikipedia.org/wiki/Two-man_rule https://en.wikipedia.org/wiki/Two-man_rule
        michaelmior 4y ago
        I guess my point was given my limited knowledge, it doesn't seem as though heat-seekers are necessarily any less fallible than humans. I'm not suggesting that "no worse than a human" should be the goal, but I'd say that's the bare minimum.
  - throw_nbvc1234 3y ago
    identify a target decide to attack the target implement the attack determine the result of the attack If humans are required for the decision, then humans have an inherent hook to prevent more attacks. Otherwise you run into scenarios like https://www.youtube.com/watch?v=pyMNIFZTQkg https://www.youtube.com/watch?v=pyMNIFZTQkg where humans are incapacitated and AI just continue on like nothing has happened. It makes the failure modes more sane. It's not a catch-all but a good agreeable step to take.
- Dma54rhs 4y ago
  Mines have always made the decision, dumb "AI" based on weight mainly. People do decide where to put them though.
  - alwaysbeconsing 4y ago
    Yes, and they tend to cause a lot of harm to non-combatants.
- r00fus 4y ago
  When every other part of a military decision chain involves AI, the military that wins is the one who can learn to poison or subvert enemy AI (prompt hijacking, etc). The race is on. Both to refine an AI as well as learn to confound them.
- DrBenCarson 4y ago
  How do you define "decision?" If I tell the AI "Go kill Joe," is that enough? What if I say to "Go kill the enemy." Do I have to visually confirm? Click a button? Locate Joe myself?
stareatgoats 4y ago
> How would you curtail their use I'm just hoping people and nations might come to their senses. People smarter than me need to figure out how. I'm not going to say that "it is not possible" however, that would be assuming way too much.
- hgsgm 4y ago
  Guess what happens when 99% of people are sensible and 1% have AI murderbots.
  - pmoriarty 4y ago
    Defensive drones will become as common as offensive drones. Unfortunately, both defensive and offensive drones might be hacked to attack their masters. As these drones gain more autonomy, they may become a threat to just about everyone, friend and foe.
    - etiam 4y ago
      > Defensive drones will become as common as offensive drones. Probably not good enough, as an ironclad defense is likely to remain much more difficult than offense which is occasionally successful.
    - OkayPhysicist 4y ago
      >> Unfortunately, both defensive and offensive drones might be hacked to attack their masters. That's an argument FOR greater local autonomy by the robot, rather than less. A bot that gets to decide for itself whether somebody is a friend, target, or neutral 3rd party doesn't need to be phoning home constantly, at risk of interception.
    - heurist 4y ago
      Availability of drones is entirely dependent on access to supply chains for the materials and products that go into those drones. There are some limits here, at least until we're mining the asteroid belt.
- 4y ago
nix0n 4y ago
> any military that commits to using them will have a huge advantage ? I don't think that's true. If autonomous weapons are deployed, the advantage will go to whoever has the best hackers.
- chongli 4y ago
  White hat and black hat. If you’ve got the best pentesters then your systems are going to be harder to break into, giving you the advantage.

throw_nbvc1234 4y ago

AI is not allowed to make the decision to kill, that must be done by a human. Having every other step done by an AI is still going to be a huge advantage over non-AI military forces.

michaelmior 4y ago
Even if the decision is made by a human, it really matters how that decision is presented. My understanding of the current state of warfare (which could be way off) is that were possible, targets are still largely selected manually based on verified intelligence. What if we allow AI to propose targets? Even if testing shows that the AI is highly effective at picking "good" targets, that changes the situation entirely. Suppose humans make the final decision of which target to strike. How much time is given to make that decision? What information is presented to them? If the answer is that humans make a decision in a short amount of time largely relying on an AI-generated confidence score, are humans really making the decision?
- Keegs 4y ago
  Exactly. It’s too easy for these decisions to become a formality. ProPublica’s reporting on Cigna[1] is a great example: > Before health insurers reject claims for medical reasons, company doctors must review them, according to insurance laws > A Cigna algorithm flags mismatches between diagnoses and what the company considers acceptable tests and procedures for those ailments Sounds fine, except > “We literally click and submit,” one former Cigna doctor said. “It takes all of 10 seconds to do 50 at a time.” [1]: https://www.propublica.org/article/cigna-pxdx-medical-health-insurance-rejection-claims https://www.propublica.org/article/cigna-pxdx-medical-health...
- hammyhavoc 4y ago
  Have you heard about https://www.theguardian.com/technology/2022/nov/20/part-of-the-kill-chain-how-can-we-control-weaponised-robots https://www.theguardian.com/technology/2022/nov/20/part-of-t... ? And also, how do you feel about already-existing heatseekers? A little different, but the killchain still leans on an automated system for targeting and engagement.
  - michaelmior 4y ago
    Thanks for the link! As far as existing heat-seekers, I would assume that they are generally used in cases where the only target which will likely be locked on to is the desired target. However, this is coming from a place of total ignorance of what the process of target acquisition might look like and what conditions they are used in.
    - hammyhavoc 4y ago
      Heatseekers can be very effective, but as with any automated system, it isn't infallible. Friendly-fire even happens by human beings using small arms, just like strikes on civilian schools and hospitals happen because the intelligence told a human being that it looked like it could be a "terrorist training ground". I'd be interested to see what the failure rate of an AI looked like in what actions it would have taken based on available data versus what actions human took over a several year sample. I have a feeling that the AI will either look terrible, or the human beings will look terrible, or they'll look pretty equal with strange fringe cases where the AI is better than the human, and vice versa. Judgement and authorization are interesting. Something else you might be interested in (which you may already know about) is PAL systems for nuclear weapons: https://en.wikipedia.org/wiki/Permissive_action_link https://en.wikipedia.org/wiki/Permissive_action_link You'll likely be interested in the "two-man rule": https://en.wikipedia.org/wiki/Two-man_rule https://en.wikipedia.org/wiki/Two-man_rule
      - michaelmior 4y ago
        I guess my point was given my limited knowledge, it doesn't seem as though heat-seekers are necessarily any less fallible than humans. I'm not suggesting that "no worse than a human" should be the goal, but I'd say that's the bare minimum.
- throw_nbvc1234 3y ago
  identify a target decide to attack the target implement the attack determine the result of the attack If humans are required for the decision, then humans have an inherent hook to prevent more attacks. Otherwise you run into scenarios like https://www.youtube.com/watch?v=pyMNIFZTQkg https://www.youtube.com/watch?v=pyMNIFZTQkg where humans are incapacitated and AI just continue on like nothing has happened. It makes the failure modes more sane. It's not a catch-all but a good agreeable step to take.
Dma54rhs 4y ago
Mines have always made the decision, dumb "AI" based on weight mainly. People do decide where to put them though.
- alwaysbeconsing 4y ago
  Yes, and they tend to cause a lot of harm to non-combatants.
r00fus 4y ago
When every other part of a military decision chain involves AI, the military that wins is the one who can learn to poison or subvert enemy AI (prompt hijacking, etc). The race is on. Both to refine an AI as well as learn to confound them.
DrBenCarson 4y ago
How do you define "decision?" If I tell the AI "Go kill Joe," is that enough? What if I say to "Go kill the enemy." Do I have to visually confirm? Click a button? Locate Joe myself?

stareatgoats 4y ago

> How would you curtail their use I'm just hoping people and nations might come to their senses. People smarter than me need to figure out how. I'm not going to say that "it is not possible" however, that would be assuming way too much.

hgsgm 4y ago
Guess what happens when 99% of people are sensible and 1% have AI murderbots.
- pmoriarty 4y ago
  Defensive drones will become as common as offensive drones. Unfortunately, both defensive and offensive drones might be hacked to attack their masters. As these drones gain more autonomy, they may become a threat to just about everyone, friend and foe.
  - etiam 4y ago
    > Defensive drones will become as common as offensive drones. Probably not good enough, as an ironclad defense is likely to remain much more difficult than offense which is occasionally successful.
  - OkayPhysicist 4y ago
    >> Unfortunately, both defensive and offensive drones might be hacked to attack their masters. That's an argument FOR greater local autonomy by the robot, rather than less. A bot that gets to decide for itself whether somebody is a friend, target, or neutral 3rd party doesn't need to be phoning home constantly, at risk of interception.
  - heurist 4y ago
    Availability of drones is entirely dependent on access to supply chains for the materials and products that go into those drones. There are some limits here, at least until we're mining the asteroid belt.
4y ago

nix0n 4y ago

> any military that commits to using them will have a huge advantage ? I don't think that's true. If autonomous weapons are deployed, the advantage will go to whoever has the best hackers.

chongli 4y ago
White hat and black hat. If you’ve got the best pentesters then your systems are going to be harder to break into, giving you the advantage.