4 ms·

How can this be worse than a bomb which "decides" to kill anything within a certain radius without any human being individually looking around and making indivi

How can this be worse than a bomb which "decides" to kill anything within a certain radius without any human being individually looking around and making individual decisions about everyone in that radius?

sogen 10y ago
Like Ubik
j-pb 10y ago
The bomb goes off once. Let me reference you to this awesome short film that illustrates the problem nicely. https://vimeo.com/67768281 https://vimeo.com/67768281
- anonymfus 10y ago
  Expected link to this instead: https://www.youtube.com/watch?v=NnJbtbh4tDE https://www.youtube.com/watch?v=NnJbtbh4tDE
tyingq 10y ago
I get your point, but in your example there was some human driven decision to approve dropping the weapon into that specific radius. The article is talking about AI driven scanning, selection, and engagement...with no human driven decision in the process at all. The open letter from a group of AI practitioners that's mentioned in the article sums up the concern well: http://futureoflife.org/open-letter-autonomous-weapons/ http://futureoflife.org/open-letter-autonomous-weapons/
- digi_owl 10y ago
  Didn't the long range Harpoon launches back during the first gulf war do that? Give it a coordinate to hit, some radar map data to find its way with, and send it on its way.
  - tyingq 10y ago
    "Give it a coordinate to hit" Part of the concern with the truly autonomous AI driven weapons is that they could patrol large areas and select their own targets, with whatever arbitrary rules you wanted.
    - digi_owl 10y ago
      Given that we already have incidents of helicopter pilots gunning down news crews because they confused a camera with a RPG, i'm not sure if such a weapon will be better or worse than the status quo.
      - JulianMorrison 10y ago
        Better, almost certainly, in terms of effectiveness, avoiding friendly fire, killing only the target. The elephant in the room is that it may not be a good idea for the US government to be better at killing people in highly targeted ways, without any risk to themselves. It seems to encourage them to do it more. Suppose the USA had a full fledged Star Wars scale drone army of robot ground troops. Do you honestly think it wouldn't be in Syria right now?
        digi_owl 10y ago
        That may indeed be the biggest risk. If the boots on the ground do not bleed red, the people in charge may be more apt at option for the violent solution. This because blown up drones do not look bad on the evening news.
    - Retric 10y ago
      With Mines we say "kill anything that enters this area". With ICBM's, "kill anything over there." If anything AI just adds "kill anything... except this and that."
    - clarry 10y ago
      Ultimately, the rules are set by people. I don't see how a machine executing arbitrary rules set by people is that different from people executing arbitrary orders given by people. Just a week ago, two Belgian fighters allegedly bombed a civilian target in Syria, killing four and and further injuring two innocent people. That's a very arbitrary action to take, yet somebody must've ordered the pilots to do that. Perhaps somebody expected there to be armed terrorists there? If they had sent an autonomous robot, with the rules and training to target actually armed targets (how would a robot tell they're terrorists though?), perhaps these civilians would still be fine.
- JumpCrisscross 10y ago
  To underscore why this is important, consider hybrid warfare. Hybrid and cyber warfare muddy the definition of an attack. Lob some artillery at America and, sure as hell, your tanks will be vaporised. Take out a power plant with malware and...we'll spend months arguing over whether you really did it? Do we bomb you in retaliation? Say mean things on Twitter? When consequences are this unclear, deterrence fails. If a robot autonomously mows down a village, is that a military strike? Does it merit physical retaliation? War, for a time, was "ready, aim, fire". Guerilla tactics changed that. New routes to plausible deniability may be our generation's blitzkrieg.
  - jfoutz 10y ago
    Missed opportunity for a great quote from another James Cameron movie "Private Frost: What the hell are we supposed to use, man, harsh language?" Let's just set aside feasibility. I think AI making tactical decisions about how to execute a goal while being shot at is going to be step 518, and we're on something like step 3. It's not clear to me why an AI executing policy is a whole lot different than a bureaucracy executing policy. For example, I believe that the guys who launch the ICBM's go through pretty extensive psychological testing, to help guarantee they'll actually launch if ordered to do so. Wargames not withstanding, is ordering a robot to turn the key to launch different somehow? Is a robot mowing down a village fundamentally different than a person mowing down a village?
    - tim333 10y ago
      Possibly a robot mowing down a village would be more discriminating as to who it mowed.
    - kwhitefoot 10y ago
      > pretty extensive psychological testing, to help guarantee they'll actually launch if ordered to do so. Yet, thankfully, some in the chain of command in military organizations retain their common sense as shown by the actions of Vasili Arkhipov (https://en.wikipedia.org/wiki/Vasili_Arkhipov https://en.wikipedia.org/wiki/Vasili_Arkhipov) and later Stanislav Petrov (https://en.wikipedia.org/wiki/Stanislav_Petrov https://en.wikipedia.org/wiki/Stanislav_Petrov). Everyone is responsible for their own actions.

sogen 10y ago

Like Ubik

j-pb 10y ago

The bomb goes off once. Let me reference you to this awesome short film that illustrates the problem nicely. https://vimeo.com/67768281 https://vimeo.com/67768281

anonymfus 10y ago
Expected link to this instead: https://www.youtube.com/watch?v=NnJbtbh4tDE https://www.youtube.com/watch?v=NnJbtbh4tDE

tyingq 10y ago

I get your point, but in your example there was some human driven decision to approve dropping the weapon into that specific radius. The article is talking about AI driven scanning, selection, and engagement...with no human driven decision in the process at all. The open letter from a group of AI practitioners that's mentioned in the article sums up the concern well: http://futureoflife.org/open-letter-autonomous-weapons/ http://futureoflife.org/open-letter-autonomous-weapons/

digi_owl 10y ago
Didn't the long range Harpoon launches back during the first gulf war do that? Give it a coordinate to hit, some radar map data to find its way with, and send it on its way.
- tyingq 10y ago
  "Give it a coordinate to hit" Part of the concern with the truly autonomous AI driven weapons is that they could patrol large areas and select their own targets, with whatever arbitrary rules you wanted.
  - digi_owl 10y ago
    Given that we already have incidents of helicopter pilots gunning down news crews because they confused a camera with a RPG, i'm not sure if such a weapon will be better or worse than the status quo.
    - JulianMorrison 10y ago
      Better, almost certainly, in terms of effectiveness, avoiding friendly fire, killing only the target. The elephant in the room is that it may not be a good idea for the US government to be better at killing people in highly targeted ways, without any risk to themselves. It seems to encourage them to do it more. Suppose the USA had a full fledged Star Wars scale drone army of robot ground troops. Do you honestly think it wouldn't be in Syria right now?
      - digi_owl 10y ago
        That may indeed be the biggest risk. If the boots on the ground do not bleed red, the people in charge may be more apt at option for the violent solution. This because blown up drones do not look bad on the evening news.
  - Retric 10y ago
    With Mines we say "kill anything that enters this area". With ICBM's, "kill anything over there." If anything AI just adds "kill anything... except this and that."
  - clarry 10y ago
    Ultimately, the rules are set by people. I don't see how a machine executing arbitrary rules set by people is that different from people executing arbitrary orders given by people. Just a week ago, two Belgian fighters allegedly bombed a civilian target in Syria, killing four and and further injuring two innocent people. That's a very arbitrary action to take, yet somebody must've ordered the pilots to do that. Perhaps somebody expected there to be armed terrorists there? If they had sent an autonomous robot, with the rules and training to target actually armed targets (how would a robot tell they're terrorists though?), perhaps these civilians would still be fine.
JumpCrisscross 10y ago
To underscore why this is important, consider hybrid warfare. Hybrid and cyber warfare muddy the definition of an attack. Lob some artillery at America and, sure as hell, your tanks will be vaporised. Take out a power plant with malware and...we'll spend months arguing over whether you really did it? Do we bomb you in retaliation? Say mean things on Twitter? When consequences are this unclear, deterrence fails. If a robot autonomously mows down a village, is that a military strike? Does it merit physical retaliation? War, for a time, was "ready, aim, fire". Guerilla tactics changed that. New routes to plausible deniability may be our generation's blitzkrieg.
- jfoutz 10y ago
  Missed opportunity for a great quote from another James Cameron movie "Private Frost: What the hell are we supposed to use, man, harsh language?" Let's just set aside feasibility. I think AI making tactical decisions about how to execute a goal while being shot at is going to be step 518, and we're on something like step 3. It's not clear to me why an AI executing policy is a whole lot different than a bureaucracy executing policy. For example, I believe that the guys who launch the ICBM's go through pretty extensive psychological testing, to help guarantee they'll actually launch if ordered to do so. Wargames not withstanding, is ordering a robot to turn the key to launch different somehow? Is a robot mowing down a village fundamentally different than a person mowing down a village?
  - tim333 10y ago
    Possibly a robot mowing down a village would be more discriminating as to who it mowed.
  - kwhitefoot 10y ago
    > pretty extensive psychological testing, to help guarantee they'll actually launch if ordered to do so. Yet, thankfully, some in the chain of command in military organizations retain their common sense as shown by the actions of Vasili Arkhipov (https://en.wikipedia.org/wiki/Vasili_Arkhipov https://en.wikipedia.org/wiki/Vasili_Arkhipov) and later Stanislav Petrov (https://en.wikipedia.org/wiki/Stanislav_Petrov https://en.wikipedia.org/wiki/Stanislav_Petrov). Everyone is responsible for their own actions.