8 ms·

UN delay could open door to robot wars

28 points by spking 11y ago

x5n1 11y ago
Robot wars, or drone strikes, have been going on for at least 20 years perhaps more. And perhaps robots have the ability to even the battlefield. Imagine China sending billions of drones to counteract American drones. Perhaps we should be looking forward to drone war, because anyone can destroy your machines, it's not a big deal for anyone to enter such a war. There will be no diplomatic incident over destroyed robots as a provocation to all out war between countries with real power. The battlefield is pretty much open to anyone with the hardware.
- baconner 11y ago
  It lowers the barrier of entry to use lethal violence and that is not a good thing. A more realistic scenerio is that rich countries will have less reason to seek diplomatic solutions now that the only people doing the dying are the poorer nations they wage war against. Just imagine how many years an all robot military could occupy a nation with no public blowback because of casualties. At that point it's a cost calculation. Just imagine how this might be applied by powers like the US, Russia, Israel, and China. IMO its a recipe for increased use of force and extended violent occupations. And that's not even getting into the issue of autonomous kill decision capabilities.
  - happyscrappy 11y ago
    Yeah, imagine an Israeli robot grabbing the arms of a suicide bomber so he can't detonate, horrible.
    - baconner 11y ago
      These aren't infallible defense only hug-bots we're talking about. They're not going to just grab your arm they are more likely to shoot you and anyone else who happens to be too close to you. Imagine instead summary execution by robot because an algorithm decides that you look enough like a known terrorist or your call metadata seems terroristish and you're in the wrong neighborhood. Imagine the violent collective punishment that can be brought to bear on a population in retaliation for something like a suicide bombing.
      - 13thLetter 11y ago
        Are you saying that human intelligence agencies would never go after innocent people and human military forces would never carry out violent collective punishment against a civilian population? Think carefully... There's this weird blind spot in the robot weapons debate where we shudder in fear of robots committing acts that are already being committed in job lots by humans every day. We don't need machines to be inhumane! The problem is not weapons, the problem is human beings choosing to do bad things, and scapegoating robots will do absolutely nothing to fix that.
        baconner 11y ago
        No that's not what I am saying at all. I'm saying I think they can have the effect of making it easier for a person to kill more people with less risk of consequence and that the lack of consequence makes violence easier to turn to as a solution. I am not afraid of robots, I am afraid of humans granted the ability to shield themselves from consequence for killing.
        13thLetter 11y ago
        And what I'm saying is that people -- real people, not some unnamed "person" from some unspecified country or culture who may exist at some point in the future -- are already turning to violence as a solution, without the need for robots. Bashir Assad doesn't have killer robots and neither does Vladimir Putin and neither do the mullahs of Iran, but somehow they're managing.
        baconner 11y ago
        I don't see the relevance of that point. The existence of violence without a particular weapon does not refute the argument that a new weapon may lead to even more death and suffering. By your logic since violence already exists apparently we shouldn't worry about new weapons making it even worse. Back to the original point of the article it is important that international standards limiting such devastating weapons be enacted and enforced. Even Assad after being caught using chemical weapons was forced to step back and many robotic weapons, especially autonomous ones, should probably be in the same category for the same reason. These things are going to be developed so we need to act now to limit their use not wait until a calamity has struck.
reitanqild 11y ago
FWIW I recently read "Kill Decision" based on a tip here on HN. The plotline of the book is about autonomous drones. In my opinion the book seemed well researched and not too far fetched.
- jonkiddy 11y ago
  That was a terrific book. I highly recommend anything from Daniel Suarez.
  - Ao7bei3s 11y ago
    I found it rather dull, and the entire notion of using pheromones to control multicopters really absurd.
    - reitanqild 11y ago
      As noted above I filed it under "not too far fetched". As I grow older and have spent more time in the it industry less and less stuff files under impossible.
mtgx 11y ago
The biggest danger is the government believing they "solved" autonomous killing machines that kill people based on the NSA's mass spying apparatus, when in fact such robots could have many false positives. In fact, they already do that. It's just a human that pushes the button. When the decision will become automated, such strikes will grow 10 or 100 fold, just like the drone strikes increased "air strikes" by 100-1000x fold in the Middle East, too. It's simple logic really. When it's easy and cheap, they'll just do a lot more of it, and with much more relaxed rules, because every single strike is not a huge deal (to them) anymore, so they can afford to kill "less important" targets, or even "false positives", because there are a lot more strikes where that one came from.
- PavlovsCat 11y ago
  > The biggest danger is the government believing they "solved" autonomous killing machines that kill people based on the NSA's mass spying apparatus, when in fact such robots could have many false positives. False positives only matter in so far as they can generate outrage that can threaten power. So if you find a way to make absolutely sure no powerful person gets killed, which doesn't seem like a hard problem to solve, you can pretty much start shooting fish in a barrel without any adverse consequences. E.g. when Mao had his leaders fulfill quotas of persecuting so-called dissidents, whether those people were really dissidents was rather secondary. What matters is that there is a certain percentage that is unemployed, starving, and/or hunted by killer robots; that in itself does wonders to keep people in line, and the people who aren't directly under attack have a huge capacity to rationalize and ignore things, if that's what it takes to not be attacked. If we can accept people needlessly starving we can accept people being killed by a random number generator no problem. I wish I was being snarky, and I hope for nothing more than to be proven wrong, but absent radical changes, that's what I see in our future. "A boot stamping on a human face, forever" is not a still frame, it's a process, and unless that process is stopped for good, that human face will become infinitely thin and infinitely helpless.
  - jpt4 11y ago
    Off Topic: I see in your profile, PavlovsCat, a link to the RetroShare Sourceforge page. Are you a contributor to the project? I see that they now have an active GitHub repository [0], when I had thought progress had slowed or ceased. [0] https://github.com/RetroShare/RetroShare https://github.com/RetroShare/RetroShare
atemerev 11y ago
Robot wars are so much better than human wars. Against humans, robots will eventually use more non-lethal options (it's easier to be humane if losing a fight means replacing a drone than when it's either your life or your enemy's). Then, robots will be mostly used against other robots, reducing death tolls. Then, perhaps, wars will be decided in friendly Starcraft matches, saving billions. :)
- madaxe_again 11y ago
  Those are some optimistic assumptions. Perhaps the Robots will refuse to fight and will all hold hands and sing kum-bay-ya, eh?
  - egor83 11y ago
    This was the premise of some sci-fi novel I read long ago: two countries at war switched to using robots, and as robo-armies became more interconnected and thus more intellectually powerful, they understood the pointlessness of destruction and refused to fight. IIRC they went to pick flowers on the battlefield instead of singing, though. I thought it was one of Stanislaw Lem's novels, but can't find the name now; I might be wrong about that.
    - teddyh 11y ago
      It was by Stanisław Lem; It’s called The First Sally; or The Trap of Gargantius
      - egor83 11y ago
        Yes, this one exactly - thanks!
      - kwhitefoot 11y ago
        Lem wrote some wonderfully whimsical stories; all the more surprising when you consider the times and place in which he wrote. For another perspective on autonomous drones read Robert Sheckley's Watchbird.
xixi77 11y ago
IMO it's not obvious that autonomous robots would be worse at things like discriminating between civilian and military targets than humans -- particularly when said humans are in a life-or-death situation that requires quick decisions. The way the article describes it, it looks more like an attempt by the parties currently lacking technology to level the playing field through diplomatic means, rather than a mutually beneficial arrangement.
13thLetter 11y ago
It's hard not to think that a lot of the theatrical upset about "killer robots" is because these are weapons mostly deployed by the United States and a few other Western nations against opponents who don't have a real military defense against them. When you can't respond with warfare, there's always lawfare. (Cynical, perhaps, but if you watch the UN human rights council for even five minutes cynicism seems quite appropriate.)
georgeecollins 11y ago
The biggest threat to automous weapons is the millitary organizations that want to keep manpower central. That's why drones were pioneered by the CIA and the Navy and not the US Air Force. They needed solutions, not pilots.

28 points by spking 11y ago

x5n1 11y ago

Robot wars, or drone strikes, have been going on for at least 20 years perhaps more. And perhaps robots have the ability to even the battlefield. Imagine China sending billions of drones to counteract American drones. Perhaps we should be looking forward to drone war, because anyone can destroy your machines, it's not a big deal for anyone to enter such a war. There will be no diplomatic incident over destroyed robots as a provocation to all out war between countries with real power. The battlefield is pretty much open to anyone with the hardware.

baconner 11y ago
It lowers the barrier of entry to use lethal violence and that is not a good thing. A more realistic scenerio is that rich countries will have less reason to seek diplomatic solutions now that the only people doing the dying are the poorer nations they wage war against. Just imagine how many years an all robot military could occupy a nation with no public blowback because of casualties. At that point it's a cost calculation. Just imagine how this might be applied by powers like the US, Russia, Israel, and China. IMO its a recipe for increased use of force and extended violent occupations. And that's not even getting into the issue of autonomous kill decision capabilities.
- happyscrappy 11y ago
  Yeah, imagine an Israeli robot grabbing the arms of a suicide bomber so he can't detonate, horrible.
  - baconner 11y ago
    These aren't infallible defense only hug-bots we're talking about. They're not going to just grab your arm they are more likely to shoot you and anyone else who happens to be too close to you. Imagine instead summary execution by robot because an algorithm decides that you look enough like a known terrorist or your call metadata seems terroristish and you're in the wrong neighborhood. Imagine the violent collective punishment that can be brought to bear on a population in retaliation for something like a suicide bombing.
    - 13thLetter 11y ago
      Are you saying that human intelligence agencies would never go after innocent people and human military forces would never carry out violent collective punishment against a civilian population? Think carefully... There's this weird blind spot in the robot weapons debate where we shudder in fear of robots committing acts that are already being committed in job lots by humans every day. We don't need machines to be inhumane! The problem is not weapons, the problem is human beings choosing to do bad things, and scapegoating robots will do absolutely nothing to fix that.
      - baconner 11y ago
        No that's not what I am saying at all. I'm saying I think they can have the effect of making it easier for a person to kill more people with less risk of consequence and that the lack of consequence makes violence easier to turn to as a solution. I am not afraid of robots, I am afraid of humans granted the ability to shield themselves from consequence for killing.
        13thLetter 11y ago
        And what I'm saying is that people -- real people, not some unnamed "person" from some unspecified country or culture who may exist at some point in the future -- are already turning to violence as a solution, without the need for robots. Bashir Assad doesn't have killer robots and neither does Vladimir Putin and neither do the mullahs of Iran, but somehow they're managing.
        baconner 11y ago
        I don't see the relevance of that point. The existence of violence without a particular weapon does not refute the argument that a new weapon may lead to even more death and suffering. By your logic since violence already exists apparently we shouldn't worry about new weapons making it even worse. Back to the original point of the article it is important that international standards limiting such devastating weapons be enacted and enforced. Even Assad after being caught using chemical weapons was forced to step back and many robotic weapons, especially autonomous ones, should probably be in the same category for the same reason. These things are going to be developed so we need to act now to limit their use not wait until a calamity has struck.

reitanqild 11y ago

FWIW I recently read "Kill Decision" based on a tip here on HN. The plotline of the book is about autonomous drones. In my opinion the book seemed well researched and not too far fetched.

jonkiddy 11y ago
That was a terrific book. I highly recommend anything from Daniel Suarez.
- Ao7bei3s 11y ago
  I found it rather dull, and the entire notion of using pheromones to control multicopters really absurd.
  - reitanqild 11y ago
    As noted above I filed it under "not too far fetched". As I grow older and have spent more time in the it industry less and less stuff files under impossible.

mtgx 11y ago

The biggest danger is the government believing they "solved" autonomous killing machines that kill people based on the NSA's mass spying apparatus, when in fact such robots could have many false positives. In fact, they already do that. It's just a human that pushes the button. When the decision will become automated, such strikes will grow 10 or 100 fold, just like the drone strikes increased "air strikes" by 100-1000x fold in the Middle East, too. It's simple logic really. When it's easy and cheap, they'll just do a lot more of it, and with much more relaxed rules, because every single strike is not a huge deal (to them) anymore, so they can afford to kill "less important" targets, or even "false positives", because there are a lot more strikes where that one came from.

PavlovsCat 11y ago
> The biggest danger is the government believing they "solved" autonomous killing machines that kill people based on the NSA's mass spying apparatus, when in fact such robots could have many false positives. False positives only matter in so far as they can generate outrage that can threaten power. So if you find a way to make absolutely sure no powerful person gets killed, which doesn't seem like a hard problem to solve, you can pretty much start shooting fish in a barrel without any adverse consequences. E.g. when Mao had his leaders fulfill quotas of persecuting so-called dissidents, whether those people were really dissidents was rather secondary. What matters is that there is a certain percentage that is unemployed, starving, and/or hunted by killer robots; that in itself does wonders to keep people in line, and the people who aren't directly under attack have a huge capacity to rationalize and ignore things, if that's what it takes to not be attacked. If we can accept people needlessly starving we can accept people being killed by a random number generator no problem. I wish I was being snarky, and I hope for nothing more than to be proven wrong, but absent radical changes, that's what I see in our future. "A boot stamping on a human face, forever" is not a still frame, it's a process, and unless that process is stopped for good, that human face will become infinitely thin and infinitely helpless.
- jpt4 11y ago
  Off Topic: I see in your profile, PavlovsCat, a link to the RetroShare Sourceforge page. Are you a contributor to the project? I see that they now have an active GitHub repository [0], when I had thought progress had slowed or ceased. [0] https://github.com/RetroShare/RetroShare https://github.com/RetroShare/RetroShare

atemerev 11y ago

Robot wars are so much better than human wars. Against humans, robots will eventually use more non-lethal options (it's easier to be humane if losing a fight means replacing a drone than when it's either your life or your enemy's). Then, robots will be mostly used against other robots, reducing death tolls. Then, perhaps, wars will be decided in friendly Starcraft matches, saving billions. :)

madaxe_again 11y ago
Those are some optimistic assumptions. Perhaps the Robots will refuse to fight and will all hold hands and sing kum-bay-ya, eh?
- egor83 11y ago
  This was the premise of some sci-fi novel I read long ago: two countries at war switched to using robots, and as robo-armies became more interconnected and thus more intellectually powerful, they understood the pointlessness of destruction and refused to fight. IIRC they went to pick flowers on the battlefield instead of singing, though. I thought it was one of Stanislaw Lem's novels, but can't find the name now; I might be wrong about that.
  - teddyh 11y ago
    It was by Stanisław Lem; It’s called The First Sally; or The Trap of Gargantius
    - egor83 11y ago
      Yes, this one exactly - thanks!
    - kwhitefoot 11y ago
      Lem wrote some wonderfully whimsical stories; all the more surprising when you consider the times and place in which he wrote. For another perspective on autonomous drones read Robert Sheckley's Watchbird.

xixi77 11y ago

IMO it's not obvious that autonomous robots would be worse at things like discriminating between civilian and military targets than humans -- particularly when said humans are in a life-or-death situation that requires quick decisions. The way the article describes it, it looks more like an attempt by the parties currently lacking technology to level the playing field through diplomatic means, rather than a mutually beneficial arrangement.

13thLetter 11y ago

It's hard not to think that a lot of the theatrical upset about "killer robots" is because these are weapons mostly deployed by the United States and a few other Western nations against opponents who don't have a real military defense against them. When you can't respond with warfare, there's always lawfare. (Cynical, perhaps, but if you watch the UN human rights council for even five minutes cynicism seems quite appropriate.)

georgeecollins 11y ago

The biggest threat to automous weapons is the millitary organizations that want to keep manpower central. That's why drones were pioneered by the CIA and the Navy and not the US Air Force. They needed solutions, not pilots.