5 ms·

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

singlow 10y ago
Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.
- gbhn 10y ago
  True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button. In Nuremberg we developed a way to think about this: people outside the central circles of power have a tremendous amount of pressures they are considering. It's definitely the case that some are sadistic and horrible, but more are just following orders and trying to get by as best they can, and punishing them for war crimes is not appropriate. The other side of that coin is that it isn't realistic to expect soldiery en masse to resist illegal orders. It's always more complicated than that.
  - paganel 10y ago
    > Somehow there's always a soldier who will push the button. And then there's always that soldier ready to denounce his camarads for having raped some poor Vietnamese women who had nothing to do with the war itself. Why risk such a PR disaster which might see your funding cut when you can use robots instead? Warzone robots don't snitch on their fellow robots in front of the press and they don't rape, they're only build to kill.
  - MaysonL 10y ago
    > True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button. Actually, the unwillingness of the Red Army to do any more invasions was arguably the precipitating factor in the fall of the Soviet empire. In 1988, Gorbachev gave a speech to the Warsaw pact meeting where he told them that the Brezhnev doctrine was no more. No socialist government would be able any longer to count on Russian aid in putting down popular uprisings. A year later, the empire crumbled. Of course, it wasn't the soldiers per se who refused, but the generals who were afraid of the potential mutinies (and the occasional actual ones).
    - idlewords 10y ago
      It was Gorbachev who refused, not the generals.
      - MaysonL 10y ago
        He couldn't have refused without the support of the generals, who had just withdrawn from Afghanistan.
        Nevermark 10y ago
        Very true. And economic problems were a big incentive for everyone in charge to step back from wars. For country's with a strong economy there would be little incentive to stop, especially if AI makes war cheaper.
  - sweetjesus 10y ago
    Unless you're talking about the Nuremberg Rallies, you've got your history screwed up: "just following orders" is NOT a defense against war crime accusations, and punishing those who commit war crimes "trying to get by as best they can" is completely appropriate, was the conclusion of the Nuremberg Trials. https://en.wikipedia.org/wiki/Nuremberg_Principles#Principle_IV https://en.wikipedia.org/wiki/Nuremberg_Principles#Principle...
- zardo 10y ago
  >There is a limit to how far a soldier will go, ethically. What horrible things to you have in mind that armies have not already done?
  - modeless 10y ago
    It's not that AIs will do worse things than humans have already done. It's that AIs will do those terrible things much more efficiently and with much lower risk to the people in charge.
    - Ygg2 10y ago
      On the plus side - no more rape.
      - Senji 10y ago
        Don't be so sure. I'm sure there will be rape drones soon.
      - keldaris 10y ago
        I don't think you can hope for even that much. Rape has historically been systematically used for terrorizing the population in order to achieve military and/or political aims. An AI free from any ethical concerns could conceivably evaluate it as an efficient strategy for achieving some set of goals and proceed accordingly.
    - deleted 10y ago
      [deleted]
- nradov 10y ago
  Really? https://en.wikipedia.org/wiki/Nanking_Massacre https://en.wikipedia.org/wiki/Nanking_Massacre
  - vkreso 10y ago
    As horrific as Nanking Massacre was, I believe that doesn't really prove your point. I'd argue that a lot of counter examples of soldiers ethical behaviour simply aren't visible and are forgotten and lost to history (https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias)
- Spooky23 10y ago
  No there isn't. It's easy enough to manipulate the individual to do anything. We speak of Nazis frequnetly, but consider what Curtis LeMay did in war and prepared to do after the war. AI and robots give plausible deniability and reduce the number of witnesses. They also make suicide raids more practical.
- kmonsen 10y ago
  Yes, almost all revolutions succeed because the army turns. It doesn't happen all the time, but some of the time is enough to put some checks on people wanting to take control.
- erikbye 10y ago
  Which limit are you talking about? I guess you have forgotten what the soldiers of the third reich did. Or American death squads in Afghanistan. Might as well call them rape and death squads. What about Guantanamo? Srebrenica? Should I go on? How unenlightened and naive are you?
  - sctb 10y ago
    Please comment civilly and substantively—without personal attacks—or not at all. https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
zdean 10y ago
I think they're largely limited by the greater public's threshold for acceptable casualties of their own soldiers during war. What the AI'ing of war does is to remove that natural limit and raise the amount of death and destruction a military can inflict without pushback from the general population.
- nindalf 10y ago
  I think you're _exactly right_. This is precisely the reason the Iraq and Afghanistan wars proved too costly to prosecute was that it proved too costly in men and materials. The logistics and cost of procuring and moving weapons, vehicles, living supplies etc was high, but manageable. The cost of each body bag which had to be explained to the public was not. Which is why President Obama pivoted to a drone war in areas like Pakistan. Scores of Pakistani civilians are killed on a regular basis for crimes no worse than standing next to a tall bearded man or attending the funeral of a neighbour. The American public has no issues with this because hey, it's cheap to deploy the drones and no American lives are ever in jeopardy. And to be fair about it, apart from the high collateral damage, the Drone War in Pakistan is generally considered to be successful at inhibiting the Pakistani Taliban. This Drone War was the first successful war that we've seen fought without boots on the ground. It's likely that we'll see many more like this as on-board AI improves.
  - wruza 10y ago
    To make big wars like that, you have to have many great robots, not AIs. If ground robots were great at war in a wide sense, we could use it today via remote control / VR interface.
PavlovsCat 10y ago
That's like asking whether we really have to worry about global wars over resources since there are already knife stabbings and car crashes, or whether the Nazis were really that much different from having a bit of a cold. Sadly, being serious about this subject is not one of the strengths of HN it seems, I was kind of spooked when I only got 1 upvote for this: https://news.ycombinator.com/item?id=11685995 https://news.ycombinator.com/item?id=11685995
hackuser 10y ago
> Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"? Currently there's a high cost to it. War is very expensive; even the richest country in the history of the world, the US, doesn't want to bear the costs. And it requires persuading masses of soldiers to go along with it. Roboticized warfare might eliminate those constraints.
simonh 10y ago
No, because their power is limited by their role in society and the organization in which they operate. Even the president of the United States has limited powers. If Obama just decided one day that the best course of action was to nuke Moscow and doubled down on doing so it's extremely unlikely he'd be able to do so. There are enough other people with careers, jobs, pension plans and common decency between him and actually launching missiles that I don't think it's credible that it would happen on a whim like that, even a persistent one. Someone would call a doctor and get the President some medication. However, that depends on the people between the president and nuclear launch being decent human beings that care about law and order and proper procedure. You need to look at the system, not just individuals. Alternatively, let's say someone came to power who genuinely believed nuking Russia was a good option. Rather than order a launch on the spot, what they'd actually do is gradually build a case, appoint pliable or similarly thinking people to key positions, get the launch protocols revised, engineer a geopolitical crisis by provoking Russia and drive events towards a situation in which a nuclear launch seems like a legitimate option.
- fergie 10y ago
  This is a really strong argument for big government. The smaller our all-powerful military command is, the easier it is for it to go rogue.
  - varjag 10y ago
    It's rather an argument for a crafted system of checks and balances than a big government per se. The totalitarian governments, for example, are absolutely massive, and many don't even have some sort of politburo to reason in a dictator.
  - VexXtreme 10y ago
    A counterargument would be that it's harder for a big government to change direction easily, once decided and set in motion. Wrong decisions can compound because it's easier for a large organization to stay its course, even in the face of increasing harm, as evident by the Vietnam war.
- 10y ago
rocqua 10y ago
There isn't a whole lot of deniability. Current drones strikes involve explosions and craters.

singlow 10y ago

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

gbhn 10y ago
True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button. In Nuremberg we developed a way to think about this: people outside the central circles of power have a tremendous amount of pressures they are considering. It's definitely the case that some are sadistic and horrible, but more are just following orders and trying to get by as best they can, and punishing them for war crimes is not appropriate. The other side of that coin is that it isn't realistic to expect soldiery en masse to resist illegal orders. It's always more complicated than that.
- paganel 10y ago
  > Somehow there's always a soldier who will push the button. And then there's always that soldier ready to denounce his camarads for having raped some poor Vietnamese women who had nothing to do with the war itself. Why risk such a PR disaster which might see your funding cut when you can use robots instead? Warzone robots don't snitch on their fellow robots in front of the press and they don't rape, they're only build to kill.
- MaysonL 10y ago
  > True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button. Actually, the unwillingness of the Red Army to do any more invasions was arguably the precipitating factor in the fall of the Soviet empire. In 1988, Gorbachev gave a speech to the Warsaw pact meeting where he told them that the Brezhnev doctrine was no more. No socialist government would be able any longer to count on Russian aid in putting down popular uprisings. A year later, the empire crumbled. Of course, it wasn't the soldiers per se who refused, but the generals who were afraid of the potential mutinies (and the occasional actual ones).
  - idlewords 10y ago
    It was Gorbachev who refused, not the generals.
    - MaysonL 10y ago
      He couldn't have refused without the support of the generals, who had just withdrawn from Afghanistan.
      - Nevermark 10y ago
        Very true. And economic problems were a big incentive for everyone in charge to step back from wars. For country's with a strong economy there would be little incentive to stop, especially if AI makes war cheaper.
- sweetjesus 10y ago
  Unless you're talking about the Nuremberg Rallies, you've got your history screwed up: "just following orders" is NOT a defense against war crime accusations, and punishing those who commit war crimes "trying to get by as best they can" is completely appropriate, was the conclusion of the Nuremberg Trials. https://en.wikipedia.org/wiki/Nuremberg_Principles#Principle_IV https://en.wikipedia.org/wiki/Nuremberg_Principles#Principle...
zardo 10y ago
>There is a limit to how far a soldier will go, ethically. What horrible things to you have in mind that armies have not already done?
- modeless 10y ago
  It's not that AIs will do worse things than humans have already done. It's that AIs will do those terrible things much more efficiently and with much lower risk to the people in charge.
  - Ygg2 10y ago
    On the plus side - no more rape.
    - Senji 10y ago
      Don't be so sure. I'm sure there will be rape drones soon.
    - keldaris 10y ago
      I don't think you can hope for even that much. Rape has historically been systematically used for terrorizing the population in order to achieve military and/or political aims. An AI free from any ethical concerns could conceivably evaluate it as an efficient strategy for achieving some set of goals and proceed accordingly.
  - deleted 10y ago
    [deleted]
nradov 10y ago
Really? https://en.wikipedia.org/wiki/Nanking_Massacre https://en.wikipedia.org/wiki/Nanking_Massacre
- vkreso 10y ago
  As horrific as Nanking Massacre was, I believe that doesn't really prove your point. I'd argue that a lot of counter examples of soldiers ethical behaviour simply aren't visible and are forgotten and lost to history (https://en.wikipedia.org/wiki/Survivorship_bias https://en.wikipedia.org/wiki/Survivorship_bias)
Spooky23 10y ago
No there isn't. It's easy enough to manipulate the individual to do anything. We speak of Nazis frequnetly, but consider what Curtis LeMay did in war and prepared to do after the war. AI and robots give plausible deniability and reduce the number of witnesses. They also make suicide raids more practical.
kmonsen 10y ago
Yes, almost all revolutions succeed because the army turns. It doesn't happen all the time, but some of the time is enough to put some checks on people wanting to take control.
erikbye 10y ago
Which limit are you talking about? I guess you have forgotten what the soldiers of the third reich did. Or American death squads in Afghanistan. Might as well call them rape and death squads. What about Guantanamo? Srebrenica? Should I go on? How unenlightened and naive are you?
- sctb 10y ago
  Please comment civilly and substantively—without personal attacks—or not at all. https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newswelcome.html https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

zdean 10y ago

I think they're largely limited by the greater public's threshold for acceptable casualties of their own soldiers during war. What the AI'ing of war does is to remove that natural limit and raise the amount of death and destruction a military can inflict without pushback from the general population.

nindalf 10y ago
I think you're _exactly right_. This is precisely the reason the Iraq and Afghanistan wars proved too costly to prosecute was that it proved too costly in men and materials. The logistics and cost of procuring and moving weapons, vehicles, living supplies etc was high, but manageable. The cost of each body bag which had to be explained to the public was not. Which is why President Obama pivoted to a drone war in areas like Pakistan. Scores of Pakistani civilians are killed on a regular basis for crimes no worse than standing next to a tall bearded man or attending the funeral of a neighbour. The American public has no issues with this because hey, it's cheap to deploy the drones and no American lives are ever in jeopardy. And to be fair about it, apart from the high collateral damage, the Drone War in Pakistan is generally considered to be successful at inhibiting the Pakistani Taliban. This Drone War was the first successful war that we've seen fought without boots on the ground. It's likely that we'll see many more like this as on-board AI improves.
- wruza 10y ago
  To make big wars like that, you have to have many great robots, not AIs. If ground robots were great at war in a wide sense, we could use it today via remote control / VR interface.

PavlovsCat 10y ago

That's like asking whether we really have to worry about global wars over resources since there are already knife stabbings and car crashes, or whether the Nazis were really that much different from having a bit of a cold. Sadly, being serious about this subject is not one of the strengths of HN it seems, I was kind of spooked when I only got 1 upvote for this: https://news.ycombinator.com/item?id=11685995 https://news.ycombinator.com/item?id=11685995

hackuser 10y ago

> Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"? Currently there's a high cost to it. War is very expensive; even the richest country in the history of the world, the US, doesn't want to bear the costs. And it requires persuading masses of soldiers to go along with it. Roboticized warfare might eliminate those constraints.

simonh 10y ago

No, because their power is limited by their role in society and the organization in which they operate. Even the president of the United States has limited powers. If Obama just decided one day that the best course of action was to nuke Moscow and doubled down on doing so it's extremely unlikely he'd be able to do so. There are enough other people with careers, jobs, pension plans and common decency between him and actually launching missiles that I don't think it's credible that it would happen on a whim like that, even a persistent one. Someone would call a doctor and get the President some medication. However, that depends on the people between the president and nuclear launch being decent human beings that care about law and order and proper procedure. You need to look at the system, not just individuals. Alternatively, let's say someone came to power who genuinely believed nuking Russia was a good option. Rather than order a launch on the spot, what they'd actually do is gradually build a case, appoint pliable or similarly thinking people to key positions, get the launch protocols revised, engineer a geopolitical crisis by provoking Russia and drive events towards a situation in which a nuclear launch seems like a legitimate option.

fergie 10y ago
This is a really strong argument for big government. The smaller our all-powerful military command is, the easier it is for it to go rogue.
- varjag 10y ago
  It's rather an argument for a crafted system of checks and balances than a big government per se. The totalitarian governments, for example, are absolutely massive, and many don't even have some sort of politburo to reason in a dictator.
- VexXtreme 10y ago
  A counterargument would be that it's harder for a big government to change direction easily, once decided and set in motion. Wrong decisions can compound because it's easier for a large organization to stay its course, even in the face of increasing harm, as evident by the Vietnam war.
10y ago

rocqua 10y ago

There isn't a whole lot of deniability. Current drones strikes involve explosions and craters.