5 ms·

I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surre

I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surrendered, etc... before engaging and killing them. The alternative is indiscriminate death that we see in mass bombings, mine fields, artillery strikes, and drone strikes.

pmelendez 11y ago
> I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surrendered, etc... The problem is that those intelligent systems will be overlorded by the same people overlording the actual military. At least in our current situation a human has to pull the trigger and that person could be potentially charged for crimes against humanity which would make it thinking it twice and potentially resist to follow a direct order. You won't have that with a machine.
- fauigerzigerk 11y ago
  That's true, but the scenario you are thinking of is one in which the higher ups order a massacre and the soldier on the ground refuses to carry out the order. I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions, quite possibly panicking themselves. If they get it wrong, they're dead. An AI can make split second decisions without panicking and without any consideration for its own "survival". For any mistakes an AI makes, those higher up will get the blame (ideally), not some 20 year old who was scared for his own life when he made the decision to kill everyone on that overloaded truck because he it wouldn't stop 10 seconds earlier. That said, I shudder at the thought of a world in which people get killed by machines who will never be whistleblowers, who do not go home with post traumatic stress disorder, telling everyone how horrible war really is.
  - pmelendez 11y ago
    > I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions Tha't ok, but I think you are assuming that the "good guys" are the only ones that would have access to this. Truth is that once one government start doing this, all the others will follow and even the organized crime would have access to that technology.
    - fauigerzigerk 11y ago
      Even if both sides use machines, that doesn't make it any worse than both sides using humans, on the contrary. But as I said, the wider consequences are a different matter.
      - pmelendez 11y ago
        I think I didn't explain myself clearly... What I meant is that once one side has it, it will go outside of common warfare pretty quickly. You are thinking in Gov. A vs. Gov. B both using machines. My concern is that after that, Gov. B will use machines against humans C inside the same country, or in the neightborhood. And in those cases, "technical errors" can be used as excuse after a tragedy. And that's only one of my concerns... add organized crime and terrorism to the mix and you will have a very explosive soup.
        fauigerzigerk 11y ago
        I think your assumptions are very realistic and I share your concerns. But looking back at the history of war or combat, I don't feel that human nature has been a mitigating factor. On the contrary.
MCRed 11y ago
Human drone pilots, and gunners in manned aircraft are much better at making such assessments than any current technology-- yet they do not do so, and kill people who really it appears shouldn't be killed, including those you mention. If we can make humans do it, I suspect whatever is controlling future craft would do it too, especially if programming is much easier than it is to program humans to be murderers.
emidln 11y ago
Consider it as an optimization problem: (a) kill unfriendlies (b) save neutral lives (c) save friendlies (d) keep minimal costs (e) thoroughly destroy opponents means of warfare Given that the people setting priorities already don't keep (b) or (e) as it relates to civilian property in mind, how likely is a system to come into existence trained to consider (b) or (e) highly enough to say, not make the assertion that any amount of foreign destruction is okay if it is likely to minimize costs in the next 3 months? (i.e. We won't have to sweep that building if it's leveled.) Another alternative to indiscriminate death is calculated efficiency driven by algorithms tuned to save US money, kill their guys, save our guys, and if possible while doing the other three, don't kill neutrals, friendlies, or neutral property.
ilaksh 11y ago
Warfare is institutionalized criminal activity backed by outdated belief systems and social technologies. Technologists have a moral obligation to educate themselves about social and political issues and prevent criminals from obtaining weapons.
- fauigerzigerk 11y ago
  How do you suggest we prevent any technology we create from getting used for everything it is suitable for? We'd have to start by not publishing any scientific results and never open source anything. Then we'd have to prevent industrial espionage as well as democratically elected law makers forcing us to reveal our secrets. It's different than with nuclear technology because AI is so versatile and so cheap. And to wrap it all up, we'd have to stand by and watch as a medieval death cult like Islamic State overruns an entire region. War is very rarely the right solution. Arguably, wars had a large part in creating IS in the first place, and I was very much against most recent wars. But for reasonably enlightened democratic nations not to have the capability to win a war if need be, ultimately means to be at the mercy of any insane barbaric ideology or religion that comes along.
  - pharrington 11y ago
    I think you're misunderstanding ilaksh's point. Yes, knowledge is social potential energy. The potential energy lacks the moral quality until it is implemented for a purpose by a person in society. I, as the person who knows how to build the killer robot, have a moral obligation to not build the killer robot. The criminal in this scenario is the government.
    - fauigerzigerk 11y ago
      I don't think this is a valid argument when it comes to AI. Sure you can say I'm not going to be the one who builds that missile or that gun. But a killer robot is but a thin layer on top of a general AI or on top of a variety of specialised AI functions. The more general the AI becomes, the more "dual use" it is necessarily going to be. In other words, should we not build intelligent robots because someone might give them a gun to make them killer robots?
lawlessone 11y ago
"before engaging and killing them." Do they have to be killed though? If we have machine soldiers and the enemy combatants aren't an immediate threat to actual humans shouldn't we be trying to detain them instead?

pmelendez 11y ago

> I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surrendered, etc... The problem is that those intelligent systems will be overlorded by the same people overlording the actual military. At least in our current situation a human has to pull the trigger and that person could be potentially charged for crimes against humanity which would make it thinking it twice and potentially resist to follow a direct order. You won't have that with a machine.

fauigerzigerk 11y ago
That's true, but the scenario you are thinking of is one in which the higher ups order a massacre and the soldier on the ground refuses to carry out the order. I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions, quite possibly panicking themselves. If they get it wrong, they're dead. An AI can make split second decisions without panicking and without any consideration for its own "survival". For any mistakes an AI makes, those higher up will get the blame (ideally), not some 20 year old who was scared for his own life when he made the decision to kill everyone on that overloaded truck because he it wouldn't stop 10 seconds earlier. That said, I shudder at the thought of a world in which people get killed by machines who will never be whistleblowers, who do not go home with post traumatic stress disorder, telling everyone how horrible war really is.
- pmelendez 11y ago
  > I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions Tha't ok, but I think you are assuming that the "good guys" are the only ones that would have access to this. Truth is that once one government start doing this, all the others will follow and even the organized crime would have access to that technology.
  - fauigerzigerk 11y ago
    Even if both sides use machines, that doesn't make it any worse than both sides using humans, on the contrary. But as I said, the wider consequences are a different matter.
    - pmelendez 11y ago
      I think I didn't explain myself clearly... What I meant is that once one side has it, it will go outside of common warfare pretty quickly. You are thinking in Gov. A vs. Gov. B both using machines. My concern is that after that, Gov. B will use machines against humans C inside the same country, or in the neightborhood. And in those cases, "technical errors" can be used as excuse after a tragedy. And that's only one of my concerns... add organized crime and terrorism to the mix and you will have a very explosive soup.
      - fauigerzigerk 11y ago
        I think your assumptions are very realistic and I share your concerns. But looking back at the history of war or combat, I don't feel that human nature has been a mitigating factor. On the contrary.

MCRed 11y ago

Human drone pilots, and gunners in manned aircraft are much better at making such assessments than any current technology-- yet they do not do so, and kill people who really it appears shouldn't be killed, including those you mention. If we can make humans do it, I suspect whatever is controlling future craft would do it too, especially if programming is much easier than it is to program humans to be murderers.

emidln 11y ago

Consider it as an optimization problem: (a) kill unfriendlies (b) save neutral lives (c) save friendlies (d) keep minimal costs (e) thoroughly destroy opponents means of warfare Given that the people setting priorities already don't keep (b) or (e) as it relates to civilian property in mind, how likely is a system to come into existence trained to consider (b) or (e) highly enough to say, not make the assertion that any amount of foreign destruction is okay if it is likely to minimize costs in the next 3 months? (i.e. We won't have to sweep that building if it's leveled.) Another alternative to indiscriminate death is calculated efficiency driven by algorithms tuned to save US money, kill their guys, save our guys, and if possible while doing the other three, don't kill neutrals, friendlies, or neutral property.

ilaksh 11y ago

Warfare is institutionalized criminal activity backed by outdated belief systems and social technologies. Technologists have a moral obligation to educate themselves about social and political issues and prevent criminals from obtaining weapons.

fauigerzigerk 11y ago
How do you suggest we prevent any technology we create from getting used for everything it is suitable for? We'd have to start by not publishing any scientific results and never open source anything. Then we'd have to prevent industrial espionage as well as democratically elected law makers forcing us to reveal our secrets. It's different than with nuclear technology because AI is so versatile and so cheap. And to wrap it all up, we'd have to stand by and watch as a medieval death cult like Islamic State overruns an entire region. War is very rarely the right solution. Arguably, wars had a large part in creating IS in the first place, and I was very much against most recent wars. But for reasonably enlightened democratic nations not to have the capability to win a war if need be, ultimately means to be at the mercy of any insane barbaric ideology or religion that comes along.
- pharrington 11y ago
  I think you're misunderstanding ilaksh's point. Yes, knowledge is social potential energy. The potential energy lacks the moral quality until it is implemented for a purpose by a person in society. I, as the person who knows how to build the killer robot, have a moral obligation to not build the killer robot. The criminal in this scenario is the government.
  - fauigerzigerk 11y ago
    I don't think this is a valid argument when it comes to AI. Sure you can say I'm not going to be the one who builds that missile or that gun. But a killer robot is but a thin layer on top of a general AI or on top of a variety of specialised AI functions. The more general the AI becomes, the more "dual use" it is necessarily going to be. In other words, should we not build intelligent robots because someone might give them a gun to make them killer robots?

lawlessone 11y ago

"before engaging and killing them." Do they have to be killed though? If we have machine soldiers and the enemy combatants aren't an immediate threat to actual humans shouldn't we be trying to detain them instead?