12 ms·

1,000 tech experts warn against AI arms race

74 points by callum85 11y ago

dkx 11y ago
I fear the AI arms race may be inevitable. Even if all the nations could agree to place limits on AI research there will always be a huge incentive to develop something in secret.
callum85 11y ago
I don't see how a ban could work. We already have technology that can identify targets, so the issue is whether a human is involved to approve each kill decision. But what constitutes human approval? Is it enough for a human to sit and watch target details flashing up on a screen, and intervene if they see an incorrect target? What exactly would be banned?
- pjc50 11y ago
  This is something of a problem. The system which scans mobile phone metadata (among other things) and turns it into drone bombing targets is already enough like the latter. A human is involved in pulling the trigger, but really they're following orders rather than having made the target/not target decision. And they may not even have the security clearance for the information on which the decision is based.
  - bitJericho 11y ago
    I don't think that's so much a problem. The problem is literally robots pulling the trigger. What happens when the president is targeted due to error/hacking? A member of the military would have a major problem with assassination of her own leader; a robot may not care. What happens if everybody becomes a target? The military personnel would figure things out real quick, a robot, again, may not care.
    - TheOtherHobbes 11y ago
      A member of the military might not have any problem at all with the "accidental" assassination of their leader, whether that leader is the president or the company sergeant. AI could easily provide plausible deniability. I've been very critical of claims of the dangers of AI, but this isn't about AI - this is about giving lethal weapons to autonomous systems. And that's just stupid. All software has bugs, and even if you don't have a problem with autonomous killing systems - I certainly do - it's incredibly irresponsible to build systems that could kill just because someone left out a semicolon or couldn't convert between miles and km. It's taking the "blue screen of death" experience just a little too literally.
      - bryondowd 11y ago
        Umm, we already have machines that can kill people if someone "left out a semicolon". Airplane autopilots, medical equipment, car computers, etc. Not saying this makes it OK, but that isn't the reason. Life-critical code isn't a new thing.
        jaawn 11y ago
        And those systems all have humans operating in conjunction with them, none of those things are (currently) autonomous. The success thus far of life-critical code usually depends partially on the assistance of humans.
        bryondowd 11y ago
        Most of those systems operate on their own unless/until they hit something they can't handle and a human is standing by as failover. So, unless you're saying it's OK to have deathbots as long as a human is on hand as failover if the deathbot gets confused, I think my point stands.
        TheOtherHobbes 11y ago
        IMO, the current state of the F-35 rather suggests it doesn't.
    - pjc50 11y ago
      A member of the military would have a major problem with assassination of her own leader https://en.wikipedia.org/wiki/List_of_coups_d%27%C3%A9tat_and_coup_attempts https://en.wikipedia.org/wiki/List_of_coups_d%27%C3%A9tat_an... Usually if a leader is assassinated, the assassin is a member of the country's military. The US is comparatively rare in having had no coups since independence and only one civil war. The US also has very strong indoctrination against this kind of thing, but individuals are just as vulnerable as robots to making the wrong decision if they are fed false information. Such as endless attacks on the legitimacy of the president's election, birth certificate, allegations of being a foreign agent, etc. The US is of course not above assassinating foreign leaders, conspiring to have them deposed, or providing arms to military coups against democratically elected governments. Allende is probably the biggest example. Edit: and of course there's always the risk of a single rogue individual, from Dr. Strangelove's " Brigadier General Jack D. Ripper" to the very real Ford Hood massacre. In fact, Dr. Strangelove should be required viewing on this subject; not only is it darkly hilarious, it's all about setting in motion destructive partly-automated systems which cannot be countermanded.
    - RobertoG 11y ago
      It's funny, because probably is more relevant to think the problem the other way. What happens when leaders know that, no matter what, never are going to be "targeted" by their own automatic military?
  - jmngomes 11y ago
    You've had drone operators coming forward and exposing problems with the "program". That's a very good reason to keep humans close to these tasks.
  - TeMPOraL 11y ago
    Reminds me of The Machine in Person of Interest. (Spoiler alert for those who haven't seen the first 3 seasons.) The Samaritan works like you described - turns data and metadata into kill decisions. The Machine just outputs a social security number of a "person of interest". Figuring out why is the job of humans. The show features an interesting discussion about what happens when an AI following the first method (data -> kill decision) starts lying to you to further its own agenda. The same issue applies when you have a black box of sub-AI complicated analysis algorithms that make the decision.
    - tedunangst 11y ago
      Why in the world would you design such a system that deliberately fails to explain its decisions?
      - pjc50 11y ago
        Classification. If the system is using all available classified information, then an explanation of its output would have to include the security classification of every input, and the system of classification may also be such that no one person is cleared to know everything. Therefore nobody can know the rationale. (I've made this up, but I think it's the most plausible way such a thing would get built, and it's the logical outcome of "no read up/no write down" security rules.)
        TeMPOraL 11y ago
        I don't know much about the exact structure of classification. The situation you described would be certainly possible if the classification system is not a tree but a general graph. Is that the case with clearance levels?
        Retric 11y ago
        Yes, it's designed that way for counter espionage reasons. In theory only the president gets full clearance for everything.
- notahacker 11y ago
  The advantage of humans sitting and watching target details flash up on screen is that not being in any immediate danger themselves they should have fewer excuses for misindentification. Technology should make humans more acccountable for kill decisions rather than less.
  - coldtea 11y ago
    In practice it would just make it easier to kill -- so more killings. In wars, invasions, etc, including those done by "democratic powers" few care about "misindentification" (or even far worse offenses to the enemies including civilians). That's why the mostly go unpunished or just get a slap on the wrist. "Kill the all and let god sort them out" is the modus operandi.
- rquantz 11y ago
  We have bans on chemical and biological weapons. You don't say "but everything is made of chemicals. What are we even banning?" The law will create a legal definition and courts will interpret it. It's not the hard.
andrewstuart 11y ago
Just makes governments want it more.
simi_ 11y ago
Because the author was incapable of linking to Stephen Hawking's AMA, here it is: https://www.reddit.com/r/science/comments/3eret9/science_ama_series_i_am_stephen_hawking/ https://www.reddit.com/r/science/comments/3eret9/science_ama...
Shivetya 11y ago
their goal should be how to circumvent it or protect against it because no government is truly going to give it up. As the technology progresses there will more and more decisions removed from people to the point where what we think the line is today is just ho hum by then.
- IshKebab 11y ago
  I don't agree that no government will give it up. There are weapons available to western governments that are not used because of moral concerns. Nuclear bombs, land mines, biological weapons and so on. No doubt there will always be a Syria or North Korea that uses AI-controlled weapons anyway, but I think civilised countries could avoid them, at least while we have the military advantage.
  - sebiol 11y ago
    But those weapons exist and research continues. They are just not used (at the moment). You could argue that a state would own an AI and just not use it to direct attaks.
- tajen 11y ago
  Worse: By signing this protest there's all chances that AI be banned for citizen. But it will certainly keep being accessible to governments while making it legitimate to squash any citizen who attempts to build a counter-power.
  - shard 11y ago
    Even worse: The coming AI overlords can use this list as the first people to target for their "re-education program"
josephmx 11y ago
Why is Stephen Hawking treated like a tech expert?
- jacquesm 11y ago
  Because when a man that smart starts to think about fields outside his direct expertise it tends to be worth listening to. That and media have never ever exaggerated anything they printed.
  - tedunangst 11y ago
    It's weird. When we like what they say, it's "this guy is really smart. You should listen." When we don't like what they say, it's "experts should stick to what they know. Don't be fooled by appeal to authority."
    - jsutton 11y ago
      In some cases, I think you have to give credence to opinions from very reputable people. Stephen Hawking, for instance, is one of the greatest minds in the world; his word isn't gospel, but it's damn worth listening to.
- tgb 11y ago
  In the actual letter, the signatures are broken into two groups, one for AI and closely related experts, the other for people like Hawking, Musk, etc. Both are quite long and impressive lists, with the expert list actually being longer.
- the_af 11y ago
  Even worse, taking a look at futureoflife.org (the site that actually hosts the letter), I see their "Scientific Advisory Board" includes Alan Alda and Morgan Freeman. Now, I happen to respect both men as actors/directors. I also see they have both participated in scientific documentaries (as narrators/hosts, I suppose). I could understand if they were part of a "public relations board", which usually requires some degree of celebrity power, but a scientific advisory board? Doesn't this require one to actually be a scientist or a subject matter expert?
CmonDev 11y ago
I don't understand what kind of AI/robotics we are talking about here: - weak non-self-replicating; - weak self-replicating; - strong. Besides, maybe it's an egoistic point of view. Many species have perished while humanity established itself. Does it really matter if we go extinct, if the result is going to be a superior species?
- coldtea 11y ago
  >Besides, maybe it's an egoistic point of view. Many species have perished while humanity established itself. Does it really matter if we go extinct, if the result is going to be a superior species? In general, not wanting to die or go extinct is not considered an "egoistic point of view". Or, let's put it this way, from all the egoistic points of view, it's the most excusable. Why the duck should we care for a "superior species" (and a mchanical one at that)? Would you let a "superior country" fuck up your own country? Would you let a "superior person" kill your family and use your resources to sustain themselves? Is it OK to extinct dolphins and lions, since we are a "superior species"? Even more so since "superior" in this context has nothing to do with "morally better" but just means "more powerful" and "more fit to overtake others and survive".
wepple 11y ago
This should be read as "race for armed AI", not "an arms race toward the goal of AI". Very sneakily worded.
- forgetsusername 11y ago
  >Very sneakily worded. Hanlon's razor. Not everything is a media conspiracy.
- oneJob 11y ago
  agreed, and this is the whole reason the argument is academic. strong ai, once developed, will be the ultimate dual-use technology. one must not only deny a strong ai access to arms, one must also deny it access to anything/anyone that might help it become freed from the constraints preventing it from obtaining, using, or directing the use of arms. this is essentially the super well known thought experiment fleshed out in Ex Machina. only one solution: ban ALL strong ai. good luck with that.
coldtea 11y ago
Naives, warning against a far-fetched Californian fantasy. Meanwhile, few warn against widespread surveillance, the repurcursions from the use of drones, etc.
- mryan 11y ago
  Calling them naive seems misguided, and this scenario is no longer particularly far-fetched. > Meanwhile, few warn against widespread surveillance, the repurcursions from the use of drones, etc. The repercussions from drones could become worse if they are upgraded to include entirely autonomous AI, where there is no human involved in the "should I kill this person?" decision.
- eli_gottlieb 11y ago
  Everyone's been warning against widespread surveillance. The entire antiwar movement opposes drones. Where have you been?
  - coldtea 11y ago
    In the land where there are no "1000 tech experts" warning about them in mainstream media and rich moguls like Kurzweill and Musk getting interviewed every second week on the matter...
- ilaksh 11y ago
  I don't think its as naive or far-fetched as you think. But I do wish more celebrities would have the balls to stand up against real political problems of today.
ghshephard 11y ago
I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surrendered, etc... before engaging and killing them. The alternative is indiscriminate death that we see in mass bombings, mine fields, artillery strikes, and drone strikes.
- pmelendez 11y ago
  > I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surrendered, etc... The problem is that those intelligent systems will be overlorded by the same people overlording the actual military. At least in our current situation a human has to pull the trigger and that person could be potentially charged for crimes against humanity which would make it thinking it twice and potentially resist to follow a direct order. You won't have that with a machine.
  - fauigerzigerk 11y ago
    That's true, but the scenario you are thinking of is one in which the higher ups order a massacre and the soldier on the ground refuses to carry out the order. I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions, quite possibly panicking themselves. If they get it wrong, they're dead. An AI can make split second decisions without panicking and without any consideration for its own "survival". For any mistakes an AI makes, those higher up will get the blame (ideally), not some 20 year old who was scared for his own life when he made the decision to kill everyone on that overloaded truck because he it wouldn't stop 10 seconds earlier. That said, I shudder at the thought of a world in which people get killed by machines who will never be whistleblowers, who do not go home with post traumatic stress disorder, telling everyone how horrible war really is.
    - pmelendez 11y ago
      > I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions Tha't ok, but I think you are assuming that the "good guys" are the only ones that would have access to this. Truth is that once one government start doing this, all the others will follow and even the organized crime would have access to that technology.
seynb 11y ago
I think these people have read too much science fiction. >First we had the legs race. Then we had the arms race. Now we're going to have the brain race. And, if we're lucky, the final stage will be the human race. >John Brunner, The Shockwave Rider (1975), Bk. 1, Ch. "The Number You Have Reached" Great book, though.
lectrick 11y ago
This is not only overblown, it is misguided. All banning does is make people continue in secret. In any event, research in that area will produce knowledge that is both useful for non-weapon purposes as well as weaponizable... The same as all knowledge that has ever existed.
winestock 11y ago
For a humorous take on the issue, see http://www.supportkillerrobots.org/ http://www.supportkillerrobots.org/ Shameless plug, I wrote it on a lark.
- jackweirdy 11y ago
  Relatedly: http://stoprobotabuse.com/ http://stoprobotabuse.com/
Qantourisc 11y ago
Could we at-least use non-lethal weapons on humans ?
freddealmeida 11y ago
Very few ML experts on that list I'm sure. But at the same time, I'm rather against autonomous weapon systems. Yet not as naive to think that modern armies will ignore the benefits of machine intelligence. If you think about it, the US has not even ratified the Nuclear weapons testing ban treaty. I doubt it will ever consider an AI weapons ban.

74 points by callum85 11y ago

dkx 11y ago

I fear the AI arms race may be inevitable. Even if all the nations could agree to place limits on AI research there will always be a huge incentive to develop something in secret.

callum85 11y ago

I don't see how a ban could work. We already have technology that can identify targets, so the issue is whether a human is involved to approve each kill decision. But what constitutes human approval? Is it enough for a human to sit and watch target details flashing up on a screen, and intervene if they see an incorrect target? What exactly would be banned?

pjc50 11y ago
This is something of a problem. The system which scans mobile phone metadata (among other things) and turns it into drone bombing targets is already enough like the latter. A human is involved in pulling the trigger, but really they're following orders rather than having made the target/not target decision. And they may not even have the security clearance for the information on which the decision is based.
- bitJericho 11y ago
  I don't think that's so much a problem. The problem is literally robots pulling the trigger. What happens when the president is targeted due to error/hacking? A member of the military would have a major problem with assassination of her own leader; a robot may not care. What happens if everybody becomes a target? The military personnel would figure things out real quick, a robot, again, may not care.
  - TheOtherHobbes 11y ago
    A member of the military might not have any problem at all with the "accidental" assassination of their leader, whether that leader is the president or the company sergeant. AI could easily provide plausible deniability. I've been very critical of claims of the dangers of AI, but this isn't about AI - this is about giving lethal weapons to autonomous systems. And that's just stupid. All software has bugs, and even if you don't have a problem with autonomous killing systems - I certainly do - it's incredibly irresponsible to build systems that could kill just because someone left out a semicolon or couldn't convert between miles and km. It's taking the "blue screen of death" experience just a little too literally.
    - bryondowd 11y ago
      Umm, we already have machines that can kill people if someone "left out a semicolon". Airplane autopilots, medical equipment, car computers, etc. Not saying this makes it OK, but that isn't the reason. Life-critical code isn't a new thing.
      - jaawn 11y ago
        And those systems all have humans operating in conjunction with them, none of those things are (currently) autonomous. The success thus far of life-critical code usually depends partially on the assistance of humans.
        bryondowd 11y ago
        Most of those systems operate on their own unless/until they hit something they can't handle and a human is standing by as failover. So, unless you're saying it's OK to have deathbots as long as a human is on hand as failover if the deathbot gets confused, I think my point stands.
        TheOtherHobbes 11y ago
        IMO, the current state of the F-35 rather suggests it doesn't.
  - pjc50 11y ago
    A member of the military would have a major problem with assassination of her own leader https://en.wikipedia.org/wiki/List_of_coups_d%27%C3%A9tat_and_coup_attempts https://en.wikipedia.org/wiki/List_of_coups_d%27%C3%A9tat_an... Usually if a leader is assassinated, the assassin is a member of the country's military. The US is comparatively rare in having had no coups since independence and only one civil war. The US also has very strong indoctrination against this kind of thing, but individuals are just as vulnerable as robots to making the wrong decision if they are fed false information. Such as endless attacks on the legitimacy of the president's election, birth certificate, allegations of being a foreign agent, etc. The US is of course not above assassinating foreign leaders, conspiring to have them deposed, or providing arms to military coups against democratically elected governments. Allende is probably the biggest example. Edit: and of course there's always the risk of a single rogue individual, from Dr. Strangelove's " Brigadier General Jack D. Ripper" to the very real Ford Hood massacre. In fact, Dr. Strangelove should be required viewing on this subject; not only is it darkly hilarious, it's all about setting in motion destructive partly-automated systems which cannot be countermanded.
  - RobertoG 11y ago
    It's funny, because probably is more relevant to think the problem the other way. What happens when leaders know that, no matter what, never are going to be "targeted" by their own automatic military?
- jmngomes 11y ago
  You've had drone operators coming forward and exposing problems with the "program". That's a very good reason to keep humans close to these tasks.
- TeMPOraL 11y ago
  Reminds me of The Machine in Person of Interest. (Spoiler alert for those who haven't seen the first 3 seasons.) The Samaritan works like you described - turns data and metadata into kill decisions. The Machine just outputs a social security number of a "person of interest". Figuring out why is the job of humans. The show features an interesting discussion about what happens when an AI following the first method (data -> kill decision) starts lying to you to further its own agenda. The same issue applies when you have a black box of sub-AI complicated analysis algorithms that make the decision.
  - tedunangst 11y ago
    Why in the world would you design such a system that deliberately fails to explain its decisions?
    - pjc50 11y ago
      Classification. If the system is using all available classified information, then an explanation of its output would have to include the security classification of every input, and the system of classification may also be such that no one person is cleared to know everything. Therefore nobody can know the rationale. (I've made this up, but I think it's the most plausible way such a thing would get built, and it's the logical outcome of "no read up/no write down" security rules.)
      - TeMPOraL 11y ago
        I don't know much about the exact structure of classification. The situation you described would be certainly possible if the classification system is not a tree but a general graph. Is that the case with clearance levels?
        Retric 11y ago
        Yes, it's designed that way for counter espionage reasons. In theory only the president gets full clearance for everything.
notahacker 11y ago
The advantage of humans sitting and watching target details flash up on screen is that not being in any immediate danger themselves they should have fewer excuses for misindentification. Technology should make humans more acccountable for kill decisions rather than less.
- coldtea 11y ago
  In practice it would just make it easier to kill -- so more killings. In wars, invasions, etc, including those done by "democratic powers" few care about "misindentification" (or even far worse offenses to the enemies including civilians). That's why the mostly go unpunished or just get a slap on the wrist. "Kill the all and let god sort them out" is the modus operandi.
rquantz 11y ago
We have bans on chemical and biological weapons. You don't say "but everything is made of chemicals. What are we even banning?" The law will create a legal definition and courts will interpret it. It's not the hard.

andrewstuart 11y ago

Just makes governments want it more.

simi_ 11y ago

Because the author was incapable of linking to Stephen Hawking's AMA, here it is: https://www.reddit.com/r/science/comments/3eret9/science_ama_series_i_am_stephen_hawking/ https://www.reddit.com/r/science/comments/3eret9/science_ama...

Shivetya 11y ago

their goal should be how to circumvent it or protect against it because no government is truly going to give it up. As the technology progresses there will more and more decisions removed from people to the point where what we think the line is today is just ho hum by then.

IshKebab 11y ago
I don't agree that no government will give it up. There are weapons available to western governments that are not used because of moral concerns. Nuclear bombs, land mines, biological weapons and so on. No doubt there will always be a Syria or North Korea that uses AI-controlled weapons anyway, but I think civilised countries could avoid them, at least while we have the military advantage.
- sebiol 11y ago
  But those weapons exist and research continues. They are just not used (at the moment). You could argue that a state would own an AI and just not use it to direct attaks.
tajen 11y ago
Worse: By signing this protest there's all chances that AI be banned for citizen. But it will certainly keep being accessible to governments while making it legitimate to squash any citizen who attempts to build a counter-power.
- shard 11y ago
  Even worse: The coming AI overlords can use this list as the first people to target for their "re-education program"

josephmx 11y ago

Why is Stephen Hawking treated like a tech expert?

jacquesm 11y ago
Because when a man that smart starts to think about fields outside his direct expertise it tends to be worth listening to. That and media have never ever exaggerated anything they printed.
- tedunangst 11y ago
  It's weird. When we like what they say, it's "this guy is really smart. You should listen." When we don't like what they say, it's "experts should stick to what they know. Don't be fooled by appeal to authority."
  - jsutton 11y ago
    In some cases, I think you have to give credence to opinions from very reputable people. Stephen Hawking, for instance, is one of the greatest minds in the world; his word isn't gospel, but it's damn worth listening to.
tgb 11y ago
In the actual letter, the signatures are broken into two groups, one for AI and closely related experts, the other for people like Hawking, Musk, etc. Both are quite long and impressive lists, with the expert list actually being longer.
the_af 11y ago
Even worse, taking a look at futureoflife.org (the site that actually hosts the letter), I see their "Scientific Advisory Board" includes Alan Alda and Morgan Freeman. Now, I happen to respect both men as actors/directors. I also see they have both participated in scientific documentaries (as narrators/hosts, I suppose). I could understand if they were part of a "public relations board", which usually requires some degree of celebrity power, but a scientific advisory board? Doesn't this require one to actually be a scientist or a subject matter expert?

CmonDev 11y ago

I don't understand what kind of AI/robotics we are talking about here: - weak non-self-replicating; - weak self-replicating; - strong. Besides, maybe it's an egoistic point of view. Many species have perished while humanity established itself. Does it really matter if we go extinct, if the result is going to be a superior species?

coldtea 11y ago
>Besides, maybe it's an egoistic point of view. Many species have perished while humanity established itself. Does it really matter if we go extinct, if the result is going to be a superior species? In general, not wanting to die or go extinct is not considered an "egoistic point of view". Or, let's put it this way, from all the egoistic points of view, it's the most excusable. Why the duck should we care for a "superior species" (and a mchanical one at that)? Would you let a "superior country" fuck up your own country? Would you let a "superior person" kill your family and use your resources to sustain themselves? Is it OK to extinct dolphins and lions, since we are a "superior species"? Even more so since "superior" in this context has nothing to do with "morally better" but just means "more powerful" and "more fit to overtake others and survive".

wepple 11y ago

This should be read as "race for armed AI", not "an arms race toward the goal of AI". Very sneakily worded.

forgetsusername 11y ago
>Very sneakily worded. Hanlon's razor. Not everything is a media conspiracy.
oneJob 11y ago
agreed, and this is the whole reason the argument is academic. strong ai, once developed, will be the ultimate dual-use technology. one must not only deny a strong ai access to arms, one must also deny it access to anything/anyone that might help it become freed from the constraints preventing it from obtaining, using, or directing the use of arms. this is essentially the super well known thought experiment fleshed out in Ex Machina. only one solution: ban ALL strong ai. good luck with that.

coldtea 11y ago

Naives, warning against a far-fetched Californian fantasy. Meanwhile, few warn against widespread surveillance, the repurcursions from the use of drones, etc.

mryan 11y ago
Calling them naive seems misguided, and this scenario is no longer particularly far-fetched. > Meanwhile, few warn against widespread surveillance, the repurcursions from the use of drones, etc. The repercussions from drones could become worse if they are upgraded to include entirely autonomous AI, where there is no human involved in the "should I kill this person?" decision.
eli_gottlieb 11y ago
Everyone's been warning against widespread surveillance. The entire antiwar movement opposes drones. Where have you been?
- coldtea 11y ago
  In the land where there are no "1000 tech experts" warning about them in mainstream media and rich moguls like Kurzweill and Musk getting interviewed every second week on the matter...
ilaksh 11y ago
I don't think its as naive or far-fetched as you think. But I do wish more celebrities would have the balls to stand up against real political problems of today.

ghshephard 11y ago

I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surrendered, etc... before engaging and killing them. The alternative is indiscriminate death that we see in mass bombings, mine fields, artillery strikes, and drone strikes.

pmelendez 11y ago
> I would think it better that we have weapon systems that intelligently make an assessment as to whether their target is (A) a combatant, (B) a threat, (C) surrendered, etc... The problem is that those intelligent systems will be overlorded by the same people overlording the actual military. At least in our current situation a human has to pull the trigger and that person could be potentially charged for crimes against humanity which would make it thinking it twice and potentially resist to follow a direct order. You won't have that with a machine.
- fauigerzigerk 11y ago
  That's true, but the scenario you are thinking of is one in which the higher ups order a massacre and the soldier on the ground refuses to carry out the order. I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions, quite possibly panicking themselves. If they get it wrong, they're dead. An AI can make split second decisions without panicking and without any consideration for its own "survival". For any mistakes an AI makes, those higher up will get the blame (ideally), not some 20 year old who was scared for his own life when he made the decision to kill everyone on that overloaded truck because he it wouldn't stop 10 seconds earlier. That said, I shudder at the thought of a world in which people get killed by machines who will never be whistleblowers, who do not go home with post traumatic stress disorder, telling everyone how horrible war really is.
  - pmelendez 11y ago
    > I believe that the more frequent case is when soldiers on the ground are having to make split second life and death decisions Tha't ok, but I think you are assuming that the "good guys" are the only ones that would have access to this. Truth is that once one government start doing this, all the others will follow and even the organized crime would have access to that technology.

seynb 11y ago

I think these people have read too much science fiction. >First we had the legs race. Then we had the arms race. Now we're going to have the brain race. And, if we're lucky, the final stage will be the human race. >John Brunner, The Shockwave Rider (1975), Bk. 1, Ch. "The Number You Have Reached" Great book, though.

lectrick 11y ago

This is not only overblown, it is misguided. All banning does is make people continue in secret. In any event, research in that area will produce knowledge that is both useful for non-weapon purposes as well as weaponizable... The same as all knowledge that has ever existed.

winestock 11y ago

For a humorous take on the issue, see http://www.supportkillerrobots.org/ http://www.supportkillerrobots.org/ Shameless plug, I wrote it on a lark.

jackweirdy 11y ago
Relatedly: http://stoprobotabuse.com/ http://stoprobotabuse.com/

Qantourisc 11y ago

Could we at-least use non-lethal weapons on humans ?

freddealmeida 11y ago

Very few ML experts on that list I'm sure. But at the same time, I'm rather against autonomous weapon systems. Yet not as naive to think that modern armies will ignore the benefits of machine intelligence. If you think about it, the US has not even ratified the Nuclear weapons testing ban treaty. I doubt it will ever consider an AI weapons ban.