9 ms·

Don't Let Robots Pull the Trigger

• opwieurposiu 8y ago
  Robots have been killing people by accident for years, soon we will find a way to turn this bug into a feature!
• krisoft 8y ago
  So how is a cruise missile not an autonomous weapon?
  • hutzlibu 8y ago
    The cruise missile does not decide on its own, when to start. Humans do.
    • maxxxxx 8y ago
      It also doesn't decide the target.
      • dsfyu404ed 8y ago
        Neither does the person launching it though. Software in the case of AI weapons is basically just enlisted personnel that follow orders perfectly.
        • maxxxxx 8y ago
          I thought this is about weapons making decisions in the field autonomously. Like spotting a target and then also deciding to kill it.
          • dsfyu404ed 8y ago
            Well right now Pvt. Crayon is making the decision on whether or not to engage. In the edge cases Pvt. Crayon is probably going to be better than AI at making the call for a hell of a lot longer than the people pushing the technology claim at any point in time. The question really becomes how much you care about edge cases. When there's a car bomb barreling toward your checkpoint being able to just hit the big red "kill everything in this predefined space" button and dive for cover would be a godsend.
• v_lisivka 8y ago
  Mines and other kinds of death traps are primitive robots too.
• johngalt 8y ago
  Aren't 'killer robots' ethically similar to landmines? Only with better fire control systems. Not saying that landmines are paragons of ethical weapon systems, but they are making a kill decison without a human being in the loop. Based on nothing more than a pressure plate or trip wire. If we had landmines that could differentiate between a tank and a school bus wouldn't that be a step forward?
  • nkingsy 8y ago
    https://www.un.org/disarmament/convarms/landmines/ https://www.un.org/disarmament/convarms/landmines/ "The ensuing Mine Ban Convention has been joined by three-quarters of the world’s countries. Since its inception more than a decade ago, it has led to a virtual halt in global production of anti-personnel mines, and a drastic reduction in their deployment. More than 40 million stockpiled mines have been destroyed, and assistance has been provided to survivors and populations living in the affected areas." I think the ideas is nip this one in the bud before it gets there.
    • GauntletWizard 8y ago
      Yes. Autonomous weapons systems don't make much sense on patrol for a long time, but as area-denial systems, they're already past viable and it's surprising they're not deployed. I think there is a big argument to be made for their ethicality over landmines: unlike landmines, they are easy to deactivate and can be made highly visible while remaining effective. I don't think this makes them ethical, but I do believe that we have to consider how that changes their potential uses. There's an argument that it's far safer if we have hard rules around certain areas - "Enter a 500m radius around this military base moving more than 5mph and you will be shot many times" - than if we have fuzzy ones around guards and schedules. Certainties are often better than including a fuzzy human in the loop, which is why electrical installers all put their own lock on the switch rather than assigning a coordinator.
      • dsfyu404ed 8y ago
        >they're already past viable and it's surprising they're not deployed. Not deployed as far as the public knows. It wouldn't exactly be hard to add a "shoot everything that comes near us" flag to the software for any existing naval CIWS
• empath75 8y ago
  A robot is just a more complicated fuse.
  • function_seven 8y ago
    A fuse can't make a decision to light itself.
    • Varcht 8y ago
      Some weapon systems arm themselves according to certain conditions, altitude, time from launch, etc. I think one could argue that some missile systems are already robots that "pull the trigger".
      • thisisweirdok 8y ago
        But a human dictates the launch and what the target will be.
        • Varcht 8y ago
          Do they? Aren't some just sent out to find a heat source and blow it up?
          • thisisweirdok 8y ago
            Oh god no. At least not as the primary guidance for long-range. It can be partially used to track a target, but it's not being used to decide where to target in a broader sense. Short range devices sure, but they still require a human to point and launch. There's not a margin of error there that will divert a missile from an armored vehicle to an elementary school.
        • Varcht 8y ago
          How is this different from programming the killer robot and turning it on?
      • 8y ago
• goldenshale 8y ago
  I think the authors are 100% right on everything, but they failed to bring up one of the most important factors that I believe is driving US policy in this area: if Russian or Chinese weapons are automatically firing at the speed of a neural network while our weapons are waiting for meatware actuation then it's going to be a short battle.
  • binarymax 8y ago
    Similar to chemical warfare. There needs to be a UN resolution to ban these types of weapons. Make it so use of them constitutes war crimes.
    • philwelch 8y ago
      Chemical weapons aren't used because they're not worth the risk. There was virtually zero battlefield use of chemical weapons in the Second World War, which was otherwise as "no holds barred" as you can imagine.
      • gotocake 8y ago
        It’s not really risk that’s an issue, it’s a combination of expensive production, dangerous and expensive storage, and expensive deployment contrasted with a total lack of efficacy in killing the intended target. Unless your goal is to murder untrained and unequipped civilians, chemical weapons are just not very good weapons. Soldiers are trained, have vehicles with overpressure seals, masks and MOPP suits, detectors and a command structure. The real use for chemical weapons is to terrify civilians and poorly organized/equipped militias. Most of all it’s a way to murder a large number of civilians without destroying infrastructure. Needless to say that’s not something you’d typically do in a war, where destruction of infrastructure is typically step one! As a result most countries recognize the economic and social burden of chemical weapons and their use, and strongly oppose them.
        • coolspot 8y ago
          > The real use for chemical weapons is to terrify civilians and poorly organized/equipped militias. See Vietnam War, Napalm, Agent Orange etc.
• lewiscollard 8y ago
  My concern about autonomous robot warfare is not that it will shoot the wrong people or go out of control; that's easily answered by "but this one won't", and there's no counter-argument which will not look like pure assertion. My concern is that they will work just fine: this would bring the domestic costs of warfare to near-zero (the cost of the robot weapons). For those of us that are sceptical of military adventurism, just consider how much more willing our politicians would be to invade some random, relatively harmless country if there were no "boots on the ground" and no possibility of body bags coming home.
  • deleted 8y ago
    [deleted]
  • philwelch 8y ago
    The long, relative peace on the Korean peninsula has persisted despite a massive manpower advantage by the North. The presence of hard defensive positions augmented by landmines and autonomous sentry guns allows South Korea to deter Northern aggression without having to keep up with North Korea's punishing and impoverishing level of military spending. Like any weapon, robotic weapons are only a problem if you use them for evil.
    • dsfyu404ed 8y ago
      >The presence of hard defensive positions augmented by landmines and autonomous sentry guns allows South Korea to deter Northern aggression without having to keep up with North Korea's punishing and impoverishing level of military spending. And the presence of plenty of artillery pointed at Seoul has prevented anyone who doesn't want to see Seoul leveled as a result of their actions from taking aggressive action against North Korea. Knowing that if you do anything bad you will get whacked hard enough that it's not worth it is a powerful deterrent. >Like any weapon, robotic weapons are only a problem if you use them for evil. Because of the inherent defensive bias of the technology (stationary is easier than mobile, defending a known area is easier than entering an new area and figuring out what to shoot) robotic weapons are a bigger problem for those seeking to do or enable evil.
• captainbland 8y ago
  I think in principle this means we need to develop robots which are capable of disabling robots using means which are generally non-lethal (not through autonomous decision making, but matter-of-fact mechanisms like EMP, sensor blocking/jamming and so on).
  • carapace 8y ago
    Yes! Defense is easier than offense in general, especially with time to prepare. One area that should be investigated is sticky hairy counter measures. Just think about what kills your vacuum cleaner? Or what about rip-stop protective clothing that can kill a chainsaw in a split second? (I have my leg now because of protective chaps with kevlar fibers. Grubbing out a nasty old stump, one moment of inattention and the chain bit my leg, but the fibers jammed the saw in an instant. I looked down and it actually took me a few moments to realize that the rip in my chaps with the streamers of kevlar hanging down could have been my own precious meat! Man did I shiver, and send up a prayer of gratitude for the folks who made the chaps!) Anyhow, once you start thinking defensively the ideas flow readily. One idea I want to try is a thing that shoots those plastic balls (for the plastic ball pits you play in) but with a non-Newtonian goo (like cornstarch goo) that goes stiff at greater energies but is fluid at lesser. Result: anti-riot "foam". People can't run around and flail in it but you can still move around (to e.g. provide first aid). There could be an enzyme or something that can be sprayed to dissolve the goo... etc.
• deogeo 8y ago
  What worries me most is how robotic warfare further concentrates power. Currently you have to convince the people in the military to follow your orders, meaning you need at least some legitimacy. What happens when even that human check on power is gone?
• thisisweirdok 8y ago
  This absolutely needs to be an agreed upon standard similar to chemical weapons and mines. Anyone remotely familiar with software development knows that automating either when to launch or what to target will not end well. Which bug will end a civilization?
  • v_lisivka 8y ago
    Forbid robots and send this guy instead. Two problems solved.
• deleted 8y ago
  [deleted]
• yboris 8y ago
  Relevant: https://www.stopkillerrobots.org/ https://www.stopkillerrobots.org/ > Formed in October 2012, the Campaign to Stop Killer Robots is a coalition of non-governmental organizations (NGOs) that is working to ban fully autonomous weapons and thereby retain meaningful human control over the use of force.
• georgeecollins 8y ago
  Autonomous robots are going to be really effective in all the places where it is difficult for a human to survive and where decision making is mediated be technology, not human senses. So, they are going to be really effective deep underwater, in space, and in high-g aerial combat at radar range. Setting aside the moral question, there is also the institutional issue that officers may not like to see pilots replaced with drones, sailors with UUAVs, etc. That creates this dangerous place where the US military can have it's manned systems disrupted by cheaper unmanned ones. I think that is dangerous for me. As far as the moral issue: What is the difference between a machine firing a missile at a plane it believes is attacking and an officer on a ship pressing a button to fire that missile based on the risk assessment of a machine that tells the officer that an airliner is an attacker. Ref: https://en.wikipedia.org/wiki/Iran_Air_Flight_655 https://en.wikipedia.org/wiki/Iran_Air_Flight_655 In that case the crew may have made a mistake in judgement. But it is easy to see a potential future case where no human interpretation is better than how a computer interprets the incoming data, and humans have to act on the computer's judgement alone. We can insert humans in the loop but that may not really mean anything.
• carapace 8y ago
  Push this to it's logical conclusion: Put the robots in an arena and decide policy based on who wins. No one has to get hurt and we don't even have to blow up real buildings and stuff. Play war games to game war.
• burfog 8y ago
  The USA already has this. Russia and China have similar systems. If you fly a kamikaze attack against a large US warship, Raytheon's Phalanx CIWS will automatically fire. I believe it handles speedboats as well. The Patriot system is similar, able to shoot down incoming fighter jets.