8 ms·
U.N. talks adjourn without deal to regulate 'killer robots'
- msla 5y agoHow are these autonomous killer robots significantly different from cruise missiles?
- ryankupyn 5y agoI think the key distinction is that, for the most part, the current generation of cruise missiles are targeted and launched with a "human in the loop" - that is, a person actively makes the decision to fire the missile at a specific target. However, the autonomous weapons being debated have significantly less direct human control - they are potentially designed to simply patrol a certain area and attack anything that the weapon classifies as an enemy. Of course, there are grey areas here - certainly there are existing missiles that can be launched without a defined target and programmed to aim for anything that, say, has radar emissions that match known enemy systems.
- ahnick 5y agoAutonomous drones with weapon systems coupled with sufficiently advanced obstacle avoidance, facial recognition, and other AI/ML identification based systems, could swarm cities and eliminate entire battalions or local populations with little to no environmental or infrastructural damage. Imagine Russia preparing to invade the Donbas region of Ukraine and ahead of that invasion deploying a couple of drone swarms to "clear out" a bulk of the Ukranian army presence. Drones could idle, lie in wait and silently hunt opposing forces. It could significantly lower the cost (number of casualties) of invasion for the attacking force.
- deleted 5y ago[deleted]
- trhway 5y agoIt is already here. Ukraine got Turkish drones, and Russia can only amass large WWII style force in response which are going to be slaughtered by the drones. Drones are the new paradigm of warfare which allows smaller players to play in many situations (like the modern limited local wars) on par with the big ones without attempting to acquire nuclear. No UN talks can prevent those smaller players from development of such capabilities.
- heyitsguay 5y agoYeah I've been kinda wondering about what the deal is there - given the slow public build-up of forces it seems hard to imagine Russia going forward with an old-school invasion when all those troops will just be drone fodder. Is it for internal propaganda? A way to gather intelligence on Ukraine's defensive responses? I'm sure it's hard to tell from the outside.
- trhway 5y agoUkraine has got strategic advance with the drones (Russia didn't get the message 2 years ago when the air defense Pantsir systems got slaughtered in Syria by those drones). Such a huge advance is though temporary - those Russian air defenses need upgrade to deal with the drones, and it will be done in the near future. So, Ukraine got window of opportunity to solve Donbass the way Azerbajan solved the Karabah using the same drones a year ago. So Russia is trying to prevent it by the public buildup of forces in the way which clearly shows that it can cut Ukranian Donbass forces from the rest of the country by scissor invasion from North and South behind those forces if Ukraine makes a move on Donbass now. Such invasion will be with huge losses and will cost Russia a lot and is very risky, and Russia doesn't want it (at least not at the cost it is going to be at right now), and i'm not sure that Russia can really pull it right now, yet it is the only response Russia got for now, thus such a public build-up of the forces looking a lot like bluffing in poker.
- 2OEH8eoCRo0 5y agoYou do the arm signal for drone swarm, shout "drones, drones, drones!", and drop to the ground and cover yourself in your poncho. My point is that we adapt. It's not worse than chemical weapons and it's preferable to bombing entire cities to achieve your objective. Is it finally dawning on HN that war is hell?
- pharke 5y agoYou're missing the point, new technological developments in warfare aren't about "worse or better" since one human wishing to inflict suffering or death on another needs only a sharp stick or a rock and their imagination to produce the worst forms of torture. They're about physical distance. Fighting hand to hand requires an immense level of personal commitment to causing harm. Each technological advance in warfare has lowered that barrier by increasing the distance at which you can maim and kill. It's now possible to kill without ever seeing the person you are killing. Autonomous weapons are the final stage of this development where you don't even need to make the decision to kill, you simply design some inevitably flawed conditions under which you would theoretically kill someone and the machine takes care of the rest. I think the objective should be preventing war and that means making it more difficult to kill not less.
- 2OEH8eoCRo0 5y ago> I think the objective should be preventing war I agree and that's usually the connotation behind the words, "war is hell." Unfortunately, either the entire world agrees to never weaponize drones or we all need to start thinking about weaponizing drones. That's how arms races work. If wars are going to happen though I think we are better off with drones rather than annihilating entire cities, civilians and soldiers together. It's important to elect leaders who are not cavalier about deploying drones needlessly.
- inglor_cz 5y agoWill the killer bots be any smarter than contemporary robotic admins that delete posts and lock accounts on Facebook? If not, the first casualty of the next war in the Middle East will be the closest Department of Algebra, because the name sounds Arabic. I wish I was joking. The potential for bad decisions in such autonomous systems is tremendous.
- dane-pgp 5y agoAs a practical example, to set our expectations, let's remember this event from 1999: "US Defence Secretary William Cohen has confirmed that an out-of-date map was used in the attack which destroyed the Chinese embassy in Belgrade." http://news.bbc.co.uk/1/hi/world/europe/340735.stm http://news.bbc.co.uk/1/hi/world/europe/340735.stm
- unchocked 5y agoFrom a normative perspective, terminators are self-evidently dangerous as all hell and should be prohibited. From a realist frame, I don’t think there will be a deal. Narrow national interests are too divergent w/r/t/ capabilities, intentions, and alternatives. Furthermore, verifibility and consequent trust is weak.
- RGamma 5y agoAs always in unstable equilibria all is fine and well as long as everyone complies, but anyone may gain a unilateral advantage misbehaving. Pragmatically it's a lack of higher (multilateral) enforcement why these power plays persist.
- Aerroon 5y agoBut the reason why higher enforcement doesn't exist is because nobody is trustworthy enough for it. American domination has been relatively good for the West, but citizens of many countries have immensely suffered as a result. Going back further it was European dominance that everyone else suffered under. Multilateral enforcement becomes yet another tool to take advantage of others that aren't invited.
- JumpCrisscross 5y ago> terminators are self-evidently dangerous as all hell and should be prohibited These are war machines. “Dangerous” per se cannot be a disqualifier.
- tbrake 5y agoAre you sure OP's use of "dangerous" wasn't around the implications of fully autonomous, no-human-intervention nature proposed in the article, and not the blatantly literal destructive nature of all war machines?
- sitkack 5y agoMany dimensions to words, your typographic skills reveal a non-accidental interpretation.
- sebow 5y agoAnd who expected a different result? No shit... people with power will keep a door available for this technology as a tool to use on dissidents.
- ProjectArcturis 5y agoHow can you possibly expect to regulate something like that? Every world power expects that the next major war will be fought with autonomous robots playing a big role. Whoever has the better technology will have a huge advantage. Consequently you can never trust that the other signatories will honor their promises, and so in turn you cannot risk signing it yourself. Like the Geneva Convention, any robot agreement will be ignored by all combatants as soon as it serves their purposes. War cannot be regulated by gentlemen's agreement.
- jMyles 5y agoI much prefer that the state which capriciously claims dominion over the land mass on which I happen to exist today should lose a war with one of its childish rivals than to acquire and advance more deadly toys. It may not be a gentlemen's agreement, but it's something with which I imagine there are broad consensus across borders.
- JoeyBananas 5y agoSo basically you don't even value your own life? Is that really the basis of your argument? You don't even try to argue "More deadly weapons are not neccessary for our survival," you just straight up concede "I'm OK with losing the war and having everyone get killed." Good for you, but the rest of us want to survive.
- jMyles 5y agoI think it's pretty obvious for those of us living in places "governed" by superpowers that the international affairs of state in which they are currently engaged have no bearing on our individual longevity. The USA state will fall eventually. Will it be in 500 years? 100? 20? Nobody knows for sure. But the fall will cause the end (or even substantial disruption) of a tiny portion of the lives of the people living in the regime's claimed territory. I don't fault your logic; I just think it's out-of-date. The world (especially those parts of it we're discussing here) is really quite different than in the 19th century. Very little of the original impetus for the state (specifically, solving problems of coordination created by the absence of established communications across the land-mass, as a non-trivial portion of The Federalist Papers seek to resolve) remains. In other words; we'll be fine. We won't merely survive.
- Ultimatt 5y agoJust for context this is what they are "debating" saying yes or no to https://youtu.be/TlO2gcs1YvM https://youtu.be/TlO2gcs1YvM
- thwarted 5y agoThis should have ended with the guy on stage getting attacked by a drone deployed by an unknown party, to really drive home the lack of defense against this kind of technology.
- sitkack 5y agohttps://www.youtube.com/watch?v=vbnySBqioB0 https://www.youtube.com/watch?v=vbnySBqioB0
- sen 5y agoIt'd be better to link to the source[0], that's a content-stealing channel you linked to. [0] https://www.youtube.com/watch?v=9CO6M2HsoIA https://www.youtube.com/watch?v=9CO6M2HsoIA
- barelysapient 5y agoI suspect ‘killer robots’ wont be banned until after they are used to great effect in some future conflict. Think chemical weapons bans that emerged after WWI.
- heyitsguay 5y agoChemical weapons were banned because they weren't very effective on the battlefield and generated horrible PR for all involved, so it was an easy win for everyone to ban them. The problem with autonomous weapon platforms (probably swarms) is that it looks like they're very powerful weapons that give their users advantages over existing capabilities. Enforcing a ban on them will be like enforcing a ban on gunpowder with swords and castles.
- deleted 5y ago[deleted]
- sitkack 5y ago> enforcing a ban on gunpowder with swords and castles Which really would have been a good idea. It isn't that all those Europeans had to die to get this current stable state.
- peakaboo 5y agoI read articles like this and I think "enjoy every minute of your lives" because it's only getting worse from here.
- tempnow987 5y agoIf you are a major power with the capability to make 100K+ killer drones or similar per year, going to be hard to get you to sign this. What is the drones annual salary? What is their annual housing cost, base allowance costs, VA medical benefits (lifetime)? How upset will your population be if one is "killed". Get the beancounters involved, and killer drones are going to be very tempting. For $75B per year or so (10% of US military budget) what kind of killer drones could I buy and how many? The cost to deploy one F35 is astronomical. The training costs and lifetime costs for the pilots alone is heavy. The F-35 dedicates MASSIVE payload to life support for its human occupant. What kind of killer drones can you build with none of that mass, none of the gforce limits? None of the redundancy requirements? Do you need an aircraft carrier (thousands of sailors! -> needs destroyers and a battle group). No? Can we design these to drop from space? Park 50K in space for quick de-orbit and insertion globally? Can you do this with people these days easily? Going to be a scary world. My quick guess is US, China, Russia, EU (maybe), India as places with enough capacity for this type of next gen battlefield? Maybe EU will move too slowly, so it'll be US / China / Russia?
- jMyles 5y ago> If you are a major power with the capability to make 100K+ killer drones or similar per year, going to be hard to get you to sign this. Surely the individual decision-makers are, as with all human beings, averse to a world swarming with killer robots. That states make indefensible decisions with much greater frequency than their individual statesfolk is so obvious at this point; it's time to simply stop tolerating their existence and move more swiftly into the next stage of information humanity. I'm a citizen of one of the states you mention and I can assure you: amidst all the disagreement and political polarization, there is complete consensus on not wanting a world dominated by killer robots.
- all2 5y ago> there is complete consensus on not wanting a world dominated by killer robots. That and nukes. It is ok to not want these things, but to survive as a nation state, these things will become necessary developments in order to survive. If GP is correct about some of the capabilities we're likely to see, then any nation state without these will need to begin development in order to ensure survival. Is that an arms race waiting to happen? Yes. Is there a better way? No. The alternative to an escalating arms race is actual war. We will never be done with power hungry ideologues, we will never be done with Communists whose sole mission is dominion over the whole world, we will never be done with petty despots (looking at you, Putin) who so wish they could play a meaningful role on the world stage. All of these have every reason to develop a new super weapon to tilt the balance of power in their favor. Iran, for example, has started its centrifuges again. They also have some new ones that spin at 10x the speed of their old ones. They are very close to possessing a nuclear bomb. Up until the end of the Trump presidency they had halted production of uranium and talks were going well. Biden has about the weakest foreign policy of any president in the last half century. Part of that effective foreign policy is the military might the US has. (Why do we have 5 carrier battle groups? Why does Britain only have one? Why does Russia have a single carrier that's in rather poor repair?) If killer drones are an effective war-fighting tool (an it appears they are), they are inevitable. What we need to ensure is that they are used as humanely as possible.
- WFHRenaissance 5y agoI wonder what group will succumb to genocide by drone first? I figure once you solve the computer vision problem of identifying a group, it's pretty much just an ammunition logistics problem. Imagine The Holocaust without trains and queues. Just a matter of time until another leader needs to scapegoat another group, and next time it will be easier than it's ever been.
- blunte 5y agoIt doesn't matter if they regulate it or not. It will happen in secret. It is a foolish hedge, as the nuclear race was. Fortunately, despite the Deepminds and such, outside limited games we humans still have the upper hand. And us techies, with some effort, can not only defeat killer robots, but we can adversely affect or even control them. Considering the many woes of humanity, and the inevitable doom we face, I find this an interesting endgame to play. At least now we have something interesting to do while the world burns.
- djohnston 5y agoYou can call the nuclear race foolish but it has prevented mass armed conflict more effectively than any diplomatic maneuver of the past 300 years.
- s1artibartfast 5y agoPeople think of terminators when they hear killer drones, but the cat has been out of the bag for a long time. We've already had Killer Robots for many decades. It started with icbm's in the 1950s. More sophisticated platforms like the US the US tamohawk missile have been around since the 80s.
- kevin_thibedeau 5y agoThey don't choose their targets and until relatively recently couldn't be retargeted after deployment.
- s1artibartfast 5y agoI think tamohawks have been able to be programmed with a target hierarchy for a long time, but this is a good point. It seems that The Relevant distinction is automated Friend or Foe identification. Killer drones are not new, but this capability is becoming possible.
- redis_mlc 5y agoThe problem with killer robots is that it lowers the political threshold for both starting wars, and for continuing them. In Europe today, even 2 or 3 dead soldiers is enough to end coalition participation. (Japan's participation in Iraq wasn on the condition of no direct combat and no casualties, so other countries had to babysit the Japanese soldiers.) The US in WW2 had to invade Japan ASAP before the political will at home faded, which is why unconditional surrender was demanded - there was no way politically to rearm and sail across the Pacific again. Meanwhile, that wasn't a factor for the Axis or Russia. Japan's behavior in China in WW2 was basically identical to what you see in the "Terminator" movies. Something to think about.
- JoeyBananas 5y agoAutonomous weapons don't worry me. There isn't anything they can do that a human controlled weapon cannot do. Most of these autonomous weapons are basically just a different vehicle bringing the payload to the target. For example, let's say there is a bomb that can be strapped to a drone with a camera on it and guided by AI. That would be easy to evade by just staying out of the open. I'd be much more afraid of my entire city being killed by chemical weapons. Facial recognition technology will not be a huge game-changer. Most weapons don't even bother to recognize your face to determine if you're a friend or a foe, they just kill you. Real world autonomous weapons have existed for a long time. The phalanx CIWS is an example of an autonomous weapon designed in the late 60s and it is far from perfect. They tend to just start randomly spraying bullets when they get confused, and they have been involved in friendly fire incidents.
- schroeding 5y ago> There isn't anything they can do that a human controlled weapon cannot do Yes, there is: They don't require human operators, which are a limited ressource. You can only operate so many drones at the same time until you run out of weapon operators / "pilots". You can't scale human operated weapons in the same way you can autonomous ones, can you? There is nothing stopping you from using hundreds of thousands of cheap, autonomous kamikaze drones at the same time, 24 / 7, if you have the money for it. > That would be easy to evade by just staying out of the open. How long can you stay in one building? How long could a drone (or two, with one recharging while the other is flying) circle above it? Bombing a whole city with chemical weapons, killing countless civilians would cause a major international incident, even with countries not involved in the war / conflict. Killing single individuals with "killer drones" based on AI facial recognition will probably not :/
- JoeyBananas 5y ago> You can't scale human operated weapons in the same way you can autonomous ones, can you? Sure you can. That's what the whole point of an army is. Instead of just having just one tank, have 500 tanks. Sure, scaling does require more manpower, but that's true even with autonomous weapons until someone invents AGI. > How long can you stay in one building? How long could a drone (or two, with one recharging while the other is flying) circle above it? Just put a ski mask on. Problem solved. There is just no way that drone swarms with facial recognition technology are ever going to be viable.
- daveac 5y agoHave a listen to this guys lecture on AI and warfare. He explains why we should not fear Skynet instead explains machines capable of selecting targets / people and automatically deciding who should die, their cheep mass production, state of current progress and ways of controlling their use if this needs done. https://www.bbc.co.uk/programmes/m00127t9 https://www.bbc.co.uk/programmes/m00127t9 It’s really up to date to as he is also delivering a message to those at the UN talks today
- daveac 5y agohttps://www.google.co.uk/amp/s/www.nytimes.com/2021/06/03/world/africa/libya-drone.amp.html https://www.google.co.uk/amp/s/www.nytimes.com/2021/06/03/wo...
- BrazzVuvuzela 5y agohttps://en.wikipedia.org/wiki/Mark_60_CAPTOR https://en.wikipedia.org/wiki/Mark_60_CAPTOR A CAPTOR mine sits underwater listening to nearby ships or submarines, for weeks to months. If it hears the right sort of ship, it can launch itself autonomously, track it and kill it, without a human in the loop. It's been in service since the 70s.
- dirtyid 5y agoOnly a handful of countries are capable of developing killer robots to maturity and they will end up monopolizing the arms trade market. The tech will proliferate for this reason alone. If AI is integral to killer drones, then expect a lot more skirmishes going forward to gather data. And realistically there's only a handful of countries / powerblocks who has the impunity to conduct such "research". And when you narrow perquisites down account for synergy with domestic AI industry, that elite club dwindles even smaller. US, PRC, EU if she gets her shit together. But IMO killer robots along with developments like prompt global strike will trigger dreadnaught tier shift in resetting global power dynamic. Arms race dilemma 101, countries that can, simply can't afford not to pursue them.
- sitkack 5y ago> Only a handful of countries are capable of developing killer robots Every nation on earth has the capability.
- dirtyid 5y agoI'm sure most countries can get a UAV to drop some munitions on a person. But that's easy mode. Caveat is: > to maturity i.e. hightech platforms designed to confront other hightech platforms. Only a handful of country has both the industrial base in terms of AI and MIC to develop drones for capabilities like Anti-air, anti-sub warfare. Most of the world do not have 5th gen planes or modern subs to even train against. Or the host of communication infra, especially space that's needed to get platforms working in field. Only a handful of countries can even contemplate doing that.
- mensetmanusman 5y agoMaybe the great filter is just competing organic life forms that eventually make swarms of killer silicon life forms (robots). It seems entirely inevitable that these machines will be everywhere policing the masses for the powerful few.
- dane-pgp 5y agoThat's an interesting idea, but I think you've missed a step in your reasoning. Sadly, a dystopia where powerful elites enslave the majority of the global population could be quite a stable state of affairs, and that dynamic could persist long enough for the civilization to spread throughout the galaxy. Even if a war between great powers were to occur in such a scenario (which would not be in the interests of the elites who profit from the status quo), I don't think killer robot swarms are any more likely to, or capable of, causing global civilization collapse than nuclear weapons already are.
- mensetmanusman 5y agoI think there is something we can glean from what happened with the millions of land mines after conflicts, they were just left there to continue to destroy human lives for decades (probably they can last about a hundred years and still trigger). Robots that are solar powered and off in remote parts of the world may continue to destroy people when their nation-state owners are gone because of the last MAD command they get.
- dane-pgp 5y agoI don't think that robots "off in remote parts of the world" constitute an existential threat and a "great filter" unless there is some failure mode where they all suddenly get bored and go off to find targets outside their programmed area.