7 ms·
The Pentagon inches toward letting AI control weapons
- hcurtiss 5y agoWhile there are obvious perils here, I fear it would be tremendously irresponsible to let our adversaries own this space. AI weapons must be developed if for no other reason than to battle AI weapons.
- swiley 5y agoNow I understand how the anti-nuclear people in the 20th century felt.
- ethbr0 5y agoNuclear MAD seems to have worked out well, so far.
- anigbrowl 5y agoThe problem is that MAD is only reliable in the absence of proliferation, but we go to such lengths to prevent proliferation that it causes other problems. When you get down to it a nuke is not that that hard to build with the resources available to almost any state, so the fragility of deterrence has a distorting effect on international relations.
- yboris 5y ago"so far" included very many near misses - where individuals with explicit authority to launch counter attacks were in position to decide whether to launch NUCLEAR MISSILES AT A COUNTRY -- because some sensors malfunctioned. See Stanislav Petrov - the man who prevented an apocalypse https://en.wikipedia.org/wiki/Stanislav_Petrov https://en.wikipedia.org/wiki/Stanislav_Petrov
- ethbr0 5y agoYes. I'll take near misses with nuclear Armageddon over total war. The anti-nuclear weapons crowd conveniently forgets that the alternative to MAD isn't reliably peace.
- yboris 5y agoWhat % chance of nuclear holocaust per year are you wiling to take over the "non reliable peace"? We're talking about potential existential risk - a wiping out of all of humanity (through severe ecological poisoning after a barrage of cross-globe nuclear strikes).
- ethbr0 5y ago0.01%? If my math works out, that's a 10% chance of an occurrence every 1,000 years.
- yboris 5y agoThank you for providing a number. I have seen estimates of what the risk was - as high as 1% per year for much of the duration of the Cold War.
- ethbr0 5y agoSo if we said '55 - '90 (35 years), that'd be a 30% chance of nukes being launched @ 1% yearly risk. That's more on the fence for me. On one hand, it's high (and the impact would have been catastrophic!). On the other hand, it prevented NATO / Warsaw Pact mechanized conflict across Europe (which also would have been catastrophic). Tough call. Especially with the uncertainty that, were you to give up your nuclear weapons, there's no guarantee your counterparties would actually do the same. In the 60s foundational position papers, there's definitely a strong thread of "We will commit to MAD not because it's the best approach, but because it's the best zero trust approach."
- 5y ago
- titzer 5y ago...until it doesn't. I would highly recommend a visit to Hiroshima Peace Memorial Museum if you'd like a different perspective on nuclear weapons, rather than a American/western one. And...man, I wish we had that $5.5 trillion back [https://www.brookings.edu/the-hidden-costs-of-our-nuclear-arsenal-overview-of-project-findings/ https://www.brookings.edu/the-hidden-costs-of-our-nuclear-ar...].
- ethbr0 5y agoSee also: Guernica, Warsaw, Dresden, London, Wesel, Tokyo, Hargeisa. Nuclear weapons are bigger explosives. Not different explosives.
- titzer 5y agoEvery one of those cities was bombed for weeks, sometimes months. Hiroshima and Nagasaki were single bombs. They cause massive radiation exposure and fallout. Tens of thousands of people were poisoned to death in the aftermath of the Japanese bombings. You haven't seen the suffering of those people and cannot even imagine the hell they were subjected to. And today, we have weapons hundreds of times as powerful. City-ending megaweapons by the thousands. And yes, nuclear winter is a real thing too; we could very seriously impact the global climate and cause mass extinctions in a couple of hours. A nuclear war is a planet-scale atrocity that will ends billions of human lives. I've been to the Hiroshima museum. Like I said, it will change your perspective. I'm not really all that inclined to continue this interchange given your flippance.
- ethbr0 5y agoIf you want to dismiss 100,000 Tokyo citizens burning to death in two nights because it doesn't fit the narrative of "nuclear is the worst," then that's your decision. But personally, I consider them equally horrific. And as stated previously, if MAD prevents either from happening, the risk seems worth it.
- deleted 5y ago[deleted]
- titzer 5y agoWe cannot allow a missile gap!
- daveslash 5y agoMr. President, we cannot allow a mineshaft gap!
- ljd 5y agoFamous last words. As an AI practitioner, I want us invest in research to manipulate other AI military systems but not use AI ourselves. Take a page from the HFT book.
- tolbish 5y agoI am curious where you would draw the line between "AI military systems" and existing systems in use such as Iron Dome and other algorithm-assisted weapons.
- eternalban 5y agoTwo obvious dimensions are: - agency: the machine makes decisions on its own. - defensive vs offensive action
- Sebb767 5y ago> - defensive vs offensive action So if Iron dome launches a rocket in defense it's okay, but that rocket mustn't use AI to find its target? I see your general point, but that's going to be a line that's very hard to draw.
- tolbish 5y agoPerhaps I'm missing something, but that first dimension seems far from obvious. To continue my Iron Dome example, isn't the system making decisions on its own when intercepting anything it classifies as an incoming ballistic threats?
- eternalban 5y agoThese are obvious dimensions that need to be considered. Of course there can be discussion as to boundaries drawn in that (here 2-d) space as to 'acceptable' and 'unacceptable'. There may be other dimensions as well, such as 'decision making input sources', etc. As to your specific example, while system x may have 'full agency (autonomous, willful)', if it is purely 'defensive' in nature, it may indeed fall within acceptable zone.
- sillysaurusx 5y agoI completely agree, and as an ML researcher, having that opinion feels heretical. So many people are of a “do no harm” mindset that I worry it means “or I won’t work with you.” And I quite like working with various researchers. I mentioned it publicly once, and I’m bracing for the day that some established researcher quote tweets it and says “we do not need this kind of thinking in our community, and we have a duty to exclude it” or some such. I don’t care much though. You seem right, and that’s good enough for me.
- elefanten 5y agoWe need researchers who will follow their moral instincts far more than we need researchers who care about conforming to the sociocultural norm. So, thank you. I guess I shouldn’t just say “researchers” —- it’s what we need in our citizens of humanity.
- mistrial9 5y agoyou are directly contradicting the code of military command, prepare for social credit demotion! </snark>
- qwertox 5y agoThere is a science fiction novel by Stanisław Lem titled Peace on Earth which touches this topic. https://en.wikipedia.org/wiki/Peace_on_Earth_(novel) https://en.wikipedia.org/wiki/Peace_on_Earth_(novel) > The evolution of artificial intelligence has allowed major world powers to sign a rather curious treaty: the Moon is divided into national zones (proportional to each nation's Earth real estate) and all weapons development and production must be moved there to be handled by factories. This is supposed to completely demilitarize Earth, achieving the long-sought dream of world peace. A MAD stabilizing factor is apparently preserved by the ability of countries, in case of war, to quickly ship weapons down from the Moon.
- proggy 5y agoI disagree. While I concede that, if there is credible information that an adversary is developing AI weapons that need to be counterbalanced to reach some sort of Nash equilibrium, simply putting time and effort into the development of any adversarial technology will turn up the temperature in the space, so to speak. More work in this area means more capability, more complexity, more risk of misuse, and a heightened risk of the possibility of additional actors. Look no further than the theory of deterrence, and how it shifted from a tolerable short term strategy during the Cold War (one with few actors), to a high risk strategy as the trove of technology was enlarged, the likelihood of it spreading increased, and the number of active and potential actors increased dramatically also. In my view AI weapons development demands a judicious, steady, paced, and thoroughly informed approach to keep the whole space from burning up —- figuratively and literally.
- elefanten 5y agoHigh risk strategy? Deterrence via MAD is overwhelmingly what keeps the relative peace between the major nuclear powers. It’s not a great situation overall, but you couldn’t uninvent nukes at that point, so it has proven remarkably effective thus far. Same dynamic with AI/ML —- can’t uninvent. If one aggression-minded power has it, balance will be required for stability.
- yboris 5y ago100% disagree. We have treaties that prevent use of gas weapons. We need governments to coordinate and cooperate rather than race each other to murderous death. https://www.stopkillerrobots.org/ https://www.stopkillerrobots.org/
- srcmap 5y agoHow do we enforce any treaties the prevent AI usage?
- mLuby 5y agoWith an AI, specifically an AI that kills younger AIs (called the Great Old 1).
- ethbr0 5y agoA cynic would say that we only banned chemical weapons because they weren't particularly useful in major power combat. See: cluster munition and mine bans
- munk-a 5y agoThat cynic would be wrong since chemical weapons are extremely effective in combat. When the scales aren't even (i.e. something like the vietnam war) the stronger side can afford equipment that allows their soldiers to operate effectively while the other side is unable to respond to aggression greatly lowering expected casualties. With powers on par you still have the factor of well coordinated chemical attacks allowing breakthroughs to be executed much easier. Chemical weapons being banned is an example of a mostly inexplicably altruistic action on the part of major powers. Well, inexplicable if you're a cynic.
- dmitriid 5y ago> That cynic would be wrong since chemical weapons are extremely effective in combat. They are not. It's the most ineffective weapon outside swords and bows. A sibling comment provided the link discussing it: https://acoup.blog/2020/03/20/collections-why-dont-we-use-chemical-weapons-anymore/ https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...
- jldugger 5y ago> it would be tremendously irresponsible to let our adversaries own this space Can't wait for your finished script of Dr Strangelove 2.
- mLuby 5y agoWonder when we'll get to the MAINPART: Militarized Artificial Intelligence Non-Proliferation And Reduction Treaty.
- tablespoon 5y ago>> it would be tremendously irresponsible to let our adversaries own this space > Can't wait for your finished script of Dr Strangelove 2. It's easy to mock and parody MAD, arms races, etc., but certain conditions make them necessary and rational, and those conditions can't easily be changed.
- sitkack 5y agoI have only a single downvote, so I'll leave a comment instead. This will directly lead to an AI arms race. We need to put a pin in this just like chemical weapons. Our civilization might not survive a WW1 level experiment. If we go down this path, and we survive, earth will be the best approximation of hell in heaven.
- alexfromapex 5y agoWe can use AI to control things but having an AI decide to attack or not without human sign off is a terrible terrible idea
- gus_massa 5y agoDo you realize it's the same reason used to start the Manhattan project? Do you realize that only one country nuked civilian cities?
- HideousKojima 5y agoThey already do, as far as I understand it Phalanx CIWS can identify and choose to target an object based entirely on radar and other internal sensors.
- systemvoltage 5y agoAnduril (start up by the founder of Oculus) is also working on AI-defense systems: https://www.anduril.com/ https://www.anduril.com/
- bserge 5y agoHas anyone tried attaching weapons (lasers, tasers, even guns) and a somewhat smart targeting system to a drone yet?
- ethbr0 5y agohttps://m.youtube.com/watch?v=4l0Dh6qJ3RE&t=48s https://m.youtube.com/watch?v=4l0Dh6qJ3RE&t=48s In war, weapon speed triumphs safety concerns. And hypersonic and laser weaponry is only going to decrease that. We've been in a delegated authority regime since the USS Vincennes / Iranian Air Flight 655. There are already shifts towards a delegated-preemption regime in EW (see: efforts to equip EW suites with self-adaptive capability). I can't imagine we won't see the same thing with actual weapons. Sure, put a human in the loop, if there's time. Otherwise...
- criddell 5y agoAnd didn't the Aegis Combat System do that as well?
- nradov 5y agoYes it can. https://www.thedrive.com/the-war-zone/39508/how-the-aegis-combat-system-is-evolving-to-dominate-naval-warfare-of-the-future https://www.thedrive.com/the-war-zone/39508/how-the-aegis-co...
- dragonwriter 5y ago
- athenot 5y agoI'm reminded of that line from Shrek: "Some of you might die, but that's a sacrifice I'm willing to make."
- bovermyer 5y agoIf they're going to go down this route, I demand that they start testing humanoid variants, as well as living tissue over a metal endoskeleton.
- bserge 5y agoOnly if they fix it to sound like a strong Austrian-American.
- lostlogin 5y agoHaving just finished some documentation on cryogen handling, I’m ready. As a further tangent and something that came as news to me - some cryogens can condense oxygen out of the atmosphere and if oil or grease is present there is a risk of fire. Asphyxiation, frostbite, fire.
- sneak 5y agoSounds to me like easy money.
- pibefision 5y agoI saw this in 1983. War games.
- at_a_remove 5y agoOf course, the deaths of innocents will dissolve in the calculus of responsibility this way, as each slice tends toward the infinitesimal. The spec was wrong for those conditions, the optics company we hired sold us something defective, the training set was bad, the review wasn't thorough, we got underpowered CPUs, the targeting system on those pinhead missiles has a known flaw, on and on, until absolutely nobody is at fault. Nobody will court martial a neural net and that'll be the end of it.
- v8dev123 5y agoThe Pentagon generals must be ill informed about AI. I believe they think it as a magic and can do everything correctly. This is very dangerous!! Look at Tesla Self Drive, It literally killed me last night. Weapons are nothing!!
- openasocket 5y agoI probably need to read the report to get more information, but I'm just having trouble visualizing this hypothetical future where we need a swarm of autonomous agents to identify their own targets and threats at all level without any human intervention, and where any manual control would be too limiting. There are circumstances, like CIWS or CRAM systems automatically shooting down incoming munitions, where that comes in handy. I'd also be more comfortable with putting autonomy in the hand of anti-aircraft systems (either SAMs or UAVs). In that case you can program in parameters and I can trust a machine to reliably stick to those parameters. But if we're talking about a machine gun mounted on an unmanned ground vehicle? If you really need to shave off that second or two of human reaction time, that's not a real battle, that's a showdown from a Western movie. I don't think it's too much effort to send a quick picture to a human, or have a human being specify where they can engage and for how long for area suppression. I think it will be multiple decades before AI has progressed to the point where I'd trust it to correctly identify enemy soldiers vs non-combatants as well or better than a human being. Fortunately, I think it will be even longer before that's something we need.
- nradov 5y agoThe Pentagon has already been letting AI control weapons for decades. The Mk 60 naval mine entered service in 1979. It used automatic pattern matching to detect a target and then launched a guided torpedo. Modern AI algorithms are more complex but nothing has fundamentally changed. https://en.wikipedia.org/wiki/Mark_60_CAPTOR https://en.wikipedia.org/wiki/Mark_60_CAPTOR
- neatze 5y agoMore resent technology; javelin missiles, hellfire missiles, and cruise missiles are self-guided and AI powered weapon systems.
- davecheney 5y agoBig WOPR energy
- Havoc 5y agoSeems essentially inevitable to me. Just like AI can play better chess than humans it’ll also fly combat planes better. Dramatically so. It’s only a matter of time till A) they do get much better And B) someone can’t resist the allure