8 ms·
US experts propose having AI control nuclear weapons
- umeshunni 7y agoLink to the actual proposal rather than the clickbait article: https://warontherocks.com/2019/08/america-needs-a-dead-hand/ https://warontherocks.com/2019/08/america-needs-a-dead-hand/
- carlmr 7y agoI was a bit disappointed that while they did cite this they didn't link to it.
- zipwitch 7y agoFrom the fine article you linked (for which link I thank you):"Time compression has placed America’s senior leadership in a situation where the existing NC3 system may not act rapidly enough. Thus, it may be necessary to develop a system based on artificial intelligence, with predetermined response decisions, that detects, decides, and directs strategic forces with such speed that the attack-time compression challenge does not place the United States in an impossible position." Had the Soviet Union had such a system in place in September, 1983, human civilization would not be around for us to be having this discussion. If the situation is so dire as to call for serious discussions about AI control of nuclear weapons, it is dire enough for us to initiate, sponsor, and encourage global discussion over massive multi-lateral nuclear weapons reduction.
- deleted 7y ago[deleted]
- fxj 7y agoBTW: Is one of the authors the same guy as in this article? https://www.jqpublicblog.com/navy-officer-director-of-usaf-school-went-to-jail-over-false-child-sex-abuse-claim/ https://www.jqpublicblog.com/navy-officer-director-of-usaf-s...
- JTechno 7y agoTerminator hasn't taught them nothing?
- mnm1 7y agoI think Congress needs to pass legislation banning such an insane, stupid idea. And we should stop calling it "artificial intelligence," as if some software is actually intelligent comparable to humans. This is just software and as with all software, it has bugs. Trusting it to not start a nuclear war is essentially saying, "this software, which we can't even explain how it works (cause ai), is bug free." Insane. This whole ai movement is dangerous but not because something like Hal or the terminator will be created but because stupid people like these will convince the government to put such weapons under the control of buggy software no one, including its creators, understands with horrific consequences. All so we can make sure that the whole world is destroyed in case of nuclear war. The insanity is truly mind boggling.
- inimino 7y agoCongress isn't gonna pass legislation to ban the military from suggesting stupid ideas, or the executive branch from approving them, because that itself would be a stupid idea that reduces American defence readiness and suppresses worthwhile ideas as well as bad ones. > as if some software is actually intelligent comparable to humans How's your Go game coming along over there? (Have you ever even played? Because if you had I think you'd have more respect for the intelligence required (for a human) to succeed even moderately at the game.) > not because something like Hal or the terminator will be created Actually it is pretty dangerous for those reasons too.
- mnm1 7y agoPlaying go or chess or any other game isn't intelligence. The computer isn't intelligent. It's programmed for one task and one task only. That's great programming and use of statistics etc. but that's not intelligent. Anyway, if you're worried about terminator and Hal like beings, it's futile to discuss intelligence when you are living in a science fiction world. In our world, those things do not exist and there is no sign that they will exist anytime soon. What does exist and is scary is bug ridden software that controls weapons. That doesn't make for great science fiction though.
- 7y ago
- danielmg 7y agoWhy not remove PAL* and delegate authority to local commanders? I'd rather take the risk of trusting a human over a human designed, highly complex computer system. In the UK, due to the lack of land-mass to absorb a strike, the PM issues final orders to the commanders of the Trident subs. This is part of the deterrence strategy. * https://en.wikipedia.org/wiki/Permissive_Action_Link https://en.wikipedia.org/wiki/Permissive_Action_Link https://en.wikipedia.org/wiki/Letters_of_last_resort https://en.wikipedia.org/wiki/Letters_of_last_resort
- michaelanckaert 7y agoThe article on Letters of Last Resort made me think how absurd democracy can be at times. It looks like PM May wrote her Letters of Last Resort, and then Johnson did the same. But from the looks of it, new Letters of Last Resort could be written pretty soon. Just thinking of those sub commanders, receiving new letters last week and then getting another set of letters just a couple of weeks later.
- xkcd-sucks 7y agoIn the past 100 years, there have been several incidents where humans have averted war by refusing to follow orders. There have been far fewer incidents of humans causing nuclear war by either following or disobeying orders
- LyndsySimon 7y ago> There have been far fewer incidents of humans causing nuclear war by either following or disobeying orders True - but if you add the words "nearly" and "allegedly", then it's not such a rosy picture. The book "Red Star Rogue" proposes the idea that the Soviet submarine K-129 was commandeered and attempted to launch a first strike on the United States in 1968. She was very far off her expected course when sunk, and allegedly there was a fail-safe system that scuttled the boat when an unauthorized attempt was made to launch her missiles.[1] The US's nuclear weapons were capable of being used by a single individual for quite some time, until the 1960s. Even with the advent of the Permissive Action Link, until 1977 the 8-digit arming code was - seriously - "00000000". They were worried they wouldn't have the code available if the weapons were needed.[2] There are numerous "close calls" that occurred, but those two came to mind right away in the context of human beings causing or having nearly caused a nuclear attack by disobeying orders in particular. 1: https://www.kobo.com/us/en/ebook/red-star-rogue https://www.kobo.com/us/en/ebook/red-star-rogue 2: https://en.wikipedia.org/wiki/Permissive_Action_Link#Development_and_dissemination https://en.wikipedia.org/wiki/Permissive_Action_Link#Develop...
- Anarch157a 7y agoČapek warned us of this in 1921. Give a machine inteligence enough to be dangerous, without the emotional safeguards of humans and disaster is certain to follow. AI controlled weapons... Holy shit... Stupid ideas like that makes contemplate moving to the middle of the Pantanal or Amazon jungle to live of the land, away from all this madness.
- vincnetas 7y agonice try skynet.
- dfawcus 7y agoSo - Skynet or WOPR?
- FakeComments 7y agoAllied Mastercomputer.
- pdkl95 7y agoThat might be one of the worst ideas I've ever heard. Do you really want to be near a nuclear weapon that is controlled by an opaque, unexplainable process? Even without the insane unobservable complexity of machine learning, do you want to be near a nuclear weapon co9ntrolled by a simpler, theoretically explainable expert system "AI"? Safety critical systems - including nuclear bombs - need to be simple, with realistically-understandable deterministic behavior. The answer to "what would our nuclear weapons do in ${any_hypothetical_situation}" should never be "I don't know". Even worse, if an "AI" is involved (by any definition of "AI"), that means the control system involves a Turing complete language. Asking questions about the behavior of the control system for a given set of inputs probably shouldn't require solving the Halting Problem. edit: I suspect these so-called experts need to read the report[1] from the Therac-25 investigation. [1] http://sunnyday.mit.edu/papers/therac.pdf http://sunnyday.mit.edu/papers/therac.pdf
- arcticbull 7y ago> Do you really want to be near a nuclear weapon that is controlled by an opaque, unexplainable process? You mean humans? Jokes aside this idea is awful.
- nighthawk648 7y agoLol... terminator dark fate sub 1, the parallel. Never before has a movie shown the events that are simultaneously manifested in reality. Come on down this fall it’s going to be scorching out. I mean the sky will literally be on fire as your mind explodes from both radiation and the insane surelism that was terminator!
- 7952 7y agoAccording to MAD principles a "safe" deterrent is less effective. Because your adversary must always think you are capable of delivering a retaliation. A perfectly safe system, may prevent that retaliation and invalidate the detterent effect. This issue applies to all technology and human systems connected with nuclear weapons. The outcome of nuclear war is so horrific that no technoogy would ever be good enough. The solution is to eliminate them entirely and stop creating such insane choices.
- Peej255 7y agoColossus: The Forbin Project (1970) addressed this method. https://www.imdb.com/title/tt0064177/ https://www.imdb.com/title/tt0064177/
- jacquesm 7y agoOf all the stupid proposals I've seen over a lifetime this one has to be the worst of all. So much for the 'experts'.
- dogma1138 7y agoTerminator 6 marketing is getting out of hand.
- mozey 7y agoWas the "expert" suggesting this perhaps also same the AI?
- mlang23 7y agoMaybe people should watch more old movies from the 20th century. This reads like satire, but its true!
- mc32 7y agoAutomate defensive systems, don’t automate the offensive launch of nuclear weapon warheads. Whoever came up with that idea should be fired. You have to know what the taboos are, what lines not to cross. What’s next, simulated casualties are exchanged for actual casualties? I think one star trek series had a plot reaching that logical conclusion.
- spsrich2 7y agoCome on it's not as if there is any chance something might go wrong after all!
- ekianjo 7y agoIs this a modern version of Skynet?
- omarforgotpwd 7y agofire them all
- lwhi 7y agoStep 1 .. Connect nuclear weapon to network. Step 2 .. End civilisation.
- a3n 7y agoThe problem with AI is that it doesn't get cold feet, and can't intuit the likely truth against the present "facts." https://duckduckgo.com/?q=Russian+officer+prevented+wwiii&t=fpas&ia=web https://duckduckgo.com/?q=Russian+officer+prevented+wwiii&t=...
- icebraining 7y agoI'm not saying it's a good idea, but software can and has been written to evaluate the likelihood of the validity of certain facts using heuristics. Petrov said the official training taught that an American first attack would be massive, not just a few missiles; that same knowledge could be used in an algorithm.
- Nasrudith 7y agoDoesn't that suggest the worst possible act of mass slaughter in that system would be to get a "massive" array of fake missle drones or dud missles from the direction of a nuclear power to spark dual MAD strikes from a third party? It would shift it to a hackable spoofable system. Said bad actor could state a "technically I didn't kill millions/billions you did" and be right in a way.
- icebraining 7y agoYes, but that's a failure mode of any system, including with humans: a realistic false flag attack can provoke a counter-attack against your real target. The solution is to improve your capacity of distinguishing true attacks, regardless of whether the analysis is made by a human or software.
- Nasrudith 7y agoI suspect the real danger of the automated system is speed and with it being beyond the ability of others to counter it - which means the system may overreact immediately with nothing to be done to stop it in matters of ultimate stake.
- fauigerzigerk 7y agoBetter hope that zero shot learning works. Otherwise it's one shot annihilation.
- inimino 7y agoquote from the actual article: > Unlike the game of Go, which the current world champion is a supercomputer, Alpha Go Zero, that learned through an iterative process, in nuclear conflict there is no iterative learning process.
- tempodox 7y agoThis proposal is so obviously and completely insane that it would only make sense as a sort of distraction. I wonder from what.
- credit_guy 7y agoMaybe it’s some form of argument where you present three hard-to-swallow options (“Admittedly, each of the three options — robust second strike, preemption, and equivalent danger — has drawbacks” [1]), and then you pretend to offer a better alternative, which is actually so preposterous that the first three options start looking perfectly reasonable. [1]https://warontherocks.com/2019/08/america-needs-a-dead-hand/ https://warontherocks.com/2019/08/america-needs-a-dead-hand/
- alfiedotwtf 7y agoI want off this planet. Software engineers: ... Politicians: AI all the things
- dublor 7y agopeople who think AI can't hurt seems to neglect human stupidity
- dusted 7y agoSkyNet? :D
- dimitar 7y agoI don't see how faster ballistic missiles change the game. In the past you still could do surprise attacks and example eliminate the chain of command, or the lines of communication. While there are reports that suitcase nuclear devices were developed. However what ultimately fixed this issue in the 60s was the adoption of nuclear submarines - you can surprise the adversary, but because you cannot find the subs the retaliatory strikes are assured.
- imvetri 7y agoAI do not have a thinking mind like humans do. Developer/ Researcher: Hold down boy, we can do that. Me: Can you create with a goodness in mind? Developer/ Researcher: Oh yeah, we can train it speaking to a good mind. Me: Wrong!. you gotta ask whats a good mind first!. Define it, you will stand corrected and you will realise whole concept of tech is absurd.
- inimino 7y agoTheory: they know something about AI that we don't and because of what they know, decoded to scare all the world's AI practitioners into getting their shit together at the same time. Alternate theory: they are idiots. Alternate theory number two: they know something about AI and a lot about nuclear doctrine that most people don't, and they are actually onto something...
- nighthawk648 7y agoEven then it is irresponsible. Dogma is rapid in industries filled with compartmentalizations. To best avert disaster tech needs to first be open to the public and submisiable to regulation.
- inimino 7y ago> Dogma is rapid in industries filled with compartmentalizations. Yes 100% agreed. > To best avert disaster tech needs to first be open to the public and submisiable to regulation. But we know that's never gonna happen with defence tech so...
- sweeneyrod 7y agoThis must surely be an instance of "add 'AI' to whatever topic" in order to get attention, rather than a serious proposal, right? I'm sure we'll see "put nuclear weapons on blockchain" soon.
- jacknews 7y agoWhat could go wrong? https://www.rottentomatoes.com/m/colossus_the_forbin_project https://www.rottentomatoes.com/m/colossus_the_forbin_project
- lostjohnny 7y agoI came here to see if someone already mentioned Colossus. Funny trivia: there are parts of that movie shot in Rome, my city; you can see the Vatican, San Peters, the Orange Garden, the Janiculum and Isola Tiberina, specifically the hospital "Fate bene fratelli", where I was born. The Americans meet there with Russians, joining forces to stop Colossus, and you can see written on a wall a mirrored "W Lenin". The writing was actually there before they shot the movie, it was not added on purpose and stayed there until not long ago. https://i.imgur.com/wIha0mr.png https://i.imgur.com/wIha0mr.png
- jacknews 7y agocool! great classic movie, beautiful city!
- heyflyguy 7y agoShall we play a game?
- _iyig 7y agoFor the unfamiliar: https://youtu.be/NHWjlCaIrQo https://youtu.be/NHWjlCaIrQo EDIT: Well sue me for footnoting a cultural reference, I guess.
- ramblerman 7y agoDoes the term expert as used in the media still convey any meaning?
- liability 7y agoAlternate title: Men erroneously considered experts issue a public correction.
- b3orn 7y agoThe "experts" in this case seem to be defense experts, not AI experts.
- de_watcher 7y agoThey use it when "scientists" looks out of place.
- mschuster91 7y agoWhat the flying fuck? We don't even understand how modern AI algorithms actually work and how they decide stuff. And we want to give such blackboxes the power to annihilate all life on Earth?
- jacobwilliamroy 7y agoI think the offense industry will read your comment and get all big-headed about how much stuff they think they can kill. They can definitely kill off homo sapien, but earth life will go on. There's still a few billion year left in the sun's main sequence. Maybe something more compassionate will evolve out of the leftovers.
- klyrs 7y ago> Maybe something more compassionate will evolve out of the leftovers. My money is on squid... and that they'll be no better than us.
- melling 7y agoIn the near future, we are going to have autonomous battlefield weapons: https://www.theatlantic.com/technology/archive/2019/09/killer-robots-and-new-era-machine-driven-warfare/597130/ https://www.theatlantic.com/technology/archive/2019/09/kille...
- ddtaylor 7y agoThis article seems like rage bait.
- credit_guy 7y agoThe proposal is indisputably harebrained, but the question is why the authors decided to publish it? One of the authors is a former US Air Force colonel. And in particular why they choose to propagate falsehoods like "Hypersonic cruise missiles are powered all the way to their targets using an advanced propulsion system called a SCRAMJET. These are very, very, fast. You may have six minutes from the time it’s launched until the time it strikes." Lest it be any confusion, scramjets are fast, but are nowhere as fast as ICBM's (Mach 5-10 vs Mach 20-23).
- LyndsySimon 7y ago> One of the authors is a former US Air Force colonel. I was curious, so I looked it up. There are about 3,300 colonels in the US Air Force: https://www.statista.com/statistics/239365/total-military-personnel-of-the-us-air-force-by-grade/ https://www.statista.com/statistics/239365/total-military-pe... > Lest it be any confusion, scramjets are fast, but are nowhere as fast as ICBM's I agree that's weird phrasing and incorrect on its face. That said - as I understand it, cruise missiles don't have the launch signature ICBMs have. While it would take longer for a cruise missile to travel the same distance as an ICBM, it might in fact give the target country less time to react if they aren't detected until they are well on their way.
- objectivetruth 7y agoThe point is that enemy can launch a "slower" hypersonic cruise missile from a surfacing missile sub or a disguised freighter 100 miles off the US coast. The point of hypersonic cruise missiles is NOT to replace ICBMs in land-based silos a hemisphere away. It's to replace "normal" supersonic cruise missiles carried on subs, missile boats, etc.
- codecrusade 7y agologistic regression based nuclear missiles eh
- ohiovr 7y agoThe experts should go to the movies some time.
- Havoc 7y agoTried that one already. Works great...if your name is Hollywood and you're making a action movie
- amai 7y agoSee also the Russian Perimeter system: https://en.wikipedia.org/wiki/Dead_Hand https://en.wikipedia.org/wiki/Dead_Hand