4 ms·
Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and
by jerf 8mo ago
Some of the most reassuring and scariest things you can read are about the incidents that have already occurred where computers said "launch all the nukes" and the humans refused. On the one hand, good news! We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. Bad news, it's been skin-of-our-teeth multiple times already.
https://www.warhistoryonline.com/cold-war/refused-to-launch-nuclear-missiles.html https://www.warhistoryonline.com/cold-war/refused-to-launch-... - This isn't even the incident I was searching for to reference! This one was news to me.
https://en.wikipedia.org/wiki/Stanislav_Petrov#Incident https://en.wikipedia.org/wiki/Stanislav_Petrov#Incident - This is the one I was looking for.
- blibble 8mo ago> We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. previously no-one had spent trillions of dollars trying to convince the world that those computers were "Artificial Intelligence"
- nine_k 8mo agoThey had to do with "state-of-the-art radars", "military-grade communication systems", etc.
- lesuorac 8mo agoYeah but they dealt with sota and military-spec systems their entire career and they know that it just means "lowest bidder".
- Barrin92 8mo agoof course they did. That's the literal topic of War Games (1983). You should actually be somewhat reassured that we aren't living during the era of Dr. Strangelove where you had characters in the military industrial complex who were significantly more insane when it came to the beliefs of what computer systems and nukes can do. There was a time when people wanted to dig tunnels with nukes https://en.wikipedia.org/wiki/Project_Plowshare https://en.wikipedia.org/wiki/Project_Plowshare
- idiotsecant 8mo agoDigging tunnels with nukes sounds better to me than shooting them at each other!
- 21asdffdsa12 8mo agoAll you need is a elephants foot burning into the ground and a way to direct it via partial cooling..
- roenxi 8mo ago> There was a time when people wanted to dig tunnels with nukes The article seems to be about mining rather than tunnelling. And the issue with the idea being? We also dig using explosives, there isn't an in-principle problem. Reading the wiki article it looks like the yields were excessive, but at the end of the day mining involves the use of things that go boom. It is easy to imagine small nukes having a place in the industry.
- throwaway27448 8mo agoThe more available they are, the more likely they are to be used on humans.
- Barrin92 8mo ago>And the issue with the idea being? See the 'rationale' section of the article. The point of it was to rebrand nuclear weapons as multi-use 'peaceful' tools and drive acceptance for nuclear weapons programs. Which was a pretty standard tactic of military projects during the cold war.
- lozf 8mo ago> That's the literal topic of War Games (1983) Probably ran out of tokens before the end, during training. "How about a nice game of Chess?"
- TitaRusell 8mo agoMacArthur recommend turning the Korea-China border into a nuclear wasteland to prevent further Chinese troop movements. Thank god everyone has nukes now it keeps people sane!
- escapecharacter 8mo agoOr "alignment" which means "let's ensure the AIs recommend launching nukes only when it makes sense to, based on our [assumed objective] values"
- badRNG 8mo agoWe shouldn't be the least bit surprised no human has complied so far. If they had, then we wouldn't be having this conversation. For all we know, there may be a vast multiverse of universes some with humans and we would only find ourselves having this conversation in one of the universes where no human pressed the button.
- thfuran 8mo agoBy that logic, it may actually be pretty common for rabbits to swallow the sun. We just haven't seen it happen because we're in the wrong universe and would've died it it happened in ours.
- sir0010010 8mo agoAnthropic Principle
- flr03 8mo agoI hope humans in charge are as wise now as they were then.
- phs318u 8mo agoSurely that’s the definition of a quixotic hope.
- ge96 8mo agoI briefly got into a "rabbithole" of watching videos about trying to intercept BMs and glide hypersonic weapons, pretty interesting, decoys deployed in space... the outcome seemed to be not good, can't guarantee 100% interception
- compass_copium 8mo agoA missile will always be cheaper than a missile interceptor, and the interceptor will never be a 1:1 kill. Building a missile interceptor system ia a good way to get your strategic opponent to build a bigger stockpile.
- LorenPechtel 8mo agoDisagree on always being cheaper. Military planners are obsessed with the best weapons, such interceptors are pricey. But look at Israel: Iron Dome. ~$50k/shot. They deliberately built a dumb SAM because it was designed to go against dumb opponents--objects falling freely on a ballistic trajectory. While they are usually facing light stuff that isn't even worth that they have successfully engaged longer range stuff that costs many times what the interceptor costs. Overall, though, the offense always wins this one because interceptors can only protect a limited area whereas missiles can go anywhere.
- compass_copium 8mo agoIron Dome is a great example of my point. It is a $50k interceptor designed to take out a propane tank with a rocket strapped to it, not a real ballistic missile like a Scud. Patriot missiles ($7MM) take out Scuds ($3MM).
- paxys 8mo ago> We have prior art that says humans don't just launch all the nukes just because the computers or procedures say to. This relies on processes being in place to ensure that a human will always make the final decision. What about when that gets taken away?
- trehalose 8mo agoI find it hard to imagine that the people in a position to kill those processes could ever be that zealously in love with AI, but recent events have given me a tiny bit of doubt.
- bryanrasmussen 8mo agoI mean in the cases where higher command has said launch your nukes and lower command has not done so and everything turned out ok, I think to higher command it of course is good it worked out this time but it certainly also looks like a problem with the system that needs to be automated away. So a computer that will launch all nukes when ordered must look very appealing in contrast to humans who might save humanity.
- rurp 8mo agoYeah... the more I learn about nuclear weapon history the more I discount our society's long term viability. There are way too many frighteningly close calls already, and there are probably others that aren't widely known. It's not just nukes that are concerning either. If we're unable to mitigate such a visceral existential risk, we aren't going to do any better with more subtle vulnerabilities. AI of course accelarates some risks and introduces new ones. This doesn't mean we're doomed or anything, but if I had a magic portal to peer a few hundred years in the future and saw humans had been obliterated by nukes, runaway AI, some generated supervirus, runaway climate change, or some other manufactured risk I would be completely unsurprised.