3 ms·

I think this is the biggest risk we're facing. Not necessarily ASI deciding to kill us off, but AI weapon proliferation triggering a hot war that goes nuclear.

I think this is the biggest risk we're facing. Not necessarily ASI deciding to kill us off, but AI weapon proliferation triggering a hot war that goes nuclear. Or something like an AI weapon-based broad spectrum "digital first strike" intended to disable an adversary's nuclear weapons. Imagine how well that would go down. Digital equivalents of nuclear weapons in the hands of people incompetent enough to fail rescuing hostages from Iran in the 70s. Ukraine is too close for comfort as it is. Throw AI weapons in there? Man. Not the future I wanted to see.

rl3 3y ago
>Digital equivalents of nuclear weapons in the hands of people incompetent enough to fail rescuing hostages from Iran in the 70s. Terrible analogy. I recall reading one of the soldiers who died on that mission had an American flag strapped to his chest. Despite the failure and loss of life, Carter authorized another rescue mission that never happened. The engineering and planning that went into both were staggering. What arose from the aftermath was the most elite helicopter aviation regiment in the world. Oh, and the US also had nuclear weapons for some thirty five years already by the time that mission had taken place.
- king_magic 3y ago
  I contend it's a perfect analogy. Entity A decided to try thing to get things back from Entity B. Entity A did not think the plan through nearly well enough to possibly conclude that the action was unwise, attempted it anyway, and it went real bad real fast.
  - rl3 3y ago
    I suppose it's just a bizarre choice in context of a nation's suitability to possess AI weapons. Everyone participating in that mission (including planning) knew it was a highly risky endeavor. It wasn't a nuclear first strike. In my opinion, the fact that another (even crazier) rescue mission was green-lit and worked up in the aftermath embodies the finest qualities of the United States. That said, I see your point. Things can look good enough to say "go" and it all goes to hell. Trying again isn't necessarily an option with AI. Unfortunately, given the state of current nuclear treaties, nonproliferation agreements or weapon bans for AI are probably already dead on arrival.
wudangmonk 3y ago
I do not think these "AI" programs are an issue at all. The way I see it, the only problem is that one side suddenly had the bright idea that a nuclear war could be won, these toy programs have no role in any of that. Can anyone seriously imagine a 'digital first strike' without AGI? or are we talking about unpatched wordpress 'nuclear digital strikes' here?.
- king_magic 3y ago
  > Can anyone seriously imagine a 'digital first strike' without AGI? I don't think you need strong AI to massively enhance suites of hacking tools, design misinformation campaigns that root out and target specific individuals, etc. all with the ultimate goal of disrupting your adversary's nuclear command & control structures just long enough to launch an actual first strike? Unlikely? Sure. Possible? Why not, with enough compute?

rl3 3y ago

>Digital equivalents of nuclear weapons in the hands of people incompetent enough to fail rescuing hostages from Iran in the 70s. Terrible analogy. I recall reading one of the soldiers who died on that mission had an American flag strapped to his chest. Despite the failure and loss of life, Carter authorized another rescue mission that never happened. The engineering and planning that went into both were staggering. What arose from the aftermath was the most elite helicopter aviation regiment in the world. Oh, and the US also had nuclear weapons for some thirty five years already by the time that mission had taken place.

king_magic 3y ago
I contend it's a perfect analogy. Entity A decided to try thing to get things back from Entity B. Entity A did not think the plan through nearly well enough to possibly conclude that the action was unwise, attempted it anyway, and it went real bad real fast.
- rl3 3y ago
  I suppose it's just a bizarre choice in context of a nation's suitability to possess AI weapons. Everyone participating in that mission (including planning) knew it was a highly risky endeavor. It wasn't a nuclear first strike. In my opinion, the fact that another (even crazier) rescue mission was green-lit and worked up in the aftermath embodies the finest qualities of the United States. That said, I see your point. Things can look good enough to say "go" and it all goes to hell. Trying again isn't necessarily an option with AI. Unfortunately, given the state of current nuclear treaties, nonproliferation agreements or weapon bans for AI are probably already dead on arrival.

wudangmonk 3y ago

I do not think these "AI" programs are an issue at all. The way I see it, the only problem is that one side suddenly had the bright idea that a nuclear war could be won, these toy programs have no role in any of that. Can anyone seriously imagine a 'digital first strike' without AGI? or are we talking about unpatched wordpress 'nuclear digital strikes' here?.

king_magic 3y ago
> Can anyone seriously imagine a 'digital first strike' without AGI? I don't think you need strong AI to massively enhance suites of hacking tools, design misinformation campaigns that root out and target specific individuals, etc. all with the ultimate goal of disrupting your adversary's nuclear command & control structures just long enough to launch an actual first strike? Unlikely? Sure. Possible? Why not, with enough compute?