8 ms·

Just wait until somebody think it's a good idea to declare war, based on the advice of an AI. That scares me more than anything. Or launching nuclear weapons b

Just wait until somebody think it's a good idea to declare war, based on the advice of an AI. That scares me more than anything. Or launching nuclear weapons based on the advice of AI (or allowing AI to control them altogether). All out nuclear war has so far been avoided multiple times, because of an actual human making a split second decision at the right moment.

sitkack 7y ago
Human civilization will die an embarrassing death. Maybe we will even earn the equivalent of a Darwin Award on Gwelkklak 10, "Died by Own Avoidable Science Fiction Trope". Maybe the answer to the Fermi Paradox is, "tripped on shoe laces while scoring +10 on the sosh meeds".
- bamboozled 7y ago
  You don’t know that so why bother speculating like you do ?
  - nolok 7y ago
    Are you really saying that people shouldn’t use their imagination and not talk or think about things they are not 100% sure of? Because if that’s what we did then no need to worry about getting back to the Stone Age, we would have never left it anyway... Man’s ability to dream, imagine and explore the unknown both physically and intellectually is what got us where we are.
    - deleted 7y ago
      [deleted]
    - bamboozled 7y ago
      "Human civilization will die an embarrassing death." The language used here isn't speculation, it's absolute. If the author was to prefix the statement using speculative terms then I wouldn't have said a thing. The remark came across as highly negative, unsubstantiated and not useful so I called it out. Edit: Downvotes? You're going to have to do better than that before I change my mind.
      - kiddico 7y ago
        The remark came across as highly negative Probably because it was. It was also a reply to a comment about a hypothetical future, which sort of implies speculation for all child comments. And how can something be absolutely embarrassing? What is and is not embarrassing is an opinion. How would someone word that in a way that fits your requirements?
        bamboozled 7y ago
        The comment doesn’t need to fit my requirements, it surprises me the Internet now feels like it needs to convince people to think a certain way.
        permatech 7y ago
        This is a very heavily moderated forum
savanaly 7y ago
We've almost had several nuclear weapons launched due to flaws in human decision making, two that were launched with the inputs of humans, and a ton of wars launched with the input of humans. What about an AI being involved automatically makes the situation so much worse?
- throwaway2048 7y ago
  We understand how people work and think pretty well, their edgecases, and countermeasures to said edgecases. We don't understand edge cases of AI much at all, and the terrain-space of "possible minds" they might be is vastly broader than that of a human mind, and utterly outside our experience. An AI might work with absolute 100% flawless precision for 100 years, then decided one day to kill everyone. A lot harder to see that coming from a totally alien intelligence than a human one. AI does not think like humans, have human values, or care about what humans care about, that might make them extremely unpredictable and dangerous. There will be no time for iterative development, or learning from our mistakes, a sufficiently advanced AI knows we would try to stop it, and make sure that we never can. https://www.youtube.com/watch?v=tcdVC4e6EV4 https://www.youtube.com/watch?v=tcdVC4e6EV4
- yaur 7y ago
  Look at any of the "flash crashes" of recent years were out of control AI escalated the problem much faster than humans could react to it. Just take that and imagine that there are nuclear weapons involved.
  - kasey_junk 7y ago
    Ok. None of those have had any long term negative outcomes. What was I supposed to imagine?
    - azinman2 7y ago
      A stock dipping isn’t so bad because it can go back up. A nuclear missiles cannot unexplode.
jonbaer 7y ago
Pretty sure they took the "men out of the loop" back in 1983 ... https://www.youtube.com/watch?v=5bF1_PGMAj0 https://www.youtube.com/watch?v=5bF1_PGMAj0
ekianjo 7y ago
Dr Strangelove, Skynet in Terminator, all of this has been talked about many times before. The truth is that already most of nukes are probably automated to ensure the concept of mutually assured destruction works.
- cam_l 7y ago
  But really, as long as everyone believes in MAD, there is really no reason to carry it out. Engineers working on these systems could quite reasonably, and in good faith, install break points and avoid meaningless loss of life. Then the only MAD system which actually needs implementation is if one of those engineers talks..
  - musingsole 7y ago
    But for everyone to believe it, it has to be an at least reasonable belief. Luckily, that's a low-ish bar with fairies being a somewhat common belief too (I have no numbers but want them...). So those engineers for those same good faith reasons install redundancies. Heh, it's a balance that may or may not tip positive (everything working) or neutral (nothing happens) or negative (everything sabotaged by an enemy actor) and we'll only know after the trigger has been pulled.
- acqq 7y ago
  You don't even have to automate all that and still achieve the goal of destroying both the human civilization as we know it: it was already existing in practice, simply by delegating (human) responsibilities and effective since the thermonuclear bombs were stockpiled. It's documented in the book: https://www.bloomsbury.com/us/the-doomsday-machine-9781608196746/ https://www.bloomsbury.com/us/the-doomsday-machine-978160819...
- hodgesrm 7y ago
  > The truth is that already most of nukes are probably automated to ensure the concept of mutually assured destruction works. Why do people say things like this? They aren't. There is still a human-in-the-loop for all systems I'm aware of. The problems with nukes are that (a) the command and control systems can't reliably prevent one or a small number of people from going from going rogue or (b) the 'human' in the loop has to make a quick decision whether to launch based on potentially crappy inputs. The latter case has happened enough in the past that it should be pretty clear it's a serious problem. See https://en.wikipedia.org/wiki/List_of_nuclear_close_calls https://en.wikipedia.org/wiki/List_of_nuclear_close_calls for a list.
make3 7y ago
"Or launching nuclear weapons based on the advice of AI" That's the crazy thing about hypersonic nuclear weapons... A bomb going from Russia to America in under 5 minutes, who else is going to decide than an algorithm... Knowing world peace is partly ensured by mutually assured destruction :s (To be clear, I'm for a moratorium on hypersonic nuclear warfare, for sure.)
wwice 7y ago
https://www.google.ca/amp/s/www.globalresearch.ca/strangelove-redux-us-experts-propose-having-artificial-intelligence-ai-control-nuclear-weapons/5688448/amp https://www.google.ca/amp/s/www.globalresearch.ca/strangelov...
christkv 7y ago
Command and Control: Nuclear Weapons, the Damascus Accident, and the Illusion of Safety is a great book about how close you can get to launching a war unintentionally.
- 6thaccount2 7y ago
  It is also a chilling documentary on Prime and probably YouTube. Just to think how close we came to losing part of Arkansas and Tennessee...and then to find out how common this was.
hooande 7y ago
Just wait until somebody thinks it's a good idea to declare war, based on the advice of someone in their cabinet
flukus 7y ago
If we play fast and loose with the definition of AI we've come close already with operation RYAN (https://en.wikipedia.org/wiki/RYAN https://en.wikipedia.org/wiki/RYAN): > By May of 1986, these binders had evolved into a catalogue of 292 indicators of “signs of tension.” - https://www.wilsoncenter.org/publication/forecasting-nuclear-war https://www.wilsoncenter.org/publication/forecasting-nuclear... I'm not sure how much was computerized but it basically took a bunch of signals and fed them into an algorithm to tell if the US was about to nuke them.
air7 7y ago
Maybe I'm a naive optimist but I doubt that will ever happen (or until we have AGI). We as a species dont really like to give up control. See for example the "Auto Pilot": we have the tech to fly planes safer than human pilots for decades now, yet we don't because we don't like the feeling of not being in control. I can't imagine that a group of people decide to hand control of a WMD to a computer.
- denispal 7y ago
  It may also be the need to blame someone, the pilot in this case, in case of a disaster.

sitkack 7y ago

Human civilization will die an embarrassing death. Maybe we will even earn the equivalent of a Darwin Award on Gwelkklak 10, "Died by Own Avoidable Science Fiction Trope". Maybe the answer to the Fermi Paradox is, "tripped on shoe laces while scoring +10 on the sosh meeds".

bamboozled 7y ago
You don’t know that so why bother speculating like you do ?
- nolok 7y ago
  Are you really saying that people shouldn’t use their imagination and not talk or think about things they are not 100% sure of? Because if that’s what we did then no need to worry about getting back to the Stone Age, we would have never left it anyway... Man’s ability to dream, imagine and explore the unknown both physically and intellectually is what got us where we are.
  - deleted 7y ago
    [deleted]
  - bamboozled 7y ago
    "Human civilization will die an embarrassing death." The language used here isn't speculation, it's absolute. If the author was to prefix the statement using speculative terms then I wouldn't have said a thing. The remark came across as highly negative, unsubstantiated and not useful so I called it out. Edit: Downvotes? You're going to have to do better than that before I change my mind.
    - kiddico 7y ago
      The remark came across as highly negative Probably because it was. It was also a reply to a comment about a hypothetical future, which sort of implies speculation for all child comments. And how can something be absolutely embarrassing? What is and is not embarrassing is an opinion. How would someone word that in a way that fits your requirements?
      - bamboozled 7y ago
        The comment doesn’t need to fit my requirements, it surprises me the Internet now feels like it needs to convince people to think a certain way.
        permatech 7y ago
        This is a very heavily moderated forum

savanaly 7y ago

We've almost had several nuclear weapons launched due to flaws in human decision making, two that were launched with the inputs of humans, and a ton of wars launched with the input of humans. What about an AI being involved automatically makes the situation so much worse?

throwaway2048 7y ago
We understand how people work and think pretty well, their edgecases, and countermeasures to said edgecases. We don't understand edge cases of AI much at all, and the terrain-space of "possible minds" they might be is vastly broader than that of a human mind, and utterly outside our experience. An AI might work with absolute 100% flawless precision for 100 years, then decided one day to kill everyone. A lot harder to see that coming from a totally alien intelligence than a human one. AI does not think like humans, have human values, or care about what humans care about, that might make them extremely unpredictable and dangerous. There will be no time for iterative development, or learning from our mistakes, a sufficiently advanced AI knows we would try to stop it, and make sure that we never can. https://www.youtube.com/watch?v=tcdVC4e6EV4 https://www.youtube.com/watch?v=tcdVC4e6EV4
yaur 7y ago
Look at any of the "flash crashes" of recent years were out of control AI escalated the problem much faster than humans could react to it. Just take that and imagine that there are nuclear weapons involved.
- kasey_junk 7y ago
  Ok. None of those have had any long term negative outcomes. What was I supposed to imagine?
  - azinman2 7y ago
    A stock dipping isn’t so bad because it can go back up. A nuclear missiles cannot unexplode.

jonbaer 7y ago

Pretty sure they took the "men out of the loop" back in 1983 ... https://www.youtube.com/watch?v=5bF1_PGMAj0 https://www.youtube.com/watch?v=5bF1_PGMAj0

ekianjo 7y ago

Dr Strangelove, Skynet in Terminator, all of this has been talked about many times before. The truth is that already most of nukes are probably automated to ensure the concept of mutually assured destruction works.

cam_l 7y ago
But really, as long as everyone believes in MAD, there is really no reason to carry it out. Engineers working on these systems could quite reasonably, and in good faith, install break points and avoid meaningless loss of life. Then the only MAD system which actually needs implementation is if one of those engineers talks..
- musingsole 7y ago
  But for everyone to believe it, it has to be an at least reasonable belief. Luckily, that's a low-ish bar with fairies being a somewhat common belief too (I have no numbers but want them...). So those engineers for those same good faith reasons install redundancies. Heh, it's a balance that may or may not tip positive (everything working) or neutral (nothing happens) or negative (everything sabotaged by an enemy actor) and we'll only know after the trigger has been pulled.
acqq 7y ago
You don't even have to automate all that and still achieve the goal of destroying both the human civilization as we know it: it was already existing in practice, simply by delegating (human) responsibilities and effective since the thermonuclear bombs were stockpiled. It's documented in the book: https://www.bloomsbury.com/us/the-doomsday-machine-9781608196746/ https://www.bloomsbury.com/us/the-doomsday-machine-978160819...
hodgesrm 7y ago
> The truth is that already most of nukes are probably automated to ensure the concept of mutually assured destruction works. Why do people say things like this? They aren't. There is still a human-in-the-loop for all systems I'm aware of. The problems with nukes are that (a) the command and control systems can't reliably prevent one or a small number of people from going from going rogue or (b) the 'human' in the loop has to make a quick decision whether to launch based on potentially crappy inputs. The latter case has happened enough in the past that it should be pretty clear it's a serious problem. See https://en.wikipedia.org/wiki/List_of_nuclear_close_calls https://en.wikipedia.org/wiki/List_of_nuclear_close_calls for a list.

make3 7y ago

"Or launching nuclear weapons based on the advice of AI" That's the crazy thing about hypersonic nuclear weapons... A bomb going from Russia to America in under 5 minutes, who else is going to decide than an algorithm... Knowing world peace is partly ensured by mutually assured destruction :s (To be clear, I'm for a moratorium on hypersonic nuclear warfare, for sure.)

wwice 7y ago

https://www.google.ca/amp/s/www.globalresearch.ca/strangelove-redux-us-experts-propose-having-artificial-intelligence-ai-control-nuclear-weapons/5688448/amp https://www.google.ca/amp/s/www.globalresearch.ca/strangelov...

christkv 7y ago

Command and Control: Nuclear Weapons, the Damascus Accident, and the Illusion of Safety is a great book about how close you can get to launching a war unintentionally.

6thaccount2 7y ago
It is also a chilling documentary on Prime and probably YouTube. Just to think how close we came to losing part of Arkansas and Tennessee...and then to find out how common this was.

hooande 7y ago

Just wait until somebody thinks it's a good idea to declare war, based on the advice of someone in their cabinet

flukus 7y ago

If we play fast and loose with the definition of AI we've come close already with operation RYAN (https://en.wikipedia.org/wiki/RYAN https://en.wikipedia.org/wiki/RYAN): > By May of 1986, these binders had evolved into a catalogue of 292 indicators of “signs of tension.” - https://www.wilsoncenter.org/publication/forecasting-nuclear-war https://www.wilsoncenter.org/publication/forecasting-nuclear... I'm not sure how much was computerized but it basically took a bunch of signals and fed them into an algorithm to tell if the US was about to nuke them.

air7 7y ago

Maybe I'm a naive optimist but I doubt that will ever happen (or until we have AGI). We as a species dont really like to give up control. See for example the "Auto Pilot": we have the tech to fly planes safer than human pilots for decades now, yet we don't because we don't like the feeling of not being in control. I can't imagine that a group of people decide to hand control of a WMD to a computer.

denispal 7y ago
It may also be the need to blame someone, the pilot in this case, in case of a disaster.