4 ms·

the amount of people seriously considered about some doomsday skynet like AI happening anytime soon is just amazing to me. I'm far more concerned about the naiv

the amount of people seriously considered about some doomsday skynet like AI happening anytime soon is just amazing to me. I'm far more concerned about the naive politicians and those who have their ear. perhaps AI will be generally intelligent, but please for the love of god, let's only consider that scenario what it actually happens.

AtlasBarfed 3y ago
So lets look at a scenario. An LLM is tied to a bunch of APIs as an event response agent. Maybe in some poduck enterprise support group. But... there's tons of other LLMs tied to event response agents, in ... less podunk groups. ... and there's some fancy black box org in the CIA that then employs them as some crisis monitor. and a cascade of false reporting leads to "oh my god there's nukes in flight, and the system has autolaunched nukes in response". What current AI is good at is faking being intelligent, so that they can be used in many many places to substitute for "real intelligence" humans. But... that means dangerous cascades in all likelihood. That isn't skynet, but it is 90% of what the Terminator skynet disaster scenario was: an AI triggered a nuclear exchange.
- endisneigh 3y ago
  this is a great example of the hysteria I'm talking about. You don't need LLM for your scenario to be true. If you're stupid enough to allow nukes to be autolaunched this can be done today with buggy code. no AI necessary.
  - Avicebron 3y ago
    The year is 2030, and ScaryBadLand announces in a shocking revelation at the United country alliance that they have hooked AI to nukes, they declare that any dissent or economic action that can be seen as harming their national interests will result immediate and punishing nuclear attacks, there is no way to warn against when this may occur so all other countries that they have immediately one all future and current negotiations, in this scenario the only option seen is to deploy our own nuclear deterrent of equal capacity to reestablish MAD.
- danaris 3y ago
  The failure here is "hooking a computer to nuclear launches without strong human verification." Has nothing to do with whether the computer is an AI or an IR tripwire.
  - pessimizer 3y ago
    Sounds as preposterous as putting cars and nuclear power plants on the internet for dubious benefit, but here we are.
    - danaris 3y ago
      There are two major differences there, each related to one of the primary reasons (IMO) that people do stupid things like that: Reason 1: Profit motive: No one's going to be making a profit off of hooking an AI up to nukes. Reason 2: For the lulz: This is the kind of thing that happens when you have very broad access to the elements in question. Since access to nukes is very tightly restricted, and their usage controlled by significant parts of governments, it's vanishingly unlikely that either a conscious collective decision will be made to hook up an AI to nukes for the hell of it, or that someone will be able to do so on their own. And yes, I know that there are lost nuclear weapons, which means both that a) there are some out there not under government control, and b) governments can be careless with them—but frankly, any non-governmental actor known to be holding a nuke would have to be considered a top-level threat no matter what they had making the decisions about whether and how to use it.
NumberWangMan 3y ago
> perhaps AI will be generally intelligent, but please for the love of god, let's only consider that scenario what it actually happens. This only makes sense if you think that at that point, it will not be a "the genie is out of the bottle" scenario and already too late to put it back in. And yet that's what people constantly say about current LLMs.
- endisneigh 3y ago
  You could say that about math and the inevitable discovery of linear algebra, industry and thus language models.
handedness 3y ago
Questionable politicians enjoying popular support is not a new thing. AI has more unknowns at this point. But you're right in that the two problems are inseparable. Wisely implementing sensible and effective AI controls will fall onto politicians and statespeople, from legislatures to constrain things domestically, to Presidents who have to lead abroad in forging meaningful and effective treaties.
pessimizer 3y ago
> perhaps AI will be generally intelligent Does AI have to be strong to kill everyone? I can imagine scenarios in which it could be fairly shit, but good enough. > please for the love of god, let's only consider that scenario what it actually happens. Not a great philosophy to guide planning.
- endisneigh 3y ago
  > I can imagine scenarios in which it could be fairly shit, but good enough. Let’s hear these scenarios that aren’t possible without AI.

AtlasBarfed 3y ago

So lets look at a scenario. An LLM is tied to a bunch of APIs as an event response agent. Maybe in some poduck enterprise support group. But... there's tons of other LLMs tied to event response agents, in ... less podunk groups. ... and there's some fancy black box org in the CIA that then employs them as some crisis monitor. and a cascade of false reporting leads to "oh my god there's nukes in flight, and the system has autolaunched nukes in response". What current AI is good at is faking being intelligent, so that they can be used in many many places to substitute for "real intelligence" humans. But... that means dangerous cascades in all likelihood. That isn't skynet, but it is 90% of what the Terminator skynet disaster scenario was: an AI triggered a nuclear exchange.

endisneigh 3y ago
this is a great example of the hysteria I'm talking about. You don't need LLM for your scenario to be true. If you're stupid enough to allow nukes to be autolaunched this can be done today with buggy code. no AI necessary.
- Avicebron 3y ago
  The year is 2030, and ScaryBadLand announces in a shocking revelation at the United country alliance that they have hooked AI to nukes, they declare that any dissent or economic action that can be seen as harming their national interests will result immediate and punishing nuclear attacks, there is no way to warn against when this may occur so all other countries that they have immediately one all future and current negotiations, in this scenario the only option seen is to deploy our own nuclear deterrent of equal capacity to reestablish MAD.
danaris 3y ago
The failure here is "hooking a computer to nuclear launches without strong human verification." Has nothing to do with whether the computer is an AI or an IR tripwire.
- pessimizer 3y ago
  Sounds as preposterous as putting cars and nuclear power plants on the internet for dubious benefit, but here we are.
  - danaris 3y ago
    There are two major differences there, each related to one of the primary reasons (IMO) that people do stupid things like that: Reason 1: Profit motive: No one's going to be making a profit off of hooking an AI up to nukes. Reason 2: For the lulz: This is the kind of thing that happens when you have very broad access to the elements in question. Since access to nukes is very tightly restricted, and their usage controlled by significant parts of governments, it's vanishingly unlikely that either a conscious collective decision will be made to hook up an AI to nukes for the hell of it, or that someone will be able to do so on their own. And yes, I know that there are lost nuclear weapons, which means both that a) there are some out there not under government control, and b) governments can be careless with them—but frankly, any non-governmental actor known to be holding a nuke would have to be considered a top-level threat no matter what they had making the decisions about whether and how to use it.

NumberWangMan 3y ago

> perhaps AI will be generally intelligent, but please for the love of god, let's only consider that scenario what it actually happens. This only makes sense if you think that at that point, it will not be a "the genie is out of the bottle" scenario and already too late to put it back in. And yet that's what people constantly say about current LLMs.

endisneigh 3y ago
You could say that about math and the inevitable discovery of linear algebra, industry and thus language models.

handedness 3y ago

Questionable politicians enjoying popular support is not a new thing. AI has more unknowns at this point. But you're right in that the two problems are inseparable. Wisely implementing sensible and effective AI controls will fall onto politicians and statespeople, from legislatures to constrain things domestically, to Presidents who have to lead abroad in forging meaningful and effective treaties.

pessimizer 3y ago

> perhaps AI will be generally intelligent Does AI have to be strong to kill everyone? I can imagine scenarios in which it could be fairly shit, but good enough. > please for the love of god, let's only consider that scenario what it actually happens. Not a great philosophy to guide planning.

endisneigh 3y ago
> I can imagine scenarios in which it could be fairly shit, but good enough. Let’s hear these scenarios that aren’t possible without AI.