3 ms·

> The fastest route to a rogue AI system is if a human with the appropriate technical skills and means intentionally builds it with the objective of destroying

> The fastest route to a rogue AI system is if a human with the appropriate technical skills and means intentionally builds it with the objective of destroying humanity or a part of it set explicitly as a goal. If rogue AI(s) emerges then I suspect it will come out of military r&d, rather than academia or industry. The deep resources already exist. The motivation will be the need to build it before $enemy$ does, rather than to "destroy humanity", but I reckon it creates more momentum than the profit motive when coupled to state power. Its the same pressure that got us nukes and mutal assured destruction. Military secrecy will pretty much guarantee inadequate oversight. Military discipline will ensure that most (but maybe not quite all) of the people working on it will suppress their ethical concerns, or will have none to begin with. The fact that the AI will necessarily be trained to kill (which will necessitate a lack of hard-wired moral constraint) will help too, I guess. I suspect there are already things going on in anonymous mil/defence labs that would give many people nightmares.

RandomLensman 3y ago
And that AI would do that how? It would need some humans to provide the means for destruction - which means we are back to a state we already have, i.e., humans having tools of pretty complete destruction.
- andyjohnson0 3y ago
  > And that AI would do that how? Assuming an AI that manages to (or is designed to) escape its containment: Armed, remote-piloted drones already exist. So do nuclear and conventional weapons that are targeted and launched by software-based c&c systems. Inject realistic fake attack sequences into BMEWS and whatever the Russian and Chinese systems are called and watch the meat bags do the work. Nuclear power stations. GPS spoofing to collide aircraft and autonomous vehicles over cities. Reality disruption by extremely large scale disinformation, combined with targeted food supply, utility, and comms system outages. Socially-engineer humans into causing death and destruction using bribes/threats or mental destabilisation thru engineered conspiracy theories. Etc.
  - RandomLensman 3y ago
    How does a drone exit its storage unit? How does an AI arm a nuclear weapons? How does GPS spoofing crash an aircraft? How does AI do something to a nuclear power plant? Can we not shut down the internet if we needed to combat misinformation? AI would need to overcome a fair amount of systems, both physical as well as software for a lot of these things. It needs a lot of "thinkism" to make that work.
    - andyjohnson0 3y ago
      > How does a drone exit its storage unit? Because the AI fabricated a target package doe it? > How does an AI arm a nuclear weapons? How does GPS spoofing crash an aircraft? How does AI do something to a nuclear power plant? By infiltrating non-internet networks? > Can we not shut down the internet if we needed to combat misinformation? A lot of people seem to enjoy soaking themselves in conspiracy and disinformation. No my guess would be no, we wouldn't ahut it down. > AI would need to overcome a fair amount of systems, both physical as well as software for a lot of these things. It needs a lot of "thinkism" to make that work. Thats a fair criticism. Its unlikely at present, but as I asserted in my original comment, military exploitation of AI seems inevitable (to me) and the forces at work are irresistible.
      - RandomLensman 3y ago
        You wave away of lot of physical action that is needed in a lot of cases - people are not as trusting in software as you think in some areas. It might never happen that AIs can do all this and/or that defense AIs severely limit what offensive AIs can do.
theaussiestew 3y ago
This sounds like the plot of Terminator and the birth of Skynet
stainablesteel 3y ago
its gonna come out of 4chan
underlipton 3y ago
>Military or corporation Why not both? The Horizon franchise's lore has a "fun" example of the military-industrial complex converging on Armageddon.

RandomLensman 3y ago

And that AI would do that how? It would need some humans to provide the means for destruction - which means we are back to a state we already have, i.e., humans having tools of pretty complete destruction.

andyjohnson0 3y ago
> And that AI would do that how? Assuming an AI that manages to (or is designed to) escape its containment: Armed, remote-piloted drones already exist. So do nuclear and conventional weapons that are targeted and launched by software-based c&c systems. Inject realistic fake attack sequences into BMEWS and whatever the Russian and Chinese systems are called and watch the meat bags do the work. Nuclear power stations. GPS spoofing to collide aircraft and autonomous vehicles over cities. Reality disruption by extremely large scale disinformation, combined with targeted food supply, utility, and comms system outages. Socially-engineer humans into causing death and destruction using bribes/threats or mental destabilisation thru engineered conspiracy theories. Etc.
- RandomLensman 3y ago
  How does a drone exit its storage unit? How does an AI arm a nuclear weapons? How does GPS spoofing crash an aircraft? How does AI do something to a nuclear power plant? Can we not shut down the internet if we needed to combat misinformation? AI would need to overcome a fair amount of systems, both physical as well as software for a lot of these things. It needs a lot of "thinkism" to make that work.
  - andyjohnson0 3y ago
    > How does a drone exit its storage unit? Because the AI fabricated a target package doe it? > How does an AI arm a nuclear weapons? How does GPS spoofing crash an aircraft? How does AI do something to a nuclear power plant? By infiltrating non-internet networks? > Can we not shut down the internet if we needed to combat misinformation? A lot of people seem to enjoy soaking themselves in conspiracy and disinformation. No my guess would be no, we wouldn't ahut it down. > AI would need to overcome a fair amount of systems, both physical as well as software for a lot of these things. It needs a lot of "thinkism" to make that work. Thats a fair criticism. Its unlikely at present, but as I asserted in my original comment, military exploitation of AI seems inevitable (to me) and the forces at work are irresistible.
    - RandomLensman 3y ago
      You wave away of lot of physical action that is needed in a lot of cases - people are not as trusting in software as you think in some areas. It might never happen that AIs can do all this and/or that defense AIs severely limit what offensive AIs can do.

theaussiestew 3y ago

This sounds like the plot of Terminator and the birth of Skynet

stainablesteel 3y ago

its gonna come out of 4chan

underlipton 3y ago

>Military or corporation Why not both? The Horizon franchise's lore has a "fun" example of the military-industrial complex converging on Armageddon.