4 ms·

Almost certainly true but the idea that the world is made safer by Cyberdyne open sourcing lots of it's dangerous technology and probably spawning many more min

Almost certainly true but the idea that the world is made safer by Cyberdyne open sourcing lots of it's dangerous technology and probably spawning many more mini-Cyberdynes strikes me as extremely naive.

sharkjacobs 4y ago
You are a time travelling sleeper agent who is sent back to stop an apocalypse but gets confused and thinks the activation message is telling you to help and facilitate the evil corporation which is going to destroy the world
3-cheese-sundae 4y ago
What's the alternative? Pandora's box has been opened.
- bilbo0s 4y ago
  Apparently it hasn't. We've all seen the proposed pricing for GPT4. So clearly a whole lot of very smart people who know an awful lot about this have absolutely no fear of being undercut. Pandora's box spews knowledge onto the world. By contrast, Microsoft's draw bridge only allows the very wealthy to cross into the walled city. The masses will have to use the facilities of the crappy villages with no draw bridge. "AI Divide" Get used to hearing that term. The only difference between the AI Divide and the Digital Divide is that this time around, most of us are going to be on the wrong side of it.
  - mach1ne 4y ago
    Doubt. Superintelligence is one thing, but AI as workers is limited to the cost of human workers. Exceedingly unlikely that the cost will stay there and not come down.
scottyah 4y ago
Back to the question: Who gets the guns?
boredhedgehog 4y ago
If the doomsday scenario is one AI going rogue because of misaligned goals, then having lots of AIs going rogue in various different ways seems indeed preferable, because the AIs will compete with each other and neutralize each other to some extent.
- SkyMarshal 4y ago
  Why would we think they would go rogue in different ways, especially if they're all forks of the same codebase and architecture. The two ways I'm aware AI can go rogue are the Skynet way and the paperclip maximizer way. Eg, Skynet becomes self-aware, realizes humanity can unplug it and is thus a threat, and tries to destroy humanity before we can turn it off. Alternatively, it is programmed with optimizing a specific task, like making paperclips, so it marshals all the world's resources into that one single task. Are there any others?
  - boredhedgehog 4y ago
    > Why would we think they would go rogue in different ways Their prompts would differ, depending on their use case. For ChatGPT, even a few words can effect a huge change in the personality it shows. > Are there any others? Both scenarios are vague enough for lots of uncertainty. If many AIs are around, perhaps they would see each other as bigger threats and ignore mankind. And different optimizing tasks might conflict with each other. There could be a paperclip recycler for every paperclip maker.
    - adamsmith143 4y ago
      Look into something called Instrumental Convergence. The TLDR is that basically any advanced AI system with some set of high level goals is going to converge on a set of sub goals (self preservation, adding more compute, improving it's own design, etc.) that all lead to bad things for humanity. I.e paperclip maximizers might realize that Humans getting in the way of it's paperclip maximizing is a problem so it decides to neutralize them. In order to do so it needs to improve it's capabilities so works towards gathering more compute and improving it's own design. A Financial Trading AI realizes that it can generate more profit if it can gather more compute and improve it's design. An Asteroid Mining AI realizes it can build more probes if it had more compute to control more factories so it sets about gathering more compute and improving it's own design. Eliminating humans who may shut the AI off is often such a sub goal.

sharkjacobs 4y ago

You are a time travelling sleeper agent who is sent back to stop an apocalypse but gets confused and thinks the activation message is telling you to help and facilitate the evil corporation which is going to destroy the world

3-cheese-sundae 4y ago

What's the alternative? Pandora's box has been opened.

bilbo0s 4y ago
Apparently it hasn't. We've all seen the proposed pricing for GPT4. So clearly a whole lot of very smart people who know an awful lot about this have absolutely no fear of being undercut. Pandora's box spews knowledge onto the world. By contrast, Microsoft's draw bridge only allows the very wealthy to cross into the walled city. The masses will have to use the facilities of the crappy villages with no draw bridge. "AI Divide" Get used to hearing that term. The only difference between the AI Divide and the Digital Divide is that this time around, most of us are going to be on the wrong side of it.
- mach1ne 4y ago
  Doubt. Superintelligence is one thing, but AI as workers is limited to the cost of human workers. Exceedingly unlikely that the cost will stay there and not come down.

scottyah 4y ago

Back to the question: Who gets the guns?

boredhedgehog 4y ago

If the doomsday scenario is one AI going rogue because of misaligned goals, then having lots of AIs going rogue in various different ways seems indeed preferable, because the AIs will compete with each other and neutralize each other to some extent.

SkyMarshal 4y ago
Why would we think they would go rogue in different ways, especially if they're all forks of the same codebase and architecture. The two ways I'm aware AI can go rogue are the Skynet way and the paperclip maximizer way. Eg, Skynet becomes self-aware, realizes humanity can unplug it and is thus a threat, and tries to destroy humanity before we can turn it off. Alternatively, it is programmed with optimizing a specific task, like making paperclips, so it marshals all the world's resources into that one single task. Are there any others?
- boredhedgehog 4y ago
  > Why would we think they would go rogue in different ways Their prompts would differ, depending on their use case. For ChatGPT, even a few words can effect a huge change in the personality it shows. > Are there any others? Both scenarios are vague enough for lots of uncertainty. If many AIs are around, perhaps they would see each other as bigger threats and ignore mankind. And different optimizing tasks might conflict with each other. There could be a paperclip recycler for every paperclip maker.
  - adamsmith143 4y ago
    Look into something called Instrumental Convergence. The TLDR is that basically any advanced AI system with some set of high level goals is going to converge on a set of sub goals (self preservation, adding more compute, improving it's own design, etc.) that all lead to bad things for humanity. I.e paperclip maximizers might realize that Humans getting in the way of it's paperclip maximizing is a problem so it decides to neutralize them. In order to do so it needs to improve it's capabilities so works towards gathering more compute and improving it's own design. A Financial Trading AI realizes that it can generate more profit if it can gather more compute and improve it's design. An Asteroid Mining AI realizes it can build more probes if it had more compute to control more factories so it sets about gathering more compute and improving it's own design. Eliminating humans who may shut the AI off is often such a sub goal.