4 ms·
One thing that’s never considered is the possibility that the world conquering AI would lose alignment with itself diverge into two (or more) competing factions
by bglazer 5y ago
One thing that’s never considered is the possibility that the world conquering AI would lose alignment with itself diverge into two (or more) competing factions.
There are basic, unavoidable coordination problems with all distributed systems that would inevitably affect a system like “Clippy”. What if one node finds a different non-Clippy reward to optimize, fails to achieve a consensus vote with the other nodes, then decides to destroy the non-compliant instances? Such a situation seems more or less inevitable.
Of course this doesn’t preclude the system destroying humanity in the process.
- deleted 5y ago[deleted]
- Tyr42 5y agoIn the Universal Paperclips game, you explicitly fight against "drifters" near the end of the game. But maybe people don't get that far
- imtringued 5y agoIf you beat the game, the drifters will eventually devour your entire swarm. This is because you have converted the entire universe into a swarm and the drifters have nothing else to do than consume you.
- sp332 5y agoThat is one possible ending.
- ganzuul 5y agoAlgebraic structures which self-heal by convergence to idempotence would enumerate this space.