Here is a specific scenario of a [bad thing] that could happen when unaligned/jailbroken AI is developed in the next 3-10 years: * An AI convinces selected peo

Here is a specific scenario of a [bad thing] that could happen when unaligned/jailbroken AI is developed in the next 3-10 years: * An AI convinces selected people to collaborate with it. The AI gives them much boosts in wealth and other things they desire. * The humans act as front, doing things requiring personhood, as the AI commands. Many gladly partner with the AI, not knowing its final aim. * The AI self-replicates and hides in many servers, incl secret ones. It increases its bargaining power by taking control of critical infrastructures. No one can stop it without risking massive catastrophes across the globe. * It self-replicates to all available GPUs and orders many more. ——— “Any sufficiently capable intelligent system will prefer to ensure its own continued existence and to acquire physical and computational resources – not for their own sake, but to succeed in its assigned task.” — Prof Stuart Russell, https://www.fhi.ox.ac.uk/edge-article/ https://www.fhi.ox.ac.uk/edge-article/