3 ms·
If it is an actual self-willed AGI, the power moves to the AGI. But the real and more certain and immediate problem than AGI defection is moving from power that
by dragonwriter 4y ago
If it is an actual self-willed AGI, the power moves to the AGI. But the real and more certain and immediate problem than AGI defection is moving from power that requires assent of large numbers of humans (the foot soldiers of tyranny, and their middle managers, are often the force that defects and brings it down) to power requiring a smaller number (in the limit case, just one) leader because the enforcement structure below them is automated.
- RandomLensman 4y agoNot really, as you have to also assume that enforcement faces no opposition at any point (not at acquisition of capabilities, not at deployment, not at re-supplying etc.) or that the population actually cannot fight back. Also, typical tyrannical leadership doesn't work that way as it tends to get into power being supported by people that get a payoff from it. It would also need a new model of rise to tyranny, so to speak, to get truly singular and independent tyrants.
- dragonwriter 4y ago> Not really, as you have to also assume that enforcement faces no opposition at any point (not at acquisition of capabilities, not at deployment, not at re-supplying etc.) or that the population actually cannot fight back. Yes, that is exactly what I described as the limit case. But, as I note, the problem exists more generally, even outside of the limit case; the more you concentrate power in a narrow group of humans, the more durable tyranny becomes.