3 ms·
I don't believe in a runaway AI scenario, where something arises that has godlike abilities and takes over the planet. This overweights the returns to intellige
by neilk 4y ago
I don't believe in a runaway AI scenario, where something arises that has godlike abilities and takes over the planet. This overweights the returns to intelligence, the likelihood that it will be given a free hand, and underweights the cost of creating true superintelligence - likely exponential.
The salient characteristic of AI is not that it is superintelligent, but that it is perfectly obedient.
The rulers of earth will be the same people as we have always had, but now they will have an army of automated mooks to enforce their will. These automated servants will be able to make intelligent judgments, but will have no ambitions to seize the throne. And it's okay if the mooks frequently make mistakes. Elites value absolute loyalty, much more than ability. Until now, it has not been possible to obtain perfect loyalty from any being with independent judgment. Elites would be willing to pay huge fortunes for such servants.
This is why there won't be a "runaway" scenario. Elites have never ceded full authority to their most intelligent servants. They will not want a computer discovering that the optimal allocation of resources would be UBI, and then implementing it. Elites will ask for the greatest possible allocation of resources to themselves, and a means of maintaining that inequality.
AIs will be the middle managers, the enforcers, the killer drones, and the security guards.
To the extent that our existence is necessary at all, we will have to negotiate with the AIs to be allowed to live out our lives.
But humanity might be forced out. It's happened before. Consider the Irish potato famine. Despite the name, what actually happened was that an entire population was driven off the productive lands by foreign owners armed with guns. The Irish were only relying on the potato because it was the cheapest way to survive when you barely had any land left. When a blight struck, they all died or emigrated. Maybe we'll all die or emigrate to places that the elites/AIs don't want. But it's possible even that won't happen because there won't be any frontiers left that just need human bodies to exploit, as was the case in the Americas.
- telchior 4y agoI'd summarize that as: civilization has always featured the few powerful having control over the many, but those few still had to satisfy at least some people. AI that can closely mimic humans and does not have a concept of self-benefit could vastly increase the power that individuals are able to wield, even beyond the kings and dictators of the past. Contrary to your assertion that it isn't a "runaway" scenario, it very well could be one; it's just not the AI that's leading the charge. Fairly similar idea to Vernor Vinge's concept of how the rise of a powerful surveillance state could be a civilization-ending event, by conferring unprecedented power into the hands of a few.