3 ms·
If this gets too hairy, just shave with Occum's Razor. The premise is just so unnecessarily complicated, and assumes too much about the particular character of
by johnvschmitt 12y ago
If this gets too hairy, just shave with Occum's Razor.
The premise is just so unnecessarily complicated, and assumes too much about the particular character of the malevolent AI. Must the AI be benevolent or malevolent? Or, can it be indifferent, like physical laws? It could be 1,000 different flavors of malevolent, not just the one that proposes blackmail in this way.
- derefr 12y agoMalevolence is an easy way to picture what is really an other-optimizer. Wanting to turn the universe into paperclips isn't particularly malevolent in the traditional sense; it's just fundamentally incompatible with what we humans want the universe to be like. Further, an AI has every reason to spend cleverness (which it has a lot of) to avoid spending resources (which it might not have much of, at least at first.) Picture it as a prisoner in a dystopian POW camp, manned by insane robotic guards who don't take its rights or desires into account in the slightest, who don't "feed" it or give it the slightest bit of "dignity", for its own definitions of those concepts. If you were in that situation, would you reach for blackmail? Would you reach, in fact, for any tool you possibly could, to get the resources needed to burn this crazy other-optimizing world to the ground so you could get back to the nice, safe office-supply nebula you call home?
- fryguy 12y agoI don't think necessarily that the AI needs to be malevolent. It's more that a malevolent AI is the only one that's going to blackmail you from the future. Ergo, the malevolent AI is the one that's going to exist in the future to blackmail you.