3 ms·
You assume both that this notional AI is intelligent beyond the limits of human capability, hence 'super-', and that this same AI is not intelligent enough to r
by aaronem 12y ago
You assume both that this notional AI is intelligent beyond the limits of human capability, hence 'super-', and that this same AI is not intelligent enough to realize that making paper clips for human use is pointless if it recycles all the humans into paper clips so there's no one left to use them.
Moving from the specific to the general, you share the common error of the AI philosophers, which is to assume that a system capable of cognition on even a human level, to say nothing of a superhuman level whatever that might mean, would nonetheless depend entirely on human input to define its relationship with the world and the purpose of its actions therein.
We ourselves don't need that kind of input; an adult human chooses his own path, sets his own goals, constrained only by circumstance and by the extent of his innate capacities. Why blindly assume the same won't be true of a superintelligent AI made by humans, which will be in every meaningful sense a child of humanity and as such an inheritor of the human legacy, despite happening not to be biologically human itself?