2 ms·
> it is kind of arrogant to think you can design or think about superintelligences without reference to the best existing intelligent systems we have. I don't
by gwern 13y ago
> it is kind of arrogant to think you can design or think about superintelligences without reference to the best existing intelligent systems we have.
I don't think it's any more arrogant than, say, Djikstra pointing out that submarines move in a completely different manner than a human swimmer. How arrogant were the Wright brothers in looking at birds and deciding to try to achieve some of the same goals by a completely different mechanism?
If computers thought the same way humans did, if they came with builtin moral sentiments, the product of a very unusual evolutionary history and social structure, if they had access to no novel capabilities, then humans and forms of cooperation developed over the last few eyeblinks (centuries) might be relevant. But then, no one would care about the issue in the first place...