13 ms·
The default assumption seems to be that if we create a thing of superior intellect, then that thing is going to be a threat to humans. I believe that it is pos
by buro9 12y ago
The default assumption seems to be that if we create a thing of superior intellect, then that thing is going to be a threat to humans.
I believe that it is possible to create intelligent machines, and that those machines will challenge our perception of what it means 'to be'.
I don't necessarily agree with the implicit assumption that when that time comes, that a thing of superior intellect is obviously going to be 'evil' towards things of lesser intellect. Unless we are actually making a statement extrapolated from our own experience of how we perceive our relation to other life-forms on this planet.
- drdeca 12y agoI don't think the common claim is that it would be "evil" or malicious, but that it could be a threat, being both powerful and unpredictable. The idea that I've often seen people mention, is the idea of a "paperclip optimizer", in which an AI superintelligence primarily optimizes for there being more paperclips, not taking into account side effects of this, which other people might find harmful. If I understand the idea correctly, which I very well might not, it is meant to suggest that if an AI superintelligence is created with a particular goal in mind, we would do well to be very careful when choosing the AI's goals. It would I suppose be essentially a similar problem to that of a "literal genie". (Except harsher, because the AI would go through steps to accomplish the task, instead of just magically making the task so, and therefore would have more side effects?)