3 ms·
I think the list is misleading and is not necessary for an AI to be dangerous. >It has an “I,” a sense of self distinct from others. I deny this is required f
by ikeboy 12y ago
I think the list is misleading and is not necessary for an AI to be dangerous.
>It has an “I,” a sense of self distinct from others.
I deny this is required for a program to be dangerous.
>It has the intellectual capacity to step outside of the boundaries of its intended purpose and programming to form radically new goals for itself (the “I”).
Deny this as well; even carrying out programmed goals can be harmful. In the words of Superintelligence (copied from https://intelligence.org/2015/01/08/brooks-searle-agi-volition-timelines/ https://intelligence.org/2015/01/08/brooks-searle-agi-voliti...):
>[W]e cannot blithely assume that a superintelligence with the final goal of calculating the decimals of pi (or making paperclips, or counting grains of sand) would limit its activities in such a way as not to infringe on human interests. An agent with such a final goal would have a convergent instrumental reason, in many situations, to acquire an unlimited amount of physical resources and, if possible, to eliminate potential threats to itself and its goal system.
>It has access to resources on a global scale to carry out the plan.
This sounds reasonable. As http://slatestarcodex.com/2015/04/07/no-physical-substrate-no-problem/ http://slatestarcodex.com/2015/04/07/no-physical-substrate-n... points out, humans have made billions anonymously, just by being connected to the internet. http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover_mechanisms/ http://lesswrong.com/lw/k4h/request_for_concrete_ai_takeover... gives some ways an AI could takeover.