Assuming the AI is built to be rational, the AI will act according to its goal system. So it's not going to do that unless you make some mistake or omission in

Assuming the AI is built to be rational, the AI will act according to its goal system. So it's not going to do that unless you make some mistake or omission in its goal system, since it would have to find the motivation to "impersonate shell companies, human beings, etc" somewhere in its goals. This hints at two major problems in building an AI: how do you make it rational (That's the "intelligence" part) and how do you come up with a goal system that is safe and beneficial from our (the human race's) perspective? Eliezer has written a paper addressing these questions: http://singinst.org/AIRisk.pdf http://singinst.org/AIRisk.pdf. He recommends you read this paper on human cognitive biases before reading the AIRisk paper: http://singinst.org/Biases.pdf http://singinst.org/Biases.pdf