> What is superhuman machine intelligence (SMI) ? SMI is advanced decision making software that literally eats the world. Think of HAL killing its crew so they

> What is superhuman machine intelligence (SMI) ? SMI is advanced decision making software that literally eats the world. Think of HAL killing its crew so they don't jeopardize the mission, but on a global scale. Like a stock trading algorithm that determines the best way to maximize profits is to wipe out all human life on the planet or something. --EDIT-- See http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer > What anecdotes (or evidence) is leading to this concern? Books, movies, pop culture... plus when all you have are first-world problems, you gotta find something to worry about.