> If we're talking about bad AI, shouldn't we ask why it would want to kill us? The AI doesn't want to kill you. It is indifferent to you. The problem is that

> If we're talking about bad AI, shouldn't we ask why it would want to kill us? The AI doesn't want to kill you. It is indifferent to you. The problem is that you're made of resources it can use... [1] > it's hard to imagine something qualifying as AI without it understanding things like ethics, morals, humility, aesthetics etc Suppose your mother wants you to get married. You understand her desires, but you don't share them. Understanding morals and desires is distinct from being motivated by them. [2] 1: http://wiki.lesswrong.com/wiki/Basic_AI_drives http://wiki.lesswrong.com/wiki/Basic_AI_drives 2: http://lesswrong.com/lw/sy/sorting_pebbles_into_correct_heaps/ http://lesswrong.com/lw/sy/sorting_pebbles_into_correct_heap...