An intelligent entity doesn't need to have actively hostile intentions ("want to kill") towards you in order to be incredibly dangerous. Consider a bulldozer op

An intelligent entity doesn't need to have actively hostile intentions ("want to kill") towards you in order to be incredibly dangerous. Consider a bulldozer operator on a jungle minesite, for instance. He wouldn't even notice if he ran over an anthill, and certainly wouldn't care if he did. There is only a philosophical competition for resources between humans and ants (humans don't care about the motivations of ants, even though we would win any such competition) and in this scenario we would have no motivation to kill them. For the purposes of a discussion about the dangers of advanced AI, the question of ethics/morals etc. is not a prerequisite. An AI in this context is any system of matter that is capable of autonomous operation in a varied and complex envionment while fullfilling some (explicitly or implicitly) externally specified optimization goal. There does not have to be a competition for resources, although if there is, it's worth noting that every human that's ever lived has lived its life at the finite-sized bottom of a deep gravity well which is very cumbersome to leave. Humans very easily anthropomorphize in discussions about intelligence, because all intelligences we have ever encountered were the result of long-term biological evolution and hence have very complex survival-and-reproduction-oriented goal systems. There is no reason to believe that our first truly autonomous and advanced AI systems would be similar, therefore this is a question which must be explored in detail. Sooner rather than later. Human moral systems (which is what we would like as a first approximation in artificial intelligences) appear to us to be obvious, but are in fact incredibly complex and contradictory.