Self consciousness is almost certainly required for intelligence. So much so that one way to categorize animal intelligence is whether or not they pass a "dot

Self consciousness is almost certainly required for intelligence. So much so that one way to categorize animal intelligence is whether or not they pass a "dot test". When seeing a dot on themselves in a mirror they recognize that the dot is, in fact, on themselves and try to see / clean it on themselves. Until there is a decent demonstration of generalization to the point of self-awareness runaway AI will be an "overpopulation on Mars" problem. Even when awareness is demonstrated in an algorithm the algorithm will not be able to solve NP problems in P time. EDIT: ok, it is extremely unlikely they will be able to solve NP problems in P time. If they privately demonstrate P=NP then we are screwed. However, we are going to have all manner of dense, but self-aware programs before we get a CS super genius. We will make the first C3PO level intelligence well before we have super-intelligence, and we haven't even made anything close to that yet.