What if "runaway" isn't possible? It might be possible for an AI to be roughly as "intelligent" (depending on how one measures this) as the smartest humans, bu

What if "runaway" isn't possible? It might be possible for an AI to be roughly as "intelligent" (depending on how one measures this) as the smartest humans, but that intelligence is the result of many millions or even billions of years of accumulated evolutionary learning. It might be -- for fundamental information and machine learning theory reasons -- fundamentally harder to go where there are no roads. Start looking into combinatorics and the problems of searching large spaces. The observation that genius is often tied to madness may be indirect circumstantial evidence for this. When we try to push the boundaries of human intellect, we seen to run rapidly into weird problems.