Sure, we could wait until such a thing is invented before we start worrying about it. But by then it's far, far too late. It's not the same as worrying about a

Sure, we could wait until such a thing is invented before we start worrying about it. But by then it's far, far too late. It's not the same as worrying about a giant space goat appearing and sneezing us all to death. A lot of very smart, very well-funded people are actively trying to make better, more general AI, capable of learning. Evidence of progress in that effort are all around us. You seem remarkably confident that they'll all run into an as-yet-invisible brick wall before reaching the goal of superintelligence. Superintelligence doesn't have to be malicious to be worrying; concepts like "malice" are very unlikely to be applicable to it at all. The worry is that as things stand we have no frickin' idea what it'll do; the first challenge for policy is to come up with a robust, practical consensus on what we'd want it to do.