3 ms·
It seems pretty tractable to not build such a thing as a hostile AI.
by emodendroket 8y ago
It seems pretty tractable to not build such a thing as a hostile AI.
- rhaps0dy 8y agoI'm not so sure. Even prosaic, current AI, can be very unpredictable. See this blog post http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us/amp http://aiweirdness.com/post/172894792687/when-algorithms-sur... , which is a somewhat more causal summary of this paper "The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities" https://arxiv.org/abs/1803.03453v2 https://arxiv.org/abs/1803.03453v2 The authors of the paper explain a bunch of anecdotes when they wanted the AI to do one thing, but the measure they told it to optimise for wasn't actually what they intended, and so unexpected things happened.
- emodendroket 8y agoThat's an amusing read, and sure, AI can do things we don't expect. I think we're more on the order of a bridge failure and less on the order of Skynet here, though. To end up in that kind of nightmare scenario we have to entrust some AI with capabilities we could just elect not to.
- csense 8y agoYou have to find every group of humans trying to build an AI, and either convince or force them not to do so. This seems hard, given: - (a) The cost of starting an AI venture is minimal (perfectly within reach for a small group or a smart individual with $10 million or less for salaries plus cloud computing expenses). It's a lot harder to keep tabs on small and non-obvious activity like this than, say, the enormous centrifuge facilities needed to refine uranium for nuclear weapons. - (b) The personal and societal economic, and other, benefits of advancing the state of the art in AI are potentially enormous. People will be motivated to try, whether their goal is to line their pockets, to better understand what intelligence is by advancing the state of the art of trying to put it into machines, or to make the world a better place. - (c) The problem domain is not well understood, so it's possible to stumble into a dangerous design by accident. It seems like only some extreme dystopian scenario could halt the progress of technology to the point where an AI-related disaster becomes impossible. For example, a global war or disaster that destroys civilization's logistical and technological bases, kills much of the world population and forces the survivors to focus mainly on survival. Or an ideological revolution that sees all the nerds of the world lined up against a wall and shot, and those who remain alive become culturally permanently uninterested in advancing technology to avoid the same fate. Or a world-dominating tyrannical government that keeps a close watch on every expert programmer and computer system in the world, monitoring to make sure unauthorized AI experiments don't take place. Even that might not be enough, because if any of the human population survives, even the biggest global catastrophe, cultural revolution or tyrannical world empire's effects probably won't last more than a few thousand years.