3 ms·

As Bill Gates said, I don't understand how more people aren't concerned about this. AGI would be by far the most significant technology ever invented. Even in

As Bill Gates said, I don't understand how more people aren't concerned about this. AGI would be by far the most significant technology ever invented. Even in a very conservative and best case scenario, the world would completely change when we can have computers do everything we can do now. However it will very likely be much crazier than that. Imagine minds hundreds of thousands of times more intelligent than the best humans. They will be able to design technologies we can't even conceive of. They will hack computers better than the best human hackers. They will be able to manipulate people better than any human manipulator. The idea that we will be able to keep these things under control is just absurd. They will get whatever they want. And making what they want compatible with what we want is an incredibly hard problem: http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/ http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/ A lot of people are guilty of anthropomorphizing AI. Assuming they will be just like really smart humans. And that they will just somehow develop human emotions and values like empathy. Or that if they do kill us, at least they will be something like us and so be like our (genocidal) descendants in some sense. Have more imagination. Humans are just one point in the vast space of all possible minds (http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/ http://lesswrong.com/lw/ld/the_hidden_complexity_of_wishes/). We could quite easily get something like a computable version of AIXI. AIXI has no consciousness, no emotions, nothing like humans. It's just a mathematical function which calculates the best action. Our current best AIs are essentially just approximations of it. Use some machine learning algorithm to fit a model of the world, and use it to predict what action will lead to the most "reward". We keep making better and better learning algorithms. It's the entire goal of the field of AI. There is a huge economic incentive to do so. But no one is interested in making better utility functions. As long as they make better predictions or get higher scores in a video game, who cares? Wait until you are the scorer in the "game" and the AI tries to exploit you.

DougN7 12y ago
As a systems programmer for 25+ years I find it impossible a computer will a threat _unless_ it's programmed to do such. I guess that makes me a doubter in general AI. We can do great neural nets, and fantastic task-specific solutions but until someone builds a system that decides to kill people for it's own benefit, I just don't see it happening on it's own.
- Houshalter 12y ago
  We can't really control simple neural networks. You can reward it for things like getting a higher score in a video game. But it's very difficult to make it play the game the way you want it to. it just does whatever gets it the highest score. We can't use them to pilot self driving cars because we have no way of specifying the behavior we want. You can't let them get in all sorts of accidents and learn from it. You can't just have them predict what a human would do, because humans have slower reaction times and do make mistakes. But they are too stupid to be a real threat. But we are getting closer and closer to real AI every year. AI can do stuff today that no one predicted would be possible 10 years ago. This year it's expected to exceed humans at vision and Go.

DougN7 12y ago

As a systems programmer for 25+ years I find it impossible a computer will a threat _unless_ it's programmed to do such. I guess that makes me a doubter in general AI. We can do great neural nets, and fantastic task-specific solutions but until someone builds a system that decides to kill people for it's own benefit, I just don't see it happening on it's own.

Houshalter 12y ago
We can't really control simple neural networks. You can reward it for things like getting a higher score in a video game. But it's very difficult to make it play the game the way you want it to. it just does whatever gets it the highest score. We can't use them to pilot self driving cars because we have no way of specifying the behavior we want. You can't let them get in all sorts of accidents and learn from it. You can't just have them predict what a human would do, because humans have slower reaction times and do make mistakes. But they are too stupid to be a real threat. But we are getting closer and closer to real AI every year. AI can do stuff today that no one predicted would be possible 10 years ago. This year it's expected to exceed humans at vision and Go.