6 ms·
It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI
by bwood 11y ago
It might be a little far-fetched, but there's a thought experiment called the "paperclip maximizer" that illustrates how even a well-designed, non-malicious AI could be an existential threat. See here: http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer
- return0 11y agoi dont understand how that thing will develop intention. We perceive ourselves having it, but its really an accident that we do; if we didnt we wouldnt be here to observe it. Why would an artificial machine, no matter how smart , want to do anything?
- DanAndersen 11y agoDon't think of "wanting" as a soul-like action of a little person in the machine. "Wanting" is a shorthand for anything that is the result of a goal system and autonomy. In the simplest sense, giving a drone a map of its surroundings and giving it a location as the goal for its path-finding optimization algorithms, the drone now "wants" to go to that location.
- deleted 11y ago[deleted]
- bwood 11y agoIf the AI doesn't have its own intention (perhaps having intention is a prerequisite for being truly intelligent), it's conceivable that we would program it with an intention or goal, such "find the most efficient way to produce as many paperclips as possible". If you assigned such a task to a human, it's fairly obvious that the task is subject to other things which are important to humans (for example, don't enslave or kill people to build paperclips, don't replace strip-mine all our farmland to find more paperclip material, etc.). But since an AI doesn't necessarily share, understand, or agree with all the implicit values we have, it's myopic view of the world could cause it to destroy us as a mere side-effect of trying to achieve it's singular goal.
- return0 11y agoyeah but that's bad design or terrorism, there is always another person to blame here. I think we are talking here about machines that take the blame themselves.
- DanAndersen 11y agoI'm not sure what the purpose is of focusing on blame. You don't need to assign moral culpability to a machine for its catastrophically poor design to be something to worry about.
- SCHiM 11y agoIt 'wants' what we programmed it to want. It's not that the machine decided that it wanted to build paperclips. It's what we've told it to do. The _means_ to build paperclips however is where the 'intelligence' comes into play. We've programmed the drive to build paperclicps, but we've left it to the AI to figure out how to do this. The article that you've read assumes that planning and subgoals are necessary for the AI to achieve its goal. Eg. That it needs to understand that in order to make paperclips it needs materials, and the further insight that to acquire more materials it needs to remove the other consumers of materials (us).
- return0 11y agoif (nearby() == human) {dontTouch(); abort() ; } There, i solved that. The question is under what conditions would a machine remove that instruction and how could those conditions be reached?
- TeMPOraL 11y agoDefine nearby(). Because if the AI starts eating chunks of our planet we will get hurt as well. INB4 Asimov's Three Laws of Robotics don't work. Actually, a lot of Asimov's writing is concerned with exploring their corner cases.
- digi_owl 11y agoI have come to believe that corner cases quickly turn fractal. Once you think you have solved it, changing the "zoom" will demonstrate that your solution has created X more cases. It may be a side effect of the halting problem.
- SCHiM 11y agoWhat's a human then? Is a mentally retarded person a human? An embryo? A person who's lost his/her legs? A person who has education? auto HumanObject = nearby(); if(Humanobject.IsPregnantWoman()) { PoisionEmbryoButNotHuman(); } else { abort(); }
- DanAndersen 11y ago