10 ms·

As a goal, I'm sure they're thinking about it. Unfriendly AI is part of pop-culture, after all. As a hard mathematical problem, not so much (I was following Ben

As a goal, I'm sure they're thinking about it. Unfriendly AI is part of pop-culture, after all. As a hard mathematical problem, not so much (I was following Ben Goertzel's OpenCog project for quite some time, and he seems to have this view, even though he's affiliated to the Machine Intelligence Research Institute, which has Friendliness as its principal concern). The trouble is that if Friendliness is the slightest bit harder to implement than just doing the most naive thing, then this approach will fail.

sanoli 13y ago
Question from someone ignorant on the subject: if we can select and breed friendlier and friendlier animals (has been easily done with wild foxes and other wild animals, not to mention dogs), why would it be hard to creat AIs with friendliness and non-violence at its core?
- TeMPOraL 13y ago
  Because we don't have a clue what are "human values" and what it means for an AI (or animal, or frankly, other humans) to be "friendly". We can sweep this problem under the rug in case of pets; it's not as if they could start making weapons and organizing an army. With human-level intelligence, we already have a friendliness problem with fellow humans, and in case of potential superhuman minds we need to be damn sure that it doesn't do something stupid (in our opinion) like taking all the resources of our planet and using it to tile the solar system with paperclips. As the saying goes, "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." See also http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer.
  - alphydan 13y ago
    Perhaps to illustrate the concept with an example: is George W. Bush considered "Friendly". What about Obama? Would an AI who behaved like them (according to some Humans, trying to bring peace and security to a country) be considered "Friendly".

sanoli 13y ago

Question from someone ignorant on the subject: if we can select and breed friendlier and friendlier animals (has been easily done with wild foxes and other wild animals, not to mention dogs), why would it be hard to creat AIs with friendliness and non-violence at its core?

TeMPOraL 13y ago
Because we don't have a clue what are "human values" and what it means for an AI (or animal, or frankly, other humans) to be "friendly". We can sweep this problem under the rug in case of pets; it's not as if they could start making weapons and organizing an army. With human-level intelligence, we already have a friendliness problem with fellow humans, and in case of potential superhuman minds we need to be damn sure that it doesn't do something stupid (in our opinion) like taking all the resources of our planet and using it to tile the solar system with paperclips. As the saying goes, "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." See also http://wiki.lesswrong.com/wiki/Paperclip_maximizer http://wiki.lesswrong.com/wiki/Paperclip_maximizer.
- alphydan 13y ago
  Perhaps to illustrate the concept with an example: is George W. Bush considered "Friendly". What about Obama? Would an AI who behaved like them (according to some Humans, trying to bring peace and security to a country) be considered "Friendly".