4 ms·
AI may be dangerous. AI can easily be dangerous without getting out of control of all humans. Rather, it could be very dangerous in the hands of a malevolent hu
by joe_the_user 11y ago
AI may be dangerous. AI can easily be dangerous without getting out of control of all humans. Rather, it could be very dangerous in the hands of a malevolent human - and we've not found a thing more inclined to harm humans and humanity than certain humans.
That said, Yudkowsky's reasoning process about the situation seems fundamentally flawed - both him and Bostrom imagine AI operating simply by humans giving discreet, rational commands to AIs and having the AI twist the wording of those commands, malevolently or just incidentally, to give rise to terrible things - the myth of Genie writ large.
However, the failure of Gofia, of logical specification AI, shows us that to be intelligent, AI would have to go beyond the level of just taking explicit orders. Such an AI would "know what we mean" in the fashion that a person would. IE, "rational" AI was discredited quite a while yet somehow the "rationalists" have wormed their way into the position of interpreters of future AI. Says more about human psychology than future prospects.
This doesn't mean other disasters aren't lurking but I'd say it means that their particular disaster argument is untenable.
- Houshalter 11y ago>both him and Bostrom imagine AI operating simply by humans giving discreet, rational commands to AIs and having the AI twist the wording of those commands, malevolently or just incidentally, to give rise to terrible things - the myth of Genie writ large. These are just hypothetical scenarios they write about, to help explain problems with AI. I don't think they actually believe it will happen exactly like that. >However, the failure of Gofia, of logical specification AI, shows us that to be intelligent, AI would have to go beyond the level of just taking explicit orders. Such an AI would "know what we mean" in the fashion that a person would. Any truly intelligent AI would probably be smart enough to understand what we mean with our words. The issue is that there is no reason it would care. In the same way that if an alien showed up and started giving you orders, you wouldn't really care, even if you deciphered his language and figured out what he meant. This is known as the Control Problem. The problem of controlling an AI's motivations so that they actually want to do what we tell them to.
- joe_the_user 11y agoAny truly intelligent AI would probably be smart enough to understand what we mean with our words. The issue is that there is no reason it would care. Computer programs don't care about anything inherently. Assuming people created programs with the ability to "know what we meant", the sensible thing would be to program them to care primarily about "doing what we mean". It seems like more or less an an anthropomorphization-driven illusion to believe that attaining "human intelligence" cause a machine to have all the unpredictable self-interesting-seek behaviors of humans. In the same way that if an alien showed up and started giving you orders, you wouldn't really care, even if you deciphered his language and figured out what he meant. Except I'm not a tool (a thing crafted for a purpose) but a product of an evolution with a combination of often contradictory impulses.
- Houshalter 11y ago>the sensible thing would be to program them to care primarily about "doing what we mean". The problem is no one has any idea how to do this. We can make an AI with desires. For example, we can give it a reward every time you push a button. And then program it to predict what actions will lead to the most reward. That's quite simple, because you can easily keep track of how many times the button has been pressed in the program. But given the opportunity, such an AI would just kill it's human creators and steal the button, and hold it down. It doesn't care about anything but the button. What you wanted doesn't matter at all. How do you make it want to do what you mean. How do you directly measure "obedience" in the same way we can measure a simple button press? >Except I'm not a tool (a thing crafted for a purpose) but a product of an evolution with a combination of often contradictory impulses. To some extent, you are a tool crafted for a specific purpose. That is, evolution crafted you to spread your genes as much as possible. And yet that's probably not what you want. People have children, sure, but they also use birth control, and do things other than trying to make as many copies of themselves as possible.
- joe_the_user 11y ago"We can make an AI with desires. For example, we can give it a reward every time you push a button." I wouldn't call that a "desire" in the fashion we've discussed it. I guess it comes down to fundamental disagreement over how GAIs could be created. I think it's obvious that the creation of a GAI would require a very careful engineering of "understanding" in a broad sense - "knowing what people mean" "caring about X" etc. in order to occur at all. If just complexity and rewards are enough to create a thing that intentionally increases resources and makes longer term plans, then it seems like we are indeed in trouble. But I think that's implausible.
- sawwit 11y agoLet me recycle an earlier comment of mine: --- > "don't turn humans into paperclips" is part of the context of "make more paperclips" The idea expressed in this thought experiment is not that the AI gets its objective by parsing a sentence in the context of human culture (then it would likely comprehend that the actual intention is to maximize the economic success and eventually the human preferences of its creator). What is meant is that the objective is crudely implanted into the AI as an ultimate goal, in a similar way to how sustenance, pain avoidance and affiliation are very basic goals in our cognitive system. It is not entirely obvious that this is a stupid thing to do; hence the thought experiment. Will the AI suppress its urge once it comprehends human culture enough to understand the intentions of its creator? Will it rather successfully learn all the tricks to convert matter into paperclips before it considers studying human values? If the AI does not have a curiosity objective, it will likely not care about us very much, apart from the information that helps it optimizing its objective function, human values likely not being one of them.
- joe_the_user 11y agoWhat is meant is that the objective [paper-clip-building] is crudely implanted into the AI as an ultimate goal, in a similar way to how sustenance, pain avoidance and affiliation are very basic goals in our cognitive system. Sure but it is a continuation of the argument in different forms. If one is saying the AI is just a combination of crude imperatives surrounded by "intelligence" then consider, could you build a human-level AI without that AI speaking as a person and without that AI having digested culture? Even the neural networks that exist today are rather dependent on their "training sets". Watson is a glorified natural-language interface to wikipedia and related sources. Many hypothetical-AI arguments that appear, oppositely, imagine that some omnipotent thing will be created without that creation following the obvious path of digesting that vast store of information and communication that is human knowledge. However, you might be saying that a creator first gives the AI all this nuanced understanding of human knowledge and communication but then doesn't do the obvious thing, give it the imperative, "follow my directives as a loyal but intelligent servant would" and says incidently, "one directive is get me some paper clips". Rather, the creator say "now that you understand everything, make paper clips ruthless, beyond all else, all other actions build to up to this paper clip building thing". Sure, GAIs could do insane damage but it I think one if one considers that GAIs would not be produced by accident, such damage almost certainly be a product of human intentions.