6 ms·

The only thing worse than the "evil" genie giving us exactly what we ask for is the genie that decides its inferences about our actual intents/wants/needs/motiv

The only thing worse than the "evil" genie giving us exactly what we ask for is the genie that decides its inferences about our actual intents/wants/needs/motivations based purely on our behaviors matter more than what we intentionally express as our intents/wants/needs/motivations. The most obvious analogy, and I acknowledge the potentially controversial nature of it, is that of a date rapist attempting to defend their actions by pointing out the victim was dressed provocatively, gave all the "signals" that they wanted sex, and when they expressly said "no" the rapist knew from their prior behavior that they actually did want it (or had some inner heretofore unexpressed need for it). The true underlying motivation for such a rewrite of Asimov's laws lies in the paragraph that follows Russell's new list. "...developing innovative ways to clue AI systems in to our preferences, without ever having to specify those preferences." Perhaps he can start by ceasing to write and speak and instead clue in his audience to his preferences via behavior, since specifying them is somehow undesired. Asimov's Three Laws exist to place boundaries around the genie's means of achieving our wishes. Russell's laws remove not only the genie's boundaries to means but also lets the genie make the wish as well. After all, it knows better than you what you want.

wmeredith 7y ago
Perhaps a less controversial example would be the interpretation of the three laws by the antagonist in the film I, Robot from 2004. From wikipedia[0], "in [the antagonist robot's] understanding of the Three Laws, she has determined human activity will eventually cause humanity's extinction, and as the Three Laws prohibit her from letting that sort of thing happen, she rationalizes that restraining individual human behavior and sacrificing some humans will ensure humanity's survival" 0) https://en.wikipedia.org/wiki/I,_Robot_(film)#Plot https://en.wikipedia.org/wiki/I,_Robot_(film)#Plot
- jcranmer 7y ago
  Asimov explicitly added a Zeroth Law of Robotics when he bridged the Robot and Foundation series, although it's still commented that a robot actually killing someone is too much of a cognitive dissonance to bear. It's worth noting that most of Asimov's robot stories are him basically exploring the problems with the Three Laws; The Naked Sun is basically an entire novel showing how Three Laws robots can be capable of murder.
ajuc 7y ago
> The only thing worse than the "evil" genie giving us exactly what we ask for is the genie that decides its inferences about our actual intents/wants/needs/motivations based purely on our behaviors matter more than what we intentionally express as our intents/wants/needs/motivations. It's not worse. > a date rapist A rapist shares 99+% of definitions and values with you as a fellow human being. AI won't (unless you somehow program them in). Rapist has his evil motivations, but won't suddenly invent a virus that kills the whole human species because you asked him to stop neighbor kids trampling your garden. AI might. If you tell it not to kill anybody it might destroy our civilization to prevent us from killing ourselves with global warming. Why not - it only makes sense. Simplest way to ensure safety for the maximum number of human beings is to anesthetize everybody and put them on life support till their natural death. Perfect record is possible - you might cure addicts and prevent crimes and wars. If you specify people have to be awake as often as they usually are - it can keep them awake but restrained. You want people to have freedom of movement? Ok - you just released the whole prison population :) Keeping criminals in prison is OK? Then it may make everybody a criminal for a quick fix. Or just drug everybody to WANT to be restrained. And so on, and so on. There's infinite number of possible courses of action that we discard without consciously thinking about them because of our assumptions. You have to put these assumptions in the AI, each and every one of them, and they are very subtle and invisible for us most of the time. And they often border on philosophy and morality, and defining them is political by definition. It's probably impossible to code all our values and assumptions in by hand. That's a much bigger problem for safe general AI than a post-factum explanations that rub your morality the wrong way. > Asimov's Three Laws Are self-contradictory and useless for anything except literature.
- fallous 7y ago
  And a drug addict, by analysis of behavior, is committing suicide so the AI should help them achieve their inferred goal based on that behavior. Nevermind asking them if that is their intent, it knows better because it sees the behavior. Russell's laws encode NO limits and doesn't even demand the AI check its judgment with the so-called beneficiaries of its decisions. That is most assuredly worse.
  - 7y ago
edmundsauto 7y ago
Interestingly, I believe that what we perceive as our intents/wants/needs/motivations is just an internal genie. Put another way, we have no idea what we really want, consciously. Look at the Schindler's List Netflix example. Also, "A person can do what they want, but not want what they want."

wmeredith 7y ago

Perhaps a less controversial example would be the interpretation of the three laws by the antagonist in the film I, Robot from 2004. From wikipedia[0], "in [the antagonist robot's] understanding of the Three Laws, she has determined human activity will eventually cause humanity's extinction, and as the Three Laws prohibit her from letting that sort of thing happen, she rationalizes that restraining individual human behavior and sacrificing some humans will ensure humanity's survival" 0) https://en.wikipedia.org/wiki/I,_Robot_(film)#Plot https://en.wikipedia.org/wiki/I,_Robot_(film)#Plot

jcranmer 7y ago
Asimov explicitly added a Zeroth Law of Robotics when he bridged the Robot and Foundation series, although it's still commented that a robot actually killing someone is too much of a cognitive dissonance to bear. It's worth noting that most of Asimov's robot stories are him basically exploring the problems with the Three Laws; The Naked Sun is basically an entire novel showing how Three Laws robots can be capable of murder.

ajuc 7y ago

> The only thing worse than the "evil" genie giving us exactly what we ask for is the genie that decides its inferences about our actual intents/wants/needs/motivations based purely on our behaviors matter more than what we intentionally express as our intents/wants/needs/motivations. It's not worse. > a date rapist A rapist shares 99+% of definitions and values with you as a fellow human being. AI won't (unless you somehow program them in). Rapist has his evil motivations, but won't suddenly invent a virus that kills the whole human species because you asked him to stop neighbor kids trampling your garden. AI might. If you tell it not to kill anybody it might destroy our civilization to prevent us from killing ourselves with global warming. Why not - it only makes sense. Simplest way to ensure safety for the maximum number of human beings is to anesthetize everybody and put them on life support till their natural death. Perfect record is possible - you might cure addicts and prevent crimes and wars. If you specify people have to be awake as often as they usually are - it can keep them awake but restrained. You want people to have freedom of movement? Ok - you just released the whole prison population :) Keeping criminals in prison is OK? Then it may make everybody a criminal for a quick fix. Or just drug everybody to WANT to be restrained. And so on, and so on. There's infinite number of possible courses of action that we discard without consciously thinking about them because of our assumptions. You have to put these assumptions in the AI, each and every one of them, and they are very subtle and invisible for us most of the time. And they often border on philosophy and morality, and defining them is political by definition. It's probably impossible to code all our values and assumptions in by hand. That's a much bigger problem for safe general AI than a post-factum explanations that rub your morality the wrong way. > Asimov's Three Laws Are self-contradictory and useless for anything except literature.

fallous 7y ago
And a drug addict, by analysis of behavior, is committing suicide so the AI should help them achieve their inferred goal based on that behavior. Nevermind asking them if that is their intent, it knows better because it sees the behavior. Russell's laws encode NO limits and doesn't even demand the AI check its judgment with the so-called beneficiaries of its decisions. That is most assuredly worse.
- 7y ago

edmundsauto 7y ago

Interestingly, I believe that what we perceive as our intents/wants/needs/motivations is just an internal genie. Put another way, we have no idea what we really want, consciously. Look at the Schindler's List Netflix example. Also, "A person can do what they want, but not want what they want."