4 ms·
> The only thing worse than the "evil" genie giving us exactly what we ask for is the genie that decides its inferences about our actual intents/wants/needs/mot
by ajuc 7y ago
> The only thing worse than the "evil" genie giving us exactly what we ask for is the genie that decides its inferences about our actual intents/wants/needs/motivations based purely on our behaviors matter more than what we intentionally express as our intents/wants/needs/motivations.
It's not worse.
> a date rapist
A rapist shares 99+% of definitions and values with you as a fellow human being. AI won't (unless you somehow program them in). Rapist has his evil motivations, but won't suddenly invent a virus that kills the whole human species because you asked him to stop neighbor kids trampling your garden. AI might. If you tell it not to kill anybody it might destroy our civilization to prevent us from killing ourselves with global warming. Why not - it only makes sense.
Simplest way to ensure safety for the maximum number of human beings is to anesthetize everybody and put them on life support till their natural death. Perfect record is possible - you might cure addicts and prevent crimes and wars. If you specify people have to be awake as often as they usually are - it can keep them awake but restrained. You want people to have freedom of movement? Ok - you just released the whole prison population :) Keeping criminals in prison is OK? Then it may make everybody a criminal for a quick fix. Or just drug everybody to WANT to be restrained. And so on, and so on.
There's infinite number of possible courses of action that we discard without consciously thinking about them because of our assumptions. You have to put these assumptions in the AI, each and every one of them, and they are very subtle and invisible for us most of the time. And they often border on philosophy and morality, and defining them is political by definition.
It's probably impossible to code all our values and assumptions in by hand. That's a much bigger problem for safe general AI than a post-factum explanations that rub your morality the wrong way.
> Asimov's Three Laws
Are self-contradictory and useless for anything except literature.
- fallous 7y agoAnd a drug addict, by analysis of behavior, is committing suicide so the AI should help them achieve their inferred goal based on that behavior. Nevermind asking them if that is their intent, it knows better because it sees the behavior. Russell's laws encode NO limits and doesn't even demand the AI check its judgment with the so-called beneficiaries of its decisions. That is most assuredly worse.
- ajuc 7y agoDecoding the preferences of people from their behavior is just a first step, it's not the only rule, it's a rule that makes writing other rules safer than they would be otherwise.
- baddox 7y agoI don't think that "sharing 99+% of definitions and values with us as a fellow human being" has really provided much protection against deliberate human-caused disasters. It's true that I wouldn't expect most people to invent a virus as a weapon, but I absolutely would expect governments to do so, and in fact I think they have done so.
- ajuc 7y ago> I don't think that "sharing 99+% of definitions and values with us as a fellow human being" has really provided much protection against deliberate human-caused disasters. There was this thing called "cold war".
- baddox 7y agoTens of millions of people were killed in Cold War violence. It was a huge disaster caused intentionally by “fellow humans.”
- ajuc 7y agoI have issues with that estimate, but even accepting it - it was still a big improvement over WW2, WW1, colonization, Mongol invasions etc. And certainly preferable to a global thermonuclear war which was a real possibility. Have you heard of https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar...
- baddox 7y agoHumans very deliberately created nuclear weapons, with the intention of using them to kill enormous numbers of people, which they did. The fact that humans created something horrible and then used that thing fewer times than they could have is hardly a triumph of humans, and it certainly isn’t great evidence to support the argument that humans are limited in their capacity for doing bad things because they supposedly share values with other humans.