3 ms·

Magic artificial superintelligence solving all the problems of the world and also is the antagonist: https://www.fanfiction.net/s/10327510/1/A-Bluer-Shade-of-Wh

by qznc 12y ago

Magic artificial superintelligence solving all the problems of the world and also is the antagonist: https://www.fanfiction.net/s/10327510/1/A-Bluer-Shade-of-White https://www.fanfiction.net/s/10327510/1/A-Bluer-Shade-of-Whi... (Disclaimer: Frozen fan fiction)

eli_gottlieb 12y ago
Wait... without actually clicking the link... it solves all the world's problems and is the antagonist? Goddamnit LessWrong, just write one damn story with a properly Friendly AI, just for once, just to show it can be done.
- PeterisP 12y ago
  I can't really imagine a story with a properly Friendly AI that would actually be a good story and not boring. A story generally needs some tension, conflict and resolution, and a Friendly AI story (other than about its emergence despite resistance from humanity) wouldn't have it. That being said, maybe "Friendship is optimal" fanfic (http://www.fimfiction.net/story/62074/friendship-is-optimal http://www.fimfiction.net/story/62074/friendship-is-optimal) is something that you have in mind, as it does have a properly friendly AI. I'm not just sure if it's a good enough story, for the reasons I stated above.
  - eli_gottlieb 12y ago
    On behalf of everyone at LessWrong, I wish to convey that "Destroy all nonhuman life in the universe and force all humans to play a tie-in licensed video-game nonstop for the rest of eternity" is most emphatically NOT what we have in mind when we say "Friendly AI".
    - PeterisP 12y ago
      If I make a (possibly false) assumption that our coherent extrapolated volition would include guaranteeing the survival of us and AI that makes that volition happen, then it does seem to imply that it will include destruction, assimilation or permanent power-limitation of all other life in the universe. And an outcome of Friendly AI is all humanity 'living happily ever after' - if we avoid the details of what exactly would 'living happily' look like (since we likely are unable to define that before actually implementing FAI and/or CEV, and the exact result could also extremely vary depending on flavor and degree of [un]Friendliness that is achieved, and none of us know if that will involve ponies), then 'living happily' would quite likely include converting the entire universe to an environment where the 'living happily' happens nonstop for the rest of eternity. And, unless there's the rather implausible 100% agreement of everyone or an ability to create arbitrary numbers of new, separate universes, yes, Friendly AI would still involve either some people being 'forced' to join or those people being left behind in isolated world[s] by almost everyone else. I see no reason to assume that the maximum possible level of Friendliness can make 100% of people 100% happy 100% of time. Choosing universe-future A prevents universe-future B from happening, and anyone who prefers universe-future B will be 'forced' to accept universe-future A if that is the most Friendly choice by whatever exact definition of Friendliness that happens to get implemented.

by qznc 12y ago

eli_gottlieb 12y ago

Wait... without actually clicking the link... it solves all the world's problems and is the antagonist? Goddamnit LessWrong, just write one damn story with a properly Friendly AI, just for once, just to show it can be done.

PeterisP 12y ago
I can't really imagine a story with a properly Friendly AI that would actually be a good story and not boring. A story generally needs some tension, conflict and resolution, and a Friendly AI story (other than about its emergence despite resistance from humanity) wouldn't have it. That being said, maybe "Friendship is optimal" fanfic (http://www.fimfiction.net/story/62074/friendship-is-optimal http://www.fimfiction.net/story/62074/friendship-is-optimal) is something that you have in mind, as it does have a properly friendly AI. I'm not just sure if it's a good enough story, for the reasons I stated above.
- eli_gottlieb 12y ago
  On behalf of everyone at LessWrong, I wish to convey that "Destroy all nonhuman life in the universe and force all humans to play a tie-in licensed video-game nonstop for the rest of eternity" is most emphatically NOT what we have in mind when we say "Friendly AI".
  - PeterisP 12y ago
    If I make a (possibly false) assumption that our coherent extrapolated volition would include guaranteeing the survival of us and AI that makes that volition happen, then it does seem to imply that it will include destruction, assimilation or permanent power-limitation of all other life in the universe. And an outcome of Friendly AI is all humanity 'living happily ever after' - if we avoid the details of what exactly would 'living happily' look like (since we likely are unable to define that before actually implementing FAI and/or CEV, and the exact result could also extremely vary depending on flavor and degree of [un]Friendliness that is achieved, and none of us know if that will involve ponies), then 'living happily' would quite likely include converting the entire universe to an environment where the 'living happily' happens nonstop for the rest of eternity. And, unless there's the rather implausible 100% agreement of everyone or an ability to create arbitrary numbers of new, separate universes, yes, Friendly AI would still involve either some people being 'forced' to join or those people being left behind in isolated world[s] by almost everyone else. I see no reason to assume that the maximum possible level of Friendliness can make 100% of people 100% happy 100% of time. Choosing universe-future A prevents universe-future B from happening, and anyone who prefers universe-future B will be 'forced' to accept universe-future A if that is the most Friendly choice by whatever exact definition of Friendliness that happens to get implemented.