5 ms·

The utopian future is governments run by AI. The Culture series explores this to some degree.

ben_w 3y ago
Fiction isn't a good reference point for reality. The dystopian future is also governments run by AI. Terminator so much so it's a cliché, but also I Have No Mouth, and I Must Scream. Also, never read this, but people sometimes point it out as an example of "Can we, like, not do that?": https://tvtropes.org/pmwiki/pmwiki.php/Fanfic/FriendshipIsOptimal https://tvtropes.org/pmwiki/pmwiki.php/Fanfic/FriendshipIsOp...
- unsupp0rted 3y ago
  Science fiction is a great reference point for reality. The people creating our reality today all read the same books I did as kids. Humans make lousy governors and suck at minimizing mass suffering. Barring I have no mouth and I must scream scenarios or Butlerian Jihad scenarios, AI can make a better world for us (or for whatever replaces us). The alternative is for our monkey brains to keep ramming each other with cars, slamming projectiles into ships to draw political attention, getting angry at people with the wrong skin color, etc I don’t want another 10,000 years of that.
  - ben_w 3y ago
    > Barring I have no mouth and I must scream scenarios or Butlerian Jihad scenarios, AI can make a better world for us (or for whatever replaces us). That's a fully generalisable statement: Barring ${environmental degradation and worker rights issues}, ${laissez faire economics} can make a better world. Barring ${racism}, ${colonialism} can make a better world. Barring ${dictatorships}, ${communism} can make a better world. Barring ${a Kessler cascade}, ${a Dyson swarm} can make a better world. > The alternative is for our monkey brains to keep ramming each other with cars, slamming projectiles into ships to draw political attention, getting angry at people with the wrong skin color, etc Our limited monkey brains are, unfortunately, also the reference example for all the AI we want to make. Human-like alignment with human-like interests and human-like values. The default is this plus bugs from the software not doing what we thought we wanted it to do, which means all those things but done at machine speed rather than biological speed plus a bunch of other things that just make no sense when they happen. The bugs in particular can be, and in some cases have already been, weird beyond human comprehension. I hope the safety/alignment people don't get caught up in either a scandal or a philosophical cul-de-sac.

Fiction isn't a good reference point for reality. The dystopian future is also governments run by AI. Terminator so much so it's a cliché, but also I Have No Mouth, and I Must Scream. Also, never read this, but people sometimes point it out as an example of "Can we, like, not do that?": https://tvtropes.org/pmwiki/pmwiki.php/Fanfic/FriendshipIsOptimal https://tvtropes.org/pmwiki/pmwiki.php/Fanfic/FriendshipIsOp...

unsupp0rted 3y ago
Science fiction is a great reference point for reality. The people creating our reality today all read the same books I did as kids. Humans make lousy governors and suck at minimizing mass suffering. Barring I have no mouth and I must scream scenarios or Butlerian Jihad scenarios, AI can make a better world for us (or for whatever replaces us). The alternative is for our monkey brains to keep ramming each other with cars, slamming projectiles into ships to draw political attention, getting angry at people with the wrong skin color, etc I don’t want another 10,000 years of that.
- ben_w 3y ago
  > Barring I have no mouth and I must scream scenarios or Butlerian Jihad scenarios, AI can make a better world for us (or for whatever replaces us). That's a fully generalisable statement: Barring ${environmental degradation and worker rights issues}, ${laissez faire economics} can make a better world. Barring ${racism}, ${colonialism} can make a better world. Barring ${dictatorships}, ${communism} can make a better world. Barring ${a Kessler cascade}, ${a Dyson swarm} can make a better world. > The alternative is for our monkey brains to keep ramming each other with cars, slamming projectiles into ships to draw political attention, getting angry at people with the wrong skin color, etc Our limited monkey brains are, unfortunately, also the reference example for all the AI we want to make. Human-like alignment with human-like interests and human-like values. The default is this plus bugs from the software not doing what we thought we wanted it to do, which means all those things but done at machine speed rather than biological speed plus a bunch of other things that just make no sense when they happen. The bugs in particular can be, and in some cases have already been, weird beyond human comprehension. I hope the safety/alignment people don't get caught up in either a scandal or a philosophical cul-de-sac.