4 ms·

> "Along with research grants, the program will also include meetings and outreach programs aimed at bringing together academic AI researchers, industry AI deve

by rl3 12y ago

> "Along with research grants, the program will also include meetings and outreach programs aimed at bringing together academic AI researchers, industry AI developers and other key constituents to continue exploring how to maximize the societal benefits of AI; ..." Maximizing the societal benefits of AI, at least in the context of superintelligence, is a very slippery slope. For sure, it beats a myraid of malignant failure and perverse instantiation scenarios, where humanity becomes extinct (or worse). However, I believe that if we do somehow manage to solve the seemingly intractable AGI control problem and create an entirely safe superintelligence that is loyal to our every whim, an entirely new ethical challenge will arise. For example, let us assume this has been accomplished, and humanity now has a friendly superintelligence that has established itself as a global singleton (i.e. it's incredibly powerful, and nothing terrestrial can supplant it). Humans instruct it to solve the problem of death and disease. It does so. Humans instruct it to solve the problem of crime. It does so. Humans instruct it to solve the problem of war. It does so. Humans instruct it to solve the problem of poverty. It does so. Humans instruct it to explore the universe and solve the most vexing existential questions. It does so. In that scenario, the entire worldwide medical, law enforcement, and military professions just ceased to exist. Altruism no longer has much of a place; poverty and disease no longer exist. Overnight, death became a thing of the past. Humanity no longer ponders why it exists or how the universe works, because such questions have already been pondered by the AI to whatever possible maxima. That example is incomplete for the sake of brevity, but it's easy to imagine such a scenario resulting in all of our problems being solved for us. Which then begs the question, what would become of our humanity when there is no longer struggle or suffering? Post-scarcity economics, as well as literature or philosophy that concerns itself with the perils of utopia may offer some insight into this question, but I believe the topic deserves a more in-depth investigation specifically within the context of superintelligence. For my two cents, I'm starting to believe that any superintelligence we construct that establishes itself as a global singleton should have an extremely finite set of goals, perhaps even a singular goal, where it henceforth restricts any other forms of AGI from existing. While it may be tempting to also have this entity protect us against other forms of existential risk, or perhaps fix some of the truly awful suffering in the world, doing so would still remain a very slippery slope.

eli_gottlieb 12y ago
Excuse me, but if you can't come up with anything to do with yourself after suffering and despair are gone from the world, that's your failure of imagination. I'm pretty sure that if really placed into such a situation, you will find a way to cope, just like the rest of us.
- rl3 12y ago
  I'd like to think I would cope just fine. In fact, I'd probably be elated. My concern was more what would happen on a societal scale, what would become of humanity as it was known.
lione 12y ago
I'd think that that friendly super intelligence would realize the issues associated with post-scarcity society and would act in the best interests of humanity. It would weight human happiness as more important and work to avoid the pitfalls that would be undesirable. There's more utility for it in the human population being happy/not giving up on life then most of the things it would do (assuming it's primary goal is to further and safeguard the human race). There would be aspects of society it COULD run, but feel that giving us purpose would be more beneficial then the increase in productivity or whatever of it running that aspect of society. Maybe it can make beautiful gardens and art, but instead leaves it to the humans to give them some sort of purpose. I don't see how or why a post-scarcity society would do away with art/culture/many of the subjective things that make life worth living and provide purpose. Most people don't quit doing the things they like just because someone is better at it. My two cents.
- rl3 12y ago
  I think you may be right. Admittedly, most of what I said was really only applicable to a Genie superintelligence that implements our wishes on a command-by-command basis. A friendly Sovereign superintelligence (i.e. operating under its own goal/value-directed volition) would probably not rob us of purpose, though the societal changes in either scenario would likely still be quite profound. Of course, a positive superintelligence outcome would be a miracle in and of itself, so there's that. Coming to terms with a newly created utopian society sure beats extinction.

by rl3 12y ago

eli_gottlieb 12y ago

Excuse me, but if you can't come up with anything to do with yourself after suffering and despair are gone from the world, that's your failure of imagination. I'm pretty sure that if really placed into such a situation, you will find a way to cope, just like the rest of us.

rl3 12y ago
I'd like to think I would cope just fine. In fact, I'd probably be elated. My concern was more what would happen on a societal scale, what would become of humanity as it was known.

lione 12y ago

I'd think that that friendly super intelligence would realize the issues associated with post-scarcity society and would act in the best interests of humanity. It would weight human happiness as more important and work to avoid the pitfalls that would be undesirable. There's more utility for it in the human population being happy/not giving up on life then most of the things it would do (assuming it's primary goal is to further and safeguard the human race). There would be aspects of society it COULD run, but feel that giving us purpose would be more beneficial then the increase in productivity or whatever of it running that aspect of society. Maybe it can make beautiful gardens and art, but instead leaves it to the humans to give them some sort of purpose. I don't see how or why a post-scarcity society would do away with art/culture/many of the subjective things that make life worth living and provide purpose. Most people don't quit doing the things they like just because someone is better at it. My two cents.

rl3 12y ago
I think you may be right. Admittedly, most of what I said was really only applicable to a Genie superintelligence that implements our wishes on a command-by-command basis. A friendly Sovereign superintelligence (i.e. operating under its own goal/value-directed volition) would probably not rob us of purpose, though the societal changes in either scenario would likely still be quite profound. Of course, a positive superintelligence outcome would be a miracle in and of itself, so there's that. Coming to terms with a newly created utopian society sure beats extinction.