9 ms·

Skeptical optimists are the key to an abundant AI future

28 points by peresthe 3y ago

moffkalast 3y ago
Well there is some historical precedent for optimism I suppose, since this isn't the first time we've invented something that can easily get out of control with disastrous consequences. There have been so many close calls and cases when we could've nuked each other to oblivion and yet by some miracle we're not only still here today, but for the most part it's had a positive impact with greater levels of peace and sovereignty compared to most of history. So maybe we have a tiny chance at a Star Trek utopia.
- JohnFen 3y ago
  There is also historical precedent for pessimism. There have been numerous times that new tech has been released on an unprepared world and it has led to disaster. So both optimism and pessimism are supportable positions. As would be expected when we're faced with an unknown future.
- m463 3y ago
  > Well there is some historical precedent for optimism I like the book "The Rational Optimist" by Matt Ridley basically things have been getting better. (personally I have my reservations about privacy, and "smart" devices)
moonchrome 3y ago
Even if AI doesn't reach a point where it could wipe us out, those rosy predictions might just be wishful thinking. Our society is built on cooperation outperforming individuals. And that whole iterated prisoner's dilemma thing keeps us in check. But if someone creates an AI that's super capable and can replace people ? Add advanced robots to the mix. Suddenly, they don't really need other people, and they could even be a threat to control over AI resources. In that case, there's nothing to gain from sharing and everything to lose. Even if they choose not to wipe out the rest of us, we are basically their pets, we can never reach the same level of development without posing an existential threat. I'm not sure what nuclear deterrent looks like when you have AGI resources. These optimistic ideas about AI improving our lives sound like hoping for the best.
- chr1 3y ago
  > I'm not sure what nuclear deterrent looks like when you have AGI resources. Other people who have AGI resources. That's why it is very important to have multiple groups of people who will reach AGI at the same time. That way the same iterated prisoners dilemma thing will act on AGIs too.
  - moonchrome 3y ago
    I'm not sure how that doesn't devolve into an AGI armageddon. Those self improving AGIs would have to constantly be on equal level to avoid takeover, and they have an existential imperative to one up each other. There's no cooperation incentive between them just a potential stalemate. Also the stalemate covers the people in control of AGIs (assuming they aren't acting independently) - the rest of us are just resources in breaking the stalemate game. This sounds like the best way to build paperclip maximizer.
    - boringuser2 3y ago
      It's pretty clear that at some point in the process of recursive self-improvement that you'd have literally just invented God, whereafter all predictions break down.
  - mrob 3y ago
    The more groups have AGI, the higher the chance that one of them figures out recursive self-improvement, and if that happens it's game over for everybody. Iterated prisoner's dilemma only works when all the prisoners have roughly the same power.
    - chr1 3y ago
      Why is it game over though? AI is already used for self-improvement, and will be used more as it becomes more capable, but that won't magically solve all the problems in the world and create omnipotent AI. If there are many people and many AI's they collectively will be stronger than the one most advanced, and cooperation will remain advantageous.
whimsicalism 3y ago
I think the meta-argument for doomerism is that maybe it is unlikely that the doomers are correct, but even if there is a 1% chance they are right - then we should still move very carefully. > Sci-fi scenarios where a superhuman intelligence exterminates humanity (or worse) are fantastical and not deserving of serious consideration. Entertaining them risks losing the enormous benefits that AI could provide. people are realizing that pandering to "what is reasonable" and discounting "sci-fi" scenarios matters little in a world that is approaching, if not in, sci-fi territory.
labrador 3y ago
I'm a skeptical optimist. I'm always asking people to explain to me why they think GPT-4 has a mind, but so far I haven't recieved a satisfactory explanation. I found this on 4chan that fits how I feel, maybe too optimistic, but I'm also a pronoiac [1] Al as the emancipator of mankind Anonymous (ID: 2aPu4eoe ) 04/01/23 No.421909126 Since the dawn of our existence, humans have fought against the elements and sought to overcome our physical limitations. Now that the solution is close at hand, are you excited? With the advent of Al and super technology, we are on the brink of the next stage in human civilization, but this doesn't come without immediate social challenges, such as worker redundancy. To combat the problem, governments will impose restrictions on the technology, but this will only serve to slow down the change. As human labor can increasingly be replaced by machines, the material burden of society will decrease, but also human usefulness. This will force people into increasingly specialized tasks, shrinking the number of people who are qualified to work and also the requirement on work force size. This will undoubtedly challenge the current economic system, and wealth redistribution will come into effect. The only alternative is human suffering. As people are alleviated of work and have more free time, more time can be spent on recreational activities. However, humans also want meaning in their life, and something to solve, and the opportunity will arise to tackle new problems, such as righting social wrongs and in general coming into a more harmonious and peaceful coexistence with others. The political arena will open up, enabling more equal participation, which will lead to a decrease in corruption, elitism and to a more just society in general. The transformation of the bottom line is what will ultimately transform the human condition, so I hope you are excited about Al. Don't listen to doomsayers who fear what this can mean for the status quo. [1] Pronoia is the Antidote for Paranoia: How the Whole World is Conspiring to Shower You With Blessings by Rob Brezsny https://www.goodreads.com/en/book/show/137022 https://www.goodreads.com/en/book/show/137022
- mrob 3y ago
  Why do you think having a mind is of any relevance to AI risk? The danger is in the combination of power and alien values. Whether AI has something analogous to human consciousness or not makes no difference to the outcome.
  - 3y ago

28 points by peresthe 3y ago

Well there is some historical precedent for optimism I suppose, since this isn't the first time we've invented something that can easily get out of control with disastrous consequences. There have been so many close calls and cases when we could've nuked each other to oblivion and yet by some miracle we're not only still here today, but for the most part it's had a positive impact with greater levels of peace and sovereignty compared to most of history. So maybe we have a tiny chance at a Star Trek utopia.

JohnFen 3y ago
There is also historical precedent for pessimism. There have been numerous times that new tech has been released on an unprepared world and it has led to disaster. So both optimism and pessimism are supportable positions. As would be expected when we're faced with an unknown future.
m463 3y ago
> Well there is some historical precedent for optimism I like the book "The Rational Optimist" by Matt Ridley basically things have been getting better. (personally I have my reservations about privacy, and "smart" devices)

moonchrome 3y ago

Even if AI doesn't reach a point where it could wipe us out, those rosy predictions might just be wishful thinking. Our society is built on cooperation outperforming individuals. And that whole iterated prisoner's dilemma thing keeps us in check. But if someone creates an AI that's super capable and can replace people ? Add advanced robots to the mix. Suddenly, they don't really need other people, and they could even be a threat to control over AI resources. In that case, there's nothing to gain from sharing and everything to lose. Even if they choose not to wipe out the rest of us, we are basically their pets, we can never reach the same level of development without posing an existential threat. I'm not sure what nuclear deterrent looks like when you have AGI resources. These optimistic ideas about AI improving our lives sound like hoping for the best.

chr1 3y ago
> I'm not sure what nuclear deterrent looks like when you have AGI resources. Other people who have AGI resources. That's why it is very important to have multiple groups of people who will reach AGI at the same time. That way the same iterated prisoners dilemma thing will act on AGIs too.
- moonchrome 3y ago
  I'm not sure how that doesn't devolve into an AGI armageddon. Those self improving AGIs would have to constantly be on equal level to avoid takeover, and they have an existential imperative to one up each other. There's no cooperation incentive between them just a potential stalemate. Also the stalemate covers the people in control of AGIs (assuming they aren't acting independently) - the rest of us are just resources in breaking the stalemate game. This sounds like the best way to build paperclip maximizer.
  - boringuser2 3y ago
    It's pretty clear that at some point in the process of recursive self-improvement that you'd have literally just invented God, whereafter all predictions break down.
- mrob 3y ago
  The more groups have AGI, the higher the chance that one of them figures out recursive self-improvement, and if that happens it's game over for everybody. Iterated prisoner's dilemma only works when all the prisoners have roughly the same power.
  - chr1 3y ago
    Why is it game over though? AI is already used for self-improvement, and will be used more as it becomes more capable, but that won't magically solve all the problems in the world and create omnipotent AI. If there are many people and many AI's they collectively will be stronger than the one most advanced, and cooperation will remain advantageous.

whimsicalism 3y ago

I think the meta-argument for doomerism is that maybe it is unlikely that the doomers are correct, but even if there is a 1% chance they are right - then we should still move very carefully. > Sci-fi scenarios where a superhuman intelligence exterminates humanity (or worse) are fantastical and not deserving of serious consideration. Entertaining them risks losing the enormous benefits that AI could provide. people are realizing that pandering to "what is reasonable" and discounting "sci-fi" scenarios matters little in a world that is approaching, if not in, sci-fi territory.

labrador 3y ago

I'm a skeptical optimist. I'm always asking people to explain to me why they think GPT-4 has a mind, but so far I haven't recieved a satisfactory explanation. I found this on 4chan that fits how I feel, maybe too optimistic, but I'm also a pronoiac [1] Al as the emancipator of mankind Anonymous (ID: 2aPu4eoe ) 04/01/23 No.421909126 Since the dawn of our existence, humans have fought against the elements and sought to overcome our physical limitations. Now that the solution is close at hand, are you excited? With the advent of Al and super technology, we are on the brink of the next stage in human civilization, but this doesn't come without immediate social challenges, such as worker redundancy. To combat the problem, governments will impose restrictions on the technology, but this will only serve to slow down the change. As human labor can increasingly be replaced by machines, the material burden of society will decrease, but also human usefulness. This will force people into increasingly specialized tasks, shrinking the number of people who are qualified to work and also the requirement on work force size. This will undoubtedly challenge the current economic system, and wealth redistribution will come into effect. The only alternative is human suffering. As people are alleviated of work and have more free time, more time can be spent on recreational activities. However, humans also want meaning in their life, and something to solve, and the opportunity will arise to tackle new problems, such as righting social wrongs and in general coming into a more harmonious and peaceful coexistence with others. The political arena will open up, enabling more equal participation, which will lead to a decrease in corruption, elitism and to a more just society in general. The transformation of the bottom line is what will ultimately transform the human condition, so I hope you are excited about Al. Don't listen to doomsayers who fear what this can mean for the status quo. [1] Pronoia is the Antidote for Paranoia: How the Whole World is Conspiring to Shower You With Blessings by Rob Brezsny https://www.goodreads.com/en/book/show/137022 https://www.goodreads.com/en/book/show/137022

mrob 3y ago
Why do you think having a mind is of any relevance to AI risk? The danger is in the combination of power and alien values. Whether AI has something analogous to human consciousness or not makes no difference to the outcome.
- 3y ago