6 ms·

The real cost of fearing an AGI now is an opportunity cost. The technology is a long ways away. Other events -- such as ecological collapse or nuclear disaster

by vonnik 11y ago

The real cost of fearing an AGI now is an opportunity cost. The technology is a long ways away. Other events -- such as ecological collapse or nuclear disaster to name two -- are much more likely to irremediably affect our lives. So by hopping on Bostrom's fear wagon, you are making a bet on several levels: a) this matters now; b) this matters more than other dangers we could devote ourselves to minimizing; and c) something can actually be done about it. All three bets seem like bad ones to make to me.

rw 11y ago
Fair comparison, but I didn't intend any monopoly on fear. All I'm saying is that the risk of GAI should be taken seriously, calibrated along with the other problems you mention. There's a lot of room between 'irrelevant' and 'the most important issue' :-)
- argonaut 11y ago
  Yes, and given the $1B in funding that OpenAI has, I'd seriously say that too much attention is given to the AI "risk."
  - Houshalter 11y ago
    OpenAI has nothing to do with AI risk. They want to accelerate research in AI and make it open to the public. Which is the exact opposite you would want to do to protect against superintelligent AIs. They are not focusing on solving the control problem, or anything meaningful.
    - argonaut 11y ago
      This is clearly false. OpenAI would not exist if Sam Altman and Elon Musk were not worried about AI.
    - vonnik 11y ago
      Actually, @Houshalter, OpenAI has several purposes, and one of them is to address AI risk. You may not think that is an effective approach, but that is their stated intent. They believe that the best hedge against the dangers of superintelligence is to conduct open research that encourages a multipolar world of several AGIs, rather a unipolar world where one company, such as Google, reaps the benefits and determines the course of such an advance. It's true that OpenAI accelerates AI research, which makes it paradoxical. But that's the nature of a hedge. You bet on both sides at once. I do believe that they will focus on solving meaningful problems (they are only a few weeks old...).
      - Houshalter 11y ago
        >rather a unipolar world where one company, such as Google, reaps the benefits and determines the course of such an advance Yes that is their model of AI risk. However the people worried about AI risk are not worried about any single organization controlling AI. They are worried about a world where no one controls AI. Controlling AI is actually a very difficult and unsolved problem. OpenAI is not intended to solve that problem, and if anything they are making it far worse by accelerating AI research before the problem has been solved.
        argonaut 11y ago
        What do you mean by unsolved problem? Controlling AI is not even an actual problem at all yet. It's merely a hypothetical problem.
Houshalter 11y ago
I would make those bets. I'm not afraid of those other things at all. Nukes are really unlikely to be used, and even if they were it wouldn't destroy the whole world, let alone end humanity. Climate change is going to cause lots of economic damage at worst, but it isn't going to destroy the world, let alone end humanity. Same with many other proposed risks. AI is the only thing I believe has a very high probability of destroying all humanity. Probably within our lifetimes. And there is probably something we can do about it right now - working on the control problem.
- vonnik 11y ago
  It seems to me like you're discounting real and present dangers in favor of one that's more speculative. The reason why climate change and nuclear disasters (I did not limit it to nuclear weapons, but am referring also to events like Chernobyl and Fukushima, which have poisoned significant regions of the planet for millennia to come) are real and present is because a) climate change is already under way and causing massive demographic shifts; and b) nuclear technology is already in a state to destroy large parts of the species and render much of the planet uninhabitable. We are one button click away from a bomb going off at any time, and there are lots of crazy fingers in the world itching for a button.
  - Houshalter 11y ago
    Even nuclear meltdowns aren't' enough to literally destroy the planet. In the worst case they might kill a few thousand people and cause billions in economic damage. Which sounds like a lot, but on the scale of the whole planet is almost nothing. All of them are real dangers, and present dangers at that. But they still are not existential dangers. Even in the worst case humanity wouldn't go extinct. It's unlikely even modern civilization would collapse, just a few countries. AI is one of the few things that can literally wipe out humanity, or even the entire universe. So I think it's worth special consideration. Personally I also believe it's far more likely as well. And people are already doing things about climate change and nuclear proliferation. Until very recently, there were only a few people working on AI risk, and it's still nowhere near enough.

by vonnik 11y ago

rw 11y ago

Fair comparison, but I didn't intend any monopoly on fear. All I'm saying is that the risk of GAI should be taken seriously, calibrated along with the other problems you mention. There's a lot of room between 'irrelevant' and 'the most important issue' :-)

argonaut 11y ago
Yes, and given the $1B in funding that OpenAI has, I'd seriously say that too much attention is given to the AI "risk."
- Houshalter 11y ago
  OpenAI has nothing to do with AI risk. They want to accelerate research in AI and make it open to the public. Which is the exact opposite you would want to do to protect against superintelligent AIs. They are not focusing on solving the control problem, or anything meaningful.
  - argonaut 11y ago
    This is clearly false. OpenAI would not exist if Sam Altman and Elon Musk were not worried about AI.
  - vonnik 11y ago
    Actually, @Houshalter, OpenAI has several purposes, and one of them is to address AI risk. You may not think that is an effective approach, but that is their stated intent. They believe that the best hedge against the dangers of superintelligence is to conduct open research that encourages a multipolar world of several AGIs, rather a unipolar world where one company, such as Google, reaps the benefits and determines the course of such an advance. It's true that OpenAI accelerates AI research, which makes it paradoxical. But that's the nature of a hedge. You bet on both sides at once. I do believe that they will focus on solving meaningful problems (they are only a few weeks old...).
    - Houshalter 11y ago
      >rather a unipolar world where one company, such as Google, reaps the benefits and determines the course of such an advance Yes that is their model of AI risk. However the people worried about AI risk are not worried about any single organization controlling AI. They are worried about a world where no one controls AI. Controlling AI is actually a very difficult and unsolved problem. OpenAI is not intended to solve that problem, and if anything they are making it far worse by accelerating AI research before the problem has been solved.
      - argonaut 11y ago
        What do you mean by unsolved problem? Controlling AI is not even an actual problem at all yet. It's merely a hypothetical problem.

Houshalter 11y ago

I would make those bets. I'm not afraid of those other things at all. Nukes are really unlikely to be used, and even if they were it wouldn't destroy the whole world, let alone end humanity. Climate change is going to cause lots of economic damage at worst, but it isn't going to destroy the world, let alone end humanity. Same with many other proposed risks. AI is the only thing I believe has a very high probability of destroying all humanity. Probably within our lifetimes. And there is probably something we can do about it right now - working on the control problem.

vonnik 11y ago
It seems to me like you're discounting real and present dangers in favor of one that's more speculative. The reason why climate change and nuclear disasters (I did not limit it to nuclear weapons, but am referring also to events like Chernobyl and Fukushima, which have poisoned significant regions of the planet for millennia to come) are real and present is because a) climate change is already under way and causing massive demographic shifts; and b) nuclear technology is already in a state to destroy large parts of the species and render much of the planet uninhabitable. We are one button click away from a bomb going off at any time, and there are lots of crazy fingers in the world itching for a button.
- Houshalter 11y ago
  Even nuclear meltdowns aren't' enough to literally destroy the planet. In the worst case they might kill a few thousand people and cause billions in economic damage. Which sounds like a lot, but on the scale of the whole planet is almost nothing. All of them are real dangers, and present dangers at that. But they still are not existential dangers. Even in the worst case humanity wouldn't go extinct. It's unlikely even modern civilization would collapse, just a few countries. AI is one of the few things that can literally wipe out humanity, or even the entire universe. So I think it's worth special consideration. Personally I also believe it's far more likely as well. And people are already doing things about climate change and nuclear proliferation. Until very recently, there were only a few people working on AI risk, and it's still nowhere near enough.