3 ms·

This article says that we can't trust the estimates of p(doom), therefore we should take no action. But it assumes that "no action" means spending many billions

This article says that we can't trust the estimates of p(doom), therefore we should take no action. But it assumes that "no action" means spending many billions of dollars to develop advanced AI. But why is that our default? I could just as well say we can't trust the estimates of p(survival), therefore we should fall back on a default action of not developing advanced AI.

DisgracePlacard 2y ago
I don't think there's a real need to justify technological progress as a default. That has been the default for at least a century or so, and I think it's done us quite well. The unorthodox thing is the idea that we should avoid technological progress, and if there isn't good evidence for that, then we should ignore it.
- DennisP 2y ago
  It's our default because for most technology, we have high confidence of a high p(survival).
- vannevar 2y ago
  >[Technological progress] has been the default for at least a century or so, and I think it's done us quite well. Such progress has presented an existential threat only once previously in the past century, and that was the development of nuclear weapons. The jury is still out on whether that was the right decision. I don't think the argument that, "Well, nothing bad has happened yet!" is very persuasive in the face of possible extermination of the human race.
- krageon 2y ago
  It's escalated humanity's incapability to look toward the future or do long-term planning in any reliable kind of way very problematic. It's also made the tendency for humans to go to war significantly more problematic for everyone that isn't already a murdering loon. While there's plenty of positive effects of the march of technological progress, let's not pretend it's done us an unequivocally good turn. The state of the environment is enough to explain that.
Buttons840 2y ago
And p(doom) is non-zero with or without developing AI. It may be that p(doom) is less with AI than without.
tim333 2y ago
>should fall back on a default action of not developing advanced AI Not going to happen. It's so potentially valuable economically and militarily that if one country doesn't develop it their rivals will.

DisgracePlacard 2y ago

I don't think there's a real need to justify technological progress as a default. That has been the default for at least a century or so, and I think it's done us quite well. The unorthodox thing is the idea that we should avoid technological progress, and if there isn't good evidence for that, then we should ignore it.

DennisP 2y ago
It's our default because for most technology, we have high confidence of a high p(survival).
vannevar 2y ago
>[Technological progress] has been the default for at least a century or so, and I think it's done us quite well. Such progress has presented an existential threat only once previously in the past century, and that was the development of nuclear weapons. The jury is still out on whether that was the right decision. I don't think the argument that, "Well, nothing bad has happened yet!" is very persuasive in the face of possible extermination of the human race.
krageon 2y ago
It's escalated humanity's incapability to look toward the future or do long-term planning in any reliable kind of way very problematic. It's also made the tendency for humans to go to war significantly more problematic for everyone that isn't already a murdering loon. While there's plenty of positive effects of the march of technological progress, let's not pretend it's done us an unequivocally good turn. The state of the environment is enough to explain that.

Buttons840 2y ago

And p(doom) is non-zero with or without developing AI. It may be that p(doom) is less with AI than without.

tim333 2y ago

>should fall back on a default action of not developing advanced AI Not going to happen. It's so potentially valuable economically and militarily that if one country doesn't develop it their rivals will.