14 ms·
> Is it not an existential threat that large swaths of the population will essentially become redundant in the next decades? This is precisely the crux. In the
by zygi 3y ago
> Is it not an existential threat that large swaths of the population will essentially become redundant in the next decades?
This is precisely the crux. In the scenario where there's no mass-paperclipping etc, AI is not an existential threat. It's a threat to the current regime, and I probably agree with you that the regimes it enables will be worse for people with our current values, but it's not an existential threat to humanity.
The people who worry about AI existential threats primarily worry about one thing: can humanity as a species survive? There are likely scenarios where it won't. Those scenarios should be prevented.
Another frame to think about this is: a libertarian probably wouldn't be concerned about the threats described in your comment. But both a libertarian and a socialist will be worried about paperclipping.
- mario-main 3y agoWhat is paperclipping? I looked it up and the results were all about a dating trend.
- sverona 3y agoIt's a reference to the paperclip maximizer.
- cratermoon 3y ago"The paperclip maximizer is a thought experiment described by Swedish philosopher Nick Bostrom in 2003. It illustrates the existential risk that an artificial general intelligence may pose to human beings were it to be successfully designed to pursue even seemingly harmless goals and the necessity of incorporating machine ethics into artificial intelligence design" https://en.wikipedia.org/wiki/Instrumental_convergence#Paperclip_maximizer https://en.wikipedia.org/wiki/Instrumental_convergence#Paper...
- dTal 3y agoThis has a Pascal's wager sort of feel. The possible negative consequences of a rampant AI are infinite, so we all ought to take it Very Seriously. The trouble is that there are lots of other scenarios with unbounded negative value - for instance, tight centralized control of technology supports a stable dystopia that keeps nearly all humans in a perpetual state of suffering. It's not at all clear, when you add it all up, that "prevent 'misaligned' AIs from being deployed at all costs" is the correct strategy to minimize risk.
- circuit10 3y agoThe difference to Pascal’s wager is that with that the probability is vanishingly small, but given current progress the probability of AI causing an existential threat is actually quite high. You can’t just call anything that has very bad negative consequences Pascal’s wager Video about this: “Is AI Safety a Pascal’s Mugging?” https://youtu.be/JRuNA2eK7w0 https://youtu.be/JRuNA2eK7w0
- circuit10 3y agoI sort of mixed up “wager” and “mugging” here but you get the point
- dTal 3y agoNot at all, "Pascal's mugging" was exactly the term I would have used, had I remembered it :) That video is a very good reply. One response that comes to mind is that there is an implicit subtext behind any discussion of AI safety that we must do something about it, and soon. A good "anti-god" for his payoff matrix would be "the act of calling for AI safety will inadvertently be used as political capital by those wishing to be the gatekeepers of powerful technology" - the chances of that are quite high, and the negative consequences quite bad, if not necessarily apocalyptic.