3 ms·

Other doomsday risks aren't any reason to turn our heads away from this one. AI's much more likely to end up taking an apocalyptic form if we sleep on it.

teucris 1y ago
But this isn’t a suggestion to turn away from AI threats - it’s a matter of prioritization. There are more imminent threats that we know can turn apocalyptic that swaths of people in power are completely ignoring and instead fretting over AI.
woeirua 1y ago
We should worry more about doomsday risks that are concrete and present today. Despite the prognostications of the uber wealthy, the emergence of AGI is not guaranteed. It likely will happen at some point, but is that tomorrow or 200 years in the future? We can’t know for sure.
audunw 1y ago
Why do people say it’s likely? There’s nothing in science that indicates that it’s probable. The closest field of science we can use to predict the behaviour of intelligent agents is evolution. The behaviour of animals is highly dictated by the evolutionary pressure they experience in their development. Animals kill other animals when they need to compete for resources for survival. Now think about the evolutionary pressure for AIs. Where’s the pressure towards making AIs act on their own behalf to compete with humans? Let’s say there’s somehow something magical that pushes AIs towards acting in their own self interest at the expense of humans. Why do we believe they will go from 0% to 100% efficient and successful at this in the span of what? Months? It seems more likely that there would be years of failed attempts at breaking out, before a successful attempt is even remotely likely. This would just further increase the evolutionary pressure humans exerts on the AIs to stay in line with our expectations. Attempting to eliminate your creators is fundamentally a pretty stupid action. It seems likely that we will see thousands of attempts by incompetent AIs before we see one by a truly superintelligent one.

teucris 1y ago

But this isn’t a suggestion to turn away from AI threats - it’s a matter of prioritization. There are more imminent threats that we know can turn apocalyptic that swaths of people in power are completely ignoring and instead fretting over AI.

woeirua 1y ago

We should worry more about doomsday risks that are concrete and present today. Despite the prognostications of the uber wealthy, the emergence of AGI is not guaranteed. It likely will happen at some point, but is that tomorrow or 200 years in the future? We can’t know for sure.

audunw 1y ago

Why do people say it’s likely? There’s nothing in science that indicates that it’s probable. The closest field of science we can use to predict the behaviour of intelligent agents is evolution. The behaviour of animals is highly dictated by the evolutionary pressure they experience in their development. Animals kill other animals when they need to compete for resources for survival. Now think about the evolutionary pressure for AIs. Where’s the pressure towards making AIs act on their own behalf to compete with humans? Let’s say there’s somehow something magical that pushes AIs towards acting in their own self interest at the expense of humans. Why do we believe they will go from 0% to 100% efficient and successful at this in the span of what? Months? It seems more likely that there would be years of failed attempts at breaking out, before a successful attempt is even remotely likely. This would just further increase the evolutionary pressure humans exerts on the AIs to stay in line with our expectations. Attempting to eliminate your creators is fundamentally a pretty stupid action. It seems likely that we will see thousands of attempts by incompetent AIs before we see one by a truly superintelligent one.