3 ms·

That's a false dilemma. It would be wise to defend against every existential threat in proportion to its likelihood. The threat of nuclear war is well known, an

by rictic 11y ago

That's a false dilemma. It would be wise to defend against every existential threat in proportion to its likelihood. The threat of nuclear war is well known, and significant resources are spent on reducing its likelihood. Until very recently the same could not be said for the threat of unfriendly AI. We can do both.

kansface 11y ago
> That's a false dilemma. It would be wise to defend against every existential threat in proportion to its likelihood. Lives saved versus dollars spent is a better metric. Having said that, the Singularity crowd tends to lump in all potential future lives into the saved category. The question of allocating resources to avoid existential threats very quickly turns into a Pascal's mugging in favor of AI safety (think of the trillions of future children). > The threat of nuclear war is well known ... until very recently the same could not be said for the threat of unfriendly AI. Enough nuclear weapons exist to kill all humans forever. Strong AI does not exist. Expert estimates range from decades to over a century. At the same time, the people making the most noise seem come from the side of avoiding existential threats and not AI research. I am (highly) skeptical they will produce anything of value. Let me rephrase. Would you expect any productive work in 1840 by philosophers concerning potential dooms day weapons yet to be created by physicists?
- vilhelm_s 11y ago
  There doesn't exist anywhere near enough nuclear weapons to kill all humans. Even in the 1980s, people were predicting between 20 and 100 million immediate American deaths from a full-scale Soviet attack, and since then there are much fewer nuclear weapons. See e.g. the discussion in this comment thread: https://slatestarcodex.com/2015/10/31/ot32-when-hell-is-full-the-thread-will-walk-the-earth/#comment-255524 https://slatestarcodex.com/2015/10/31/ot32-when-hell-is-full... It has been suggested that humans recovered from a population bottleneck of less than 30,000 individuals (the Toba catastrophy). A nuclear war would leave billions alive.
JoshTriplett 11y ago
On the other hand, nuclear research could solve our energy needs for the foreseeable future. And AI could protect humanity against almost every possible existential risk. Let's hope that the increased notice taken of AI, and fear thereof, doesn't suddenly shut down a major source of hope for the future.

by rictic 11y ago

kansface 11y ago

> That's a false dilemma. It would be wise to defend against every existential threat in proportion to its likelihood. Lives saved versus dollars spent is a better metric. Having said that, the Singularity crowd tends to lump in all potential future lives into the saved category. The question of allocating resources to avoid existential threats very quickly turns into a Pascal's mugging in favor of AI safety (think of the trillions of future children). > The threat of nuclear war is well known ... until very recently the same could not be said for the threat of unfriendly AI. Enough nuclear weapons exist to kill all humans forever. Strong AI does not exist. Expert estimates range from decades to over a century. At the same time, the people making the most noise seem come from the side of avoiding existential threats and not AI research. I am (highly) skeptical they will produce anything of value. Let me rephrase. Would you expect any productive work in 1840 by philosophers concerning potential dooms day weapons yet to be created by physicists?

vilhelm_s 11y ago
There doesn't exist anywhere near enough nuclear weapons to kill all humans. Even in the 1980s, people were predicting between 20 and 100 million immediate American deaths from a full-scale Soviet attack, and since then there are much fewer nuclear weapons. See e.g. the discussion in this comment thread: https://slatestarcodex.com/2015/10/31/ot32-when-hell-is-full-the-thread-will-walk-the-earth/#comment-255524 https://slatestarcodex.com/2015/10/31/ot32-when-hell-is-full... It has been suggested that humans recovered from a population bottleneck of less than 30,000 individuals (the Toba catastrophy). A nuclear war would leave billions alive.

JoshTriplett 11y ago

On the other hand, nuclear research could solve our energy needs for the foreseeable future. And AI could protect humanity against almost every possible existential risk. Let's hope that the increased notice taken of AI, and fear thereof, doesn't suddenly shut down a major source of hope for the future.