3 ms·
Who is he even arguing against, Hari Seldon? It's common for people concerned about an impending AI apocalypse to use this sort of "longtermist" argument. For
by lacker 5y ago
Who is he even arguing against, Hari Seldon?
It's common for people concerned about an impending AI apocalypse to use this sort of "longtermist" argument. For example this article:
https://oxfordpoliticalreview.com/2019/08/25/is-ai-safety-rather-speculative-long-termism/ https://oxfordpoliticalreview.com/2019/08/25/is-ai-safety-ra...
If we reduce existential risk by mere one-millionth of one percentage point, it will be worth more than 100 times the value of saving a million human lives. The expected value of any other good actions – like helping people here and now – will be trivial compared to even the slightest reduction in existential risk.
Similarly, the recent book The Precipice argued that the most important risk for humanity is AI risk because the badness of AI destroying all humanity is much worse than, say, climate change killing off 99% of humans.
- jacobr1 5y agoThe other side of the argument though, is the "reduce existential risk" part. There probably isn't anything meaningful I could do, to reduce the risk of some future war against aliens. But with AI, the presumption is that the risk is near and that AI research going on right now could meaningfully impact future AGI designs. AI research, say, 80 years ago wouldn't have the same tradeoff. I'm not sure I buy the assumptions, but the argument does require an assumption of near-term influence (albeit it allows that that influence to be very small).
- lacker 5y agoThere probably isn't anything meaningful I could do, to reduce the risk of some future war against aliens. Not your primary point, but I think we can reduce the risk of losing some future war against aliens by discovering the aliens sooner. The Aztecs would have been a lot better off if they had realized Cortés was coming a decade earlier....
- namlem 5y agoExcept those million human lives today would go on to produce trillions of human lives and untold amounts economic growth over the extremely long term.