Generalized artificial intelligence (AGI) comes with very real x-risks[1] (existential risks) and s-risks[2] (suffering risks). An expert survey of 738 researc

Generalized artificial intelligence (AGI) comes with very real x-risks[1] (existential risks) and s-risks[2] (suffering risks). An expert survey of 738 researchers who published in NeurIPS and ICML was done last year[3]. Their median estimation that AI will have an “extremely bad” long term outcome is 5%, and 48% of the researchers estimate the probability to be at least 10%. This is worryingly high considering the absolutely catastrophic consequences of the scenarios. A minority of very vocal AI researchers (e.g. Yann LeCun) dismiss these risks entirely and claim that people read too much science fiction. But when you listen to their interviews it's very clear that they have no idea what they are talking about and never actually read any scientific literature on the subject. The study of AI risks is a serious area of academic research that is worked on by labs from Stanford[4], Berkeley[5], Carnegie Melon University[6], Oxford[7], Cambridge[8], and many MANY other universities[9]. Not people who read too much science fiction. ———— [1] https://arxiv.org/pdf/2206.05862.pdf https://arxiv.org/pdf/2206.05862.pdf [2] https://longtermrisk.org/files/Sotala-Gloor-Superintelligent-AI-and-Suffering-Risks.pdf#page=8&search=%22superintelligence%22 https://longtermrisk.org/files/Sotala-Gloor-Superintelligent... [3] https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/ [4] https://web.stanford.edu/~chadj/existentialrisk.pdf https://web.stanford.edu/~chadj/existentialrisk.pdf [5] https://humancompatible.ai/about/ https://humancompatible.ai/about/ [6] https://www.cs.cmu.edu/~focal/ https://www.cs.cmu.edu/~focal/ [7] https://www.fhi.ox.ac.uk/research/research-areas/#aisafety_tab https://www.fhi.ox.ac.uk/research/research-areas/#aisafety_t... [8] https://www.camxrisk.org/ https://www.camxrisk.org/ [9] https://futureoflife.org/about-us/our-people/ai-existential-safety-community/ https://futureoflife.org/about-us/our-people/ai-existential-...