The concern is that near future systems will be much more capable than current systems, and by the time they arrive, it may be too late to react. Many people fr

The concern is that near future systems will be much more capable than current systems, and by the time they arrive, it may be too late to react. Many people from the large frontier AI companies believe that world-changing AGI is 5 years or less away; see Situational Awareness by Aschbrenner, for example. There's also a parallel concern that AIs could make terrorism easier[1]. Yoshua Bengio has written in detail about his views on AI safety recently[2][3][4]. He seems to put less weight on human level AI being very soon, but says superhuman intelligence is plausible in 5-20 years and says: > Faced with that uncertainty, the magnitude of the risk of catastrophes or worse, extinction, and the fact that we did not anticipate the rapid progress in AI capabilities of recent years, agnostic prudence seems to me to be a much wiser path. Hinton also has a detailed lecture he's been giving recently about the loss of control risk. In general, proponents see this as narrowly tailored bill to somewhat address the worst case worries about loss of control and misuse. [1] https://www.theregister.com/2023/07/28/ai_senate_bioweapon/ https://www.theregister.com/2023/07/28/ai_senate_bioweapon/ [2] https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/ [3] https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-risks/ https://yoshuabengio.org/2023/06/24/faq-on-catastrophic-ai-r... [4] https://yoshuabengio.org/2024/07/09/reasoning-through-arguments-against-taking-ai-safety-seriously/ https://yoshuabengio.org/2024/07/09/reasoning-through-argume...