They do have particular insight into the trajectory of capabilities; less insight into how the newfound capabilities will interact with the world; and no partic

They do have particular insight into the trajectory of capabilities; less insight into how the newfound capabilities will interact with the world; and no particular insight into values. The usual failure mode with experts is that they overstate how important their field is and demand money to fund further progress in their work. It's at least slightly interesting that this particular group of experts aren't self-aggrandizing in an attempt to make more money and get to develop their research interests further. Instead, they're arguing that their research needs to be frozen and severely regulated. In my opinion, the existential risk of AI is very real. The issue is that policymakers lack any real understanding of the issue, and most regulations will focus on mostly benign things (what if end users do something naughty with it?) and not the central existential risk (ew that's nerdy and alarmist). You can see that dynamic with Newsom. We'll end up with the worst of all worlds, where regulations prevent valuable small-scale use cases that improve our lives, but do nothing at all to prevent existential risk. Maybe even making the existential risk worse, though that's more speculative.