As a thought experiment, I think it’s useful to try steel manning this bill, and assume for a minute that the potential risk of a sufficiently advanced and “uns

As a thought experiment, I think it’s useful to try steel manning this bill, and assume for a minute that the potential risk of a sufficiently advanced and “unsafe” model is real and severe. It seems reasonable to regulate specific aspects of safety, e.g. it shouldn’t be possible to derive the formulas for chemical or nuclear weapons. If an AI company cannot be certain of this, it seems reasonable for the government to impose restrictions on the handling of such models. Where things get murky is how to define risk for things that are less obvious. There is deeply embedded in the tech community an ethos and belief that core tech is neutral, the potential for harmful use is an acceptable reality. I have lived and breathed this for 20 years. But information is not neutral, and the output of these models is derived from a corpus of information that is anything but neutral. A subset of this information is so incredibly harmful that the risk of its inclusion is high enough to warrant significant effort in ensuring safety. Again, assuming a sufficiently advanced model, might it be reasonable that the onus is on the creator of the model to understand what it can do and certify what it cannot do? Many have commented that the non-regulation of the early Internet was necessary and good, and that the same should be true for AI. But the eras are very unlike each other. The entire technology industry was still forming, and no one knew what would come of it. What grew was incredible. But with it came the data brokers, the worst aspects of social media, etc. It’s against this backdrop of an increasingly enshittefied landscape that regulators are looking at this potentially seismic shift in computing and not feeling willing to let the companies that cooked up the current mess go headlong into something potentially far more fantastical (for good or bad) without checks and balances. I still think the bill is far too vague and just drives AI development out of CA. But I can understand where it’s coming from, and I wouldn’t be surprised to see a push for this nationally, with similarly vague specifications because no one knows how to quantify AI risk. Edit: To be clear, I'm asking questions and trying to explore the legislator's angle here. I'm not saying I think this kind of legislation will be effective or that this is a good bill.