6 ms·

"The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are d

by jprete 2y ago

"The Democrat decided to reject the measure because it applies only to the biggest and most expensive AI models and doesn’t take into account whether they are deployed in high-risk situations, he said in his veto message." That doesn't mean you're wrong, but it's not what Newsom signaled.

mhuffman 2y ago
>and doesn’t take into account whether they are deployed in high-risk situations Am I out of the loop here? What "high-risk" situations do they have in mind for LLM's?
- jeffbee 2y ago
  Imagine the only thing you know about AI came from the opening voiceover of Terminator 2 and you are a state legislator. Now you understand the origin of this bill perfectly.
- giantg2 2y ago
  My guess is anything involving direct human safety - medicine, defense, police... but who knows.
- SonOfLilit 2y ago
  It's not about current LLMs, it's about future, much more advanced models, that are capable of serious hacking or other mass-casualty-causing activities. o-1 and AlphaProof are proofs of concept for agentic models. Imagine them as GPT-1. The GPT-4 equivalent might be a scary technology to let roam the internet. It would have no effect on current models.
  - tbrownaw 2y ago
    It looks like it would cover an ordinary chatbot than can answer "how do I $THING" questions, where $THING is both very bad and is also beyond what a normal person could dig up with a search engine. It's not based on any assumptions about the future models having any capabilities beyond providing information to a user.
    - whimsicalism 2y ago
      everyone in the safety space has realized that it is much easier to get legislators/the public to care if you say that it will be “bad actors using the AI for mass damage” as opposed to “AI does damage on its own” which triggers people’s “that’s sci-fi and i’m ignoring it” reflex.
    - SonOfLilit 2y ago
      Things you could dig up with a search engine are explicitly not covered, see my other comment quoting the bill (ctrl+f critical harm).
- tmpz22 2y ago
  Medical and legal industries are both trying to apply AI to their administrative practices. It’s absolutely awful but they’re so horny for profits they’re trying anyways.
- tbrownaw 2y ago
  That concept does not appear to be part of the bill, and was only mentioned in the quote from the governor. Presumably someone somewhere has a variety of proposed definitions, but I don't see any mention of any particular ones.
- edm0nd 2y ago
  Health insurance companies using it to approve/deny claims. The large ones are processing millions of claims a day.
JoshTriplett 2y ago
Only applying to the biggest models is the point; the biggest models are the inherently high-risk ones. The larger they get, the more that running them at all is the "high-risk situation". Passing this would not have been a complete solution, but it would have been a step in the right direction. This is a huge disappointment.
- jpk 2y ago
  > running them at all is the "high-risk situation" What is the actual, concrete concern here? That a model "breaks out", or something? The risk with AI is not in just running models, the risk is becoming overconfident in them, and then putting them in charge of real-world stuff in a way that allows them to do harm. Hooking a model up to an effector capable of harm is a deliberate act requiring assurance that it doesn't harm -- and if we should regulate anything, it's that. Without that, inference is just making datacenters warm. It seems shortsighted to set an arbitrary limit on model size when you can recklessly hook up a smaller, shittier model to something safety-critical, and cause all the havoc you want.
  - pkage 2y ago
    There is no concrete concern past "models that can simulate thinking are scary." The risk has always been connecting models to systems which are safety critical, but for some reason the discourse around this issue has been more influenced by Terminator than OSHA. As a researcher in the field, I believe there's no risk beyond overconfident automation---and we already have analogous legislation for automations, for example in what criteria are allowable and not allowable when deciding whether an individual is eligible for a loan.
    - KoolKat23 2y ago
      Well it's a mix of concerns, the models are general purpose, there are plenty of areas regulation does not exist or is being bypassed. Can't access a prohibited chemical, no need to worry the model can tell you how to synthesize it from other household chemicals etc.
    - 2y ago
comp_throw7 2y ago
He's dissembling. He vetoed the bill because VCs decided to rally the flag; if the bill had covered more models he'd have been more likely to veto it, not less. It's been vaguely mindblowing to watch various tech people & VCs argue that use-based restrictions would be better than this, when use-based restrictions are vastly more intrusive, economically inefficient, and subject to regulatory capture than what was proposed here.
jart 2y ago
If you read Gavin Newsom's statement, it sounds like he agrees with Terrance Tao's position, which is that the government should regulate the people deploying AI rather than the people inventing AI. That's why he thinks it should be stricter. For example, you wouldn't want to lead people to believe that AI in health care decisions is OK so long as it's smaller than 10^26 flops. Read his full actual statement here: https://www.gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Veto-Message.pdf https://www.gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Ve...
- Terr_ 2y ago
  > the government should regulate the people deploying AI rather than the people inventing AI Yeah, there's no point having system that is made the most scrupulous of standards and then someone else deploys it in an evil way. (Which in some cases can be done simply by choosing to do the opposite of whatever a good model recommends.)

by jprete 2y ago

mhuffman 2y ago

>and doesn’t take into account whether they are deployed in high-risk situations Am I out of the loop here? What "high-risk" situations do they have in mind for LLM's?

jeffbee 2y ago
Imagine the only thing you know about AI came from the opening voiceover of Terminator 2 and you are a state legislator. Now you understand the origin of this bill perfectly.
giantg2 2y ago
My guess is anything involving direct human safety - medicine, defense, police... but who knows.
SonOfLilit 2y ago
It's not about current LLMs, it's about future, much more advanced models, that are capable of serious hacking or other mass-casualty-causing activities. o-1 and AlphaProof are proofs of concept for agentic models. Imagine them as GPT-1. The GPT-4 equivalent might be a scary technology to let roam the internet. It would have no effect on current models.
- tbrownaw 2y ago
  It looks like it would cover an ordinary chatbot than can answer "how do I $THING" questions, where $THING is both very bad and is also beyond what a normal person could dig up with a search engine. It's not based on any assumptions about the future models having any capabilities beyond providing information to a user.
  - whimsicalism 2y ago
    everyone in the safety space has realized that it is much easier to get legislators/the public to care if you say that it will be “bad actors using the AI for mass damage” as opposed to “AI does damage on its own” which triggers people’s “that’s sci-fi and i’m ignoring it” reflex.
  - SonOfLilit 2y ago
    Things you could dig up with a search engine are explicitly not covered, see my other comment quoting the bill (ctrl+f critical harm).
tmpz22 2y ago
Medical and legal industries are both trying to apply AI to their administrative practices. It’s absolutely awful but they’re so horny for profits they’re trying anyways.
tbrownaw 2y ago
That concept does not appear to be part of the bill, and was only mentioned in the quote from the governor. Presumably someone somewhere has a variety of proposed definitions, but I don't see any mention of any particular ones.
edm0nd 2y ago
Health insurance companies using it to approve/deny claims. The large ones are processing millions of claims a day.

JoshTriplett 2y ago

Only applying to the biggest models is the point; the biggest models are the inherently high-risk ones. The larger they get, the more that running them at all is the "high-risk situation". Passing this would not have been a complete solution, but it would have been a step in the right direction. This is a huge disappointment.

jpk 2y ago
> running them at all is the "high-risk situation" What is the actual, concrete concern here? That a model "breaks out", or something? The risk with AI is not in just running models, the risk is becoming overconfident in them, and then putting them in charge of real-world stuff in a way that allows them to do harm. Hooking a model up to an effector capable of harm is a deliberate act requiring assurance that it doesn't harm -- and if we should regulate anything, it's that. Without that, inference is just making datacenters warm. It seems shortsighted to set an arbitrary limit on model size when you can recklessly hook up a smaller, shittier model to something safety-critical, and cause all the havoc you want.
- pkage 2y ago
  There is no concrete concern past "models that can simulate thinking are scary." The risk has always been connecting models to systems which are safety critical, but for some reason the discourse around this issue has been more influenced by Terminator than OSHA. As a researcher in the field, I believe there's no risk beyond overconfident automation---and we already have analogous legislation for automations, for example in what criteria are allowable and not allowable when deciding whether an individual is eligible for a loan.
  - KoolKat23 2y ago
    Well it's a mix of concerns, the models are general purpose, there are plenty of areas regulation does not exist or is being bypassed. Can't access a prohibited chemical, no need to worry the model can tell you how to synthesize it from other household chemicals etc.
  - 2y ago

comp_throw7 2y ago

He's dissembling. He vetoed the bill because VCs decided to rally the flag; if the bill had covered more models he'd have been more likely to veto it, not less. It's been vaguely mindblowing to watch various tech people & VCs argue that use-based restrictions would be better than this, when use-based restrictions are vastly more intrusive, economically inefficient, and subject to regulatory capture than what was proposed here.

jart 2y ago

If you read Gavin Newsom's statement, it sounds like he agrees with Terrance Tao's position, which is that the government should regulate the people deploying AI rather than the people inventing AI. That's why he thinks it should be stricter. For example, you wouldn't want to lead people to believe that AI in health care decisions is OK so long as it's smaller than 10^26 flops. Read his full actual statement here: https://www.gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Veto-Message.pdf https://www.gov.ca.gov/wp-content/uploads/2024/09/SB-1047-Ve...

Terr_ 2y ago
> the government should regulate the people deploying AI rather than the people inventing AI Yeah, there's no point having system that is made the most scrupulous of standards and then someone else deploys it in an evil way. (Which in some cases can be done simply by choosing to do the opposite of whatever a good model recommends.)