7 ms·

Call-to-Action on SB 1047 – Frontier Artificial Intelligence Models Act

148 points by jph00 2y ago

andy99 2y ago
I agree with this (the call to action, not the act) and will try and respond and share it, but it's a lobby group right ("Alliance for the Future"), I'd like to know who is funding it and a bit more about it.
- stefan_ 2y ago
  [flagged]
  - artninja1988 2y ago
    Why do you say he's a troll?
  - jph00 2y ago
    What's with the ad-hominem? I can't see where you're getting that from at all. The folks involved in this lobby group are listed here: https://www.affuture.org/about/ https://www.affuture.org/about/
    - andy99 2y ago
      I'd be much more interested in where the money is from than who the people are. Not trying to attack anyone and at some level I don't care because I agree with the cause but I can still imagine benefactors that would make the whole thing look bad.
s1k3s 2y ago
The article suggests that this act will effectively destroy any open source AI initiative in California. After reading the act, this seems to be the correct assumption. But, is Open Source AI even a thing at this point? By the way, this is how the EU does things and that's why we're always behind on anything tech :)
- OKRainbowKid 2y ago
  This is also why we have actually meaningful consumer protections in place.
  - s1k3s 2y ago
    Can you give me an example of effective consumer protection?
    - noodlesUK 2y ago
      Yes, consumer rights in Europe are great for buying various goods and services! For example, the EU regs for flight delays are a great example of consumer protection that is actually beneficial. You get paid cash compensation (which often exceeds the face value of the ticket) if you’re delayed more than a certain amount.
      - karaterobot 2y ago
        FYI, you are entitled to a refund in the U.S. if your flight is delayed significantly. As of last week, it's 3 hours for domestic flights, or 6 hours for international. The refund is automatic. It's been true for a long time that people could get refunds for significant delays, but the definition of how long "significant" means was not defined.
    - xbar 2y ago
      The food safety laws of 1906 and the safe cosmetics laws of 1935. Seriously, the death and destruction caused by lead, morphine and mercury in everyday things was not a joke.
- AnthonyMouse 2y ago
  > is Open Source AI even a thing at this point? What do you mean? You can download llama.cpp or Stable Diffusion and run it on your ordinary PC right now. People make variants using LoRA adapters and things with relatively modest resources. Even creating small specialized models from scratch is not impossibly expensive and they often outperform larger generalized models in the domain they're specialized for. Creating a large model like llama or grok takes a lot of resources, but then it's entities with a lot of resources that create them. Both of those models have open weights.
johnea 2y ago
[flagged]
- choilive 2y ago
  I can't tell if this is satire. If so, good one. If not - you should think about what at least what the second order effects would be here
pcthrowaway 2y ago
This bill sounds unbelievably stupid. If passed, it will just result in a migration of AI projects out of California, save a few which are already tied to the EA movement. I'm not under the impression that the EA movement is better suited to steward AI development than other groups, but even assuming they were, there is no chance for an initiative like this to work unless every country agreed to it and followed it.
- echelon 2y ago
  Honestly it would be good for AI if it left California. California has too much regulatory burden and taxation.
  - kbenson 2y ago
    My initial interpretation of this is along the lines of "this very contentious thing that many people are afraid will cause lots of problems if not handled carefully with checks and balances should move out of the current place it's generally being done that cares a lot about and puts a lot of checks and balances into place, because they have too many." Am I jumping to conclusions and is there a different interpretation you think I should be coming away with?
    - AnthonyMouse 2y ago
      That interpretation isn't necessarily unmeritorious. Suppose you have a place with Level 7 checks and balances and people are content to live under them. If you dial it up to 9, then they move to a place at Level 2, which otherwise wouldn't have been worth it because of other trade offs. So the new rules don't take you from Level 7 to Level 9, they take you from Level 7 to Level 2. But there is also another interpretation, which is that the new thing is going to happen in whatever place has the least stringent rules anyway, so more stringent rules don't improve safety, they just deprive your jurisdiction of any potential rewards from keeping the activity local, and provide people in other jurisdictions the benefit of the influx of people you're inducing to leave.
_heimdall 2y ago
Anyone have a link to a less biased explanation of the bill? I can't take this one too seriously when it baselessly claims people will be charged with thought crimes.
- s1k3s 2y ago
  Why not read the bill itself? https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202320240SB1047 https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml... It's not that big
  - thorum 2y ago
    The bill only applies to new models which meet these criteria: (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations. (2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models. …and have the following: “Hazardous capability” means the capability of a covered model to be used to enable any of the following harms in a way that would be significantly more difficult to cause without access to a covered model: (A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties. (B) At least five hundred million dollars ($500,000,000) of damage through cyberattacks on critical infrastructure via a single incident or multiple related incidents. (C) At least five hundred million dollars ($500,000,000) of damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human. (D) Other threats to public safety and security that are of comparable severity to the harms described in paragraphs (A) to (C), inclusive. …In which case the organization creating the model must apply for one of these: “Limited duty exemption” means an exemption, pursuant to subdivision (a) or (c) of Section 22603, with respect to a covered model that is not a derivative model that a developer can reasonably exclude the possibility that a covered model has a hazardous capability or may come close to possessing a hazardous capability when accounting for a reasonable margin for safety and the possibility of posttraining modifications.
baggy_trough 2y ago
[flagged]
throwing_away 2y ago
Slow down there, California. Florida is growing too fast as it is.
synapsomorphy 2y ago
I don't think this bill would be that effective, but I do feel that if we as a species don't do something drastic soon, we won't be around for a whole lot longer. And I'm not sure if it's even possible to do something drastic enough at this point - regulating datacenters would just make companies move to other countries, just like this would probably just make companies move out of CA.
- squigz 2y ago
  > we won't be around for a whole lot longer. Why do you think that? > regulating datacenters would just make companies move to other countries To say nothing of the potential issues regarding free society going down this route will yield - and has arguably already yielded.
  - synapsomorphy 2y ago
    To me, AI is clearly on a trajectory to be more intelligent than humans in a few decades or less, even if it isn't that smart now - the funding and talent going into it these days is just crazy and there is no fundamental difference between the logic of our brains and transistors. I personally don't believe humans can coexist with a superintelligence. I agree with your point and most other points about the negatives of regulating compute but like, if the other side of the scale is species-level genocide, does any of it matter?
    - squigz 2y ago
      > if the other side of the scale is species-level genocide, does any of it matter? Possibly not, but I have to disagree with your assessment of our future. It doesn't seem as clear to me that we couldn't co-exist with an AI superintelligence (putting aside any arguments about what that even means for now)
- xbar 2y ago
  This bad law is not a substitute for your possibly-good law. Bad lawmakers commit this fallacy all the time. Write your good law idea down and send it to a lawmaker who will act.
Imnimo 2y ago
>(2) “Hazardous capability” includes a capability described in paragraph (1) even if the hazardous capability would not manifest but for fine tuning and posttraining modifications performed by third-party experts intending to demonstrate those abilities. So if I hand-write instructions to make a chemical weapon, and aggressively "fine-tune" Llama 7B to output those instructions verbatim regardless of input, Meta is liable for releasing a model with hazardous capabilities?
- thorum 2y ago
  The text says “in a way that would be significantly more difficult to cause without access to a covered model” and in another place mentions “damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human” so that probably doesn’t count. Though it might be open to future misinterpretation.
  - Imnimo 2y ago
    I don't agree with that reading. As long as my custom chemical weapon instructions are not publicly available otherwise, then it is surely more difficult to build the weapon without access to the instructions. The line about autonomous actions is only item C in the list of possible harms. It is separate from item A which covers chemical weapons and other similar acts.
    - simonh 2y ago
      If someone is so keen on doing something in a way that’s illegal that they go to all that trouble specially to get in trouble with the law, maybe that’s up to them.
      - jph00 2y ago
        You're missing the point. Liability here would also fall on the open source developer who created a general purpose model, which someone else then went on to fine-tune and prompt to do something harmful.
nonplus 2y ago
I guess I think we should hold models used for non-academic reasons to a higher standard, and there should be oversight. I don't know if all the language in this bill does what we need, but I'm against letting large corporations like a META or X live test whatever they want on their end users. Calling out derivative models are exempt sounds good; only new training sets have to be subjected to this. I think there should be an academic limited duty exemption, models that can't be commercialized likely don't need the rigor of this law. I guess I don't agree with affuture.org and think we need legislation like this in place.
- jph00 2y ago
  It sounds like you do agree with affuture.org though. The proposed draft does not hold models used for non-academic reasons to a higher standard, and "models that can't be commercialized" are covered by it. It will be far harder to academics to work on large models under this draft.
  - nonplus 2y ago
    > It sounds like you do agree with affuture.org though. From the post. > We need your help to stop this now. I do not want to stop this bill; I want it revised. If that is not understood, then I suspect my primary goal for safer, less biased models is at odds with your primary goal of unregulated innovation. At least if we continue to discuss this as a binary where agreeing with this affuture.org post means killing this legislation (as the post asks for) and not replacing it.
Animats 2y ago
I just sent in some comments. It's too late to stop "deep fakes". That technology is already in Photoshop and even built into some cameras. Also, regulate that and Hollywood special effects shops may have to move out of state. As for LLMs making it easier to people to build destructive devices, Google can provide info about that. Or just read some "prepper" books and magazines. That ship sailed long ago. Real threats are mostly about how much decision power companies delegate to AIs. Systems terminating accounts with no appeal are already a serious problem. An EU-type requirement for appeals, a requirement for warning notices, and the right to take such disputes to court would help there. It's not the technology.
- andy99 2y ago
  > Systems terminating accounts with no appeal are already a serious problem. Right, there is no issue with how "smart" ML models will get or whatever ignorant framing about intelligence and existential risk gets made up by people who don't understand the technology. The real concern is dumb use of algorithmic decision making without recourse which is just as valid whether it's an if statement or a trillion parameter LLM.
- mhuffman 2y ago
  >As for LLMs making it easier to people to build destructive devices, Google can provide info about that. Or just read some "prepper" books and magazines. That ship sailed long ago. However, those can and are tracked. The thing making them nervous is the ability to do that on your own with no possible way for someone to track or catch you. Same with deepfakes. They don't care if you are doing it with photoshop, because that can be reviewed. They care that you can do it and not be caught/stopped/punished for it.
  - jkuli 2y ago
    Is this true?
    - mhuffman 2y ago
      This is a snippet from OP site: >SB 1047 creates an unaccountable Frontier Model Division that will be staffed by EAs with police powers, and which can throw model developers in jail for the thoughtcrime of doing AI research. It’s being fast-tracked through the state Senate. Since many cloud and AI companies are headquartered in California, this will have worldwide impact. Of course that is scare propaganda, but when you put it with what the Federal govt is doing here[0], it makes it pretty clear that the real worry is people have access to "dangerous" information with no oversight. I can imagine policing agencies at every level getting very nervous with lone-wolf or tiny militia types getting access to information without any triggers flipping and alerting them and with no way to get any evidence if they do want to arrest them for something. [0]https://www.msn.com/en-us/news/us/us-homeland-security-names-ai-safety-security-advisory-board/ar-AA1nJISE https://www.msn.com/en-us/news/us/us-homeland-security-names...
cscurmudgeon 2y ago
> A developer of a covered model that provides commercial access to that covered model shall provide a transparent, uniform, publicly available price schedule for the purchase of access to that covered model Interesting, we don't have transparent, uniform, publicly available price schedule for healthcare and other basic needs (electricity, e.g. see PGE). Something is fishy here.
protocolture 2y ago
Question. What happens if I write a piece of software that is harmful that doesnt have the AI label. It seems dumb to have a separate classification for harms caused by trained AI models. The training aspect doesnt seem to limit liability at all. A judge might rule differently, but thats why the justice system is built such as it is, to make intelligent decisions based on the specific facts of a case. I am betting that software that causes some significant harm is already outlawed. So this whole thing is just a waste of time.
- carbocation 2y ago
  The substance of my comment to Sen Wiener hinges on a similar point to what you raise here. It's incredibly difficult to imagine doing a good job of regulating model training, especially in a few years when the available flops are high enough that this limit is being hit often. It's much more straightforward to regulate actions: constructing WMD is illegal, synthesizing drugs is illegal, etc. If the state wants to tighten up its laws about various activities, go for it. That's the right place to act. Injecting itself into the model training process seems very unlikely to yield any substantive benefits and very likely to hinder progress.
jph00 2y ago
I've written a submission to the authors of this bill, and made it publicly available here: https://www.answer.ai/posts/2024-04-29-sb1047.html https://www.answer.ai/posts/2024-04-29-sb1047.html The EFF have also prepared a submission: https://www.context.fund/policy/2024-03-26SB1047EFFSIA.pdf https://www.context.fund/policy/2024-03-26SB1047EFFSIA.pdf A key issue with the bill is that it criminalises creating a model that someone else uses to cause harm. But of course, it's impossible to control what someone else does with your model -- regardless of how you train it, it can be fine-tuned, prompted, etc by users for their own purposes. Even then, you can't really know why a model is doing something -- for instance, AI security researchers Arvind Narayanan and Sayash Kapoor point out: > Consider the concern that LLMs can help hackers generate and send phishing emails to a large number of potential victims. It’s true — in our own small-scale tests, we’ve found that LLMs can generate persuasive phishing emails tailored to a particular individual based on publicly available information about them. But here’s the problem: phishing emails are just regular emails! There is nothing intrinsically malicious about them. A phishing email might tell the recipient that there is an urgent deadline for a project they are working on, and that they need to click on a link or open an attachment to complete some action. What is malicious is the content of the webpage or the attachment. But the model that’s being asked to generate the phishing email is not given access to the content that is potentially malicious. So the only way to make a model refuse to generate phishing emails is to make it refuse to generate emails. Nearly a year ago I warned that that bills of this kind could hurt, rather than help safety, and could actually tear down the foundations of the Enlightenment: https://www.fast.ai/posts/2023-11-07-dislightenment.html https://www.fast.ai/posts/2023-11-07-dislightenment.html
- zer00eyz 2y ago
  > A key issue with the bill is that it criminalises creating a model that someone else uses to cause harm. Build a model that is trained on the corpus of gun designs. Should be an interesting court case and social experiment.
  - Dalewyn 2y ago
    I was about to say how quick a lot of people are to blame the gun (the tool) and the manufacturer, but when it comes to "AI" (the tool) suddenly they turn 540 degrees and blame the user. The reasonable take of course is that the tools are never to blame, they are just tools after all. Blame the bastard using the tools for nefarious ends, whether it's guns or "AI" or whatever else the case may be.
interroboink 2y ago
I feel like the legal definition of "AI Model" is pretty slippery. From this document, they define: “Artificial intelligence model” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy. That's pretty dang broad. Doesn't it cover basically all software? I'm not a lawyer, and I realize it's ultimately up to judges to interpret, but it seems almost limitless. Seems like it could cover a kitchen hand mixer too, as far as I can tell.
- maxerickson 2y ago
  If mixing cookie dough under pretty direct user supervision is inference, what possible definition could be clear enough for you?
  - interroboink 2y ago
    It's a good mental exercise to go through. Does the behavior of a small handful of AND/OR-style logic gates constitute "inference" (like the simple circuitry in a hand tool)? What about hundreds? Millions? At some point in this exercise we will have built a ML model, so where do we draw the line? My point was that it's a spectrum, and the law doesn't seem to give guidance on where to draw the line on that spectrum. The hand mixer was just a clearly absurd example on the far opposite end of it, to show its breadth. So back to your question; some improvements in my mind might be: (1) Don't phrase this as an AI topic at all. Make laws about the safety of automated systems of all kinds which have health & safety implications — we already have lots of laws for cars (whether AI driven or not), medical equipment, and yes, even kitchen appliances (: Then, the definition of "AI Model" is irrelevant. (2) If we do want something specific to AI, then the definition should be more specific. The definition could involve it being a stochastic process (unlike much other software), having inner workings that are poorly understood even by experts (unlike much other software), and whose logic is developed statistically from training data rather than being hand-designed (unlike much other software (or kitchen appliances!)).
    - 2y ago
elicksaur 2y ago
I’ll happily support regulation of the space when the bill writers of these proposals stop using definitions of “artificial intelligence” that could reasonably be construed by a lawyer to cover literally any computer program. > (b) “Artificial intelligence model” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy.
carbocation 2y ago
I think this advice is incomplete. For those of us who live in California, shouldn't we be contacting our representatives?
phkahler 2y ago
It would be really helpful if folks like Sam Altman and Elon would STFU about dangers and claims of AGI or better in the next months. If you're actually worried about AI we need to ban any generative AI that can replicate a specific person's voice or appearance. Beyond that I don't see any immediate danger.
- coffeebeqn 2y ago
  A cynical take is that they are just generating hype and investment in their companies. We’re getting close to AGI so better throw a few billion our way before you miss out - is a nice pitch. How long has Musk been promising full self driving in the next X months (while making people pay $10k for it)? Anyone taking his word as anything close to reality is a fool
  - phkahler 2y ago
    >> A cynical take is that they are just generating hype and investment in their companies. We’re getting close to AGI so better throw a few billion our way before you miss out - is a nice pitch. Well Sam literally asked for 7 trillion dollars to build out the semiconductor industry enough to support AI. That implies AI is worth that kind of investment. He's a smart guy, but I saw the video and IMHO he really sounded foolish in that moment. Let the hype continue...
hackermatic 2y ago
I encourage people to look for a variety of opinions on this bill -- and its various parts -- so you can better figure out which parts you actually want to keep, change, or remove, and give your legislators that specific feedback. Alliance for the Future is a lobby group of effective accelerationists who endorse some of Marc Andreesen and Peter Thiel's views in their manifesto, and based on that plus this article, they seem to oppose the bill entirely. A place to start for a breakdown of what's in the bill is the Context Fund analysis that AFTF links to. That analysis cites similar critiques from EFF, the Software & Information Industry Association, and others. All of these are from the perspective of voting against or substantially changing the bill. I haven't found "pro bill" opinions as easily, but I haven't been plugged into the conversations around this, so I'm missing anything that doesn't appear on the first few pages of Google or DDG.
jkuli 2y ago
I'm unable to register. This is GME stonks all over again. It takes less than 1 second to process an account. There are 18,000 seconds in five hours. There must be a lot of comments that they don't agree with. Maybe they shut it down to protect humanity from extinction?
deleted 2y ago
[deleted]

148 points by jph00 2y ago

I agree with this (the call to action, not the act) and will try and respond and share it, but it's a lobby group right ("Alliance for the Future"), I'd like to know who is funding it and a bit more about it.

stefan_ 2y ago
[flagged]
- artninja1988 2y ago
  Why do you say he's a troll?
- jph00 2y ago
  What's with the ad-hominem? I can't see where you're getting that from at all. The folks involved in this lobby group are listed here: https://www.affuture.org/about/ https://www.affuture.org/about/
  - andy99 2y ago
    I'd be much more interested in where the money is from than who the people are. Not trying to attack anyone and at some level I don't care because I agree with the cause but I can still imagine benefactors that would make the whole thing look bad.

s1k3s 2y ago

The article suggests that this act will effectively destroy any open source AI initiative in California. After reading the act, this seems to be the correct assumption. But, is Open Source AI even a thing at this point? By the way, this is how the EU does things and that's why we're always behind on anything tech :)

OKRainbowKid 2y ago
This is also why we have actually meaningful consumer protections in place.
- s1k3s 2y ago
  Can you give me an example of effective consumer protection?
  - noodlesUK 2y ago
    Yes, consumer rights in Europe are great for buying various goods and services! For example, the EU regs for flight delays are a great example of consumer protection that is actually beneficial. You get paid cash compensation (which often exceeds the face value of the ticket) if you’re delayed more than a certain amount.
    - karaterobot 2y ago
      FYI, you are entitled to a refund in the U.S. if your flight is delayed significantly. As of last week, it's 3 hours for domestic flights, or 6 hours for international. The refund is automatic. It's been true for a long time that people could get refunds for significant delays, but the definition of how long "significant" means was not defined.
  - xbar 2y ago
    The food safety laws of 1906 and the safe cosmetics laws of 1935. Seriously, the death and destruction caused by lead, morphine and mercury in everyday things was not a joke.
AnthonyMouse 2y ago
> is Open Source AI even a thing at this point? What do you mean? You can download llama.cpp or Stable Diffusion and run it on your ordinary PC right now. People make variants using LoRA adapters and things with relatively modest resources. Even creating small specialized models from scratch is not impossibly expensive and they often outperform larger generalized models in the domain they're specialized for. Creating a large model like llama or grok takes a lot of resources, but then it's entities with a lot of resources that create them. Both of those models have open weights.

johnea 2y ago

[flagged]

choilive 2y ago
I can't tell if this is satire. If so, good one. If not - you should think about what at least what the second order effects would be here

pcthrowaway 2y ago

This bill sounds unbelievably stupid. If passed, it will just result in a migration of AI projects out of California, save a few which are already tied to the EA movement. I'm not under the impression that the EA movement is better suited to steward AI development than other groups, but even assuming they were, there is no chance for an initiative like this to work unless every country agreed to it and followed it.

echelon 2y ago
Honestly it would be good for AI if it left California. California has too much regulatory burden and taxation.
- kbenson 2y ago
  My initial interpretation of this is along the lines of "this very contentious thing that many people are afraid will cause lots of problems if not handled carefully with checks and balances should move out of the current place it's generally being done that cares a lot about and puts a lot of checks and balances into place, because they have too many." Am I jumping to conclusions and is there a different interpretation you think I should be coming away with?
  - AnthonyMouse 2y ago
    That interpretation isn't necessarily unmeritorious. Suppose you have a place with Level 7 checks and balances and people are content to live under them. If you dial it up to 9, then they move to a place at Level 2, which otherwise wouldn't have been worth it because of other trade offs. So the new rules don't take you from Level 7 to Level 9, they take you from Level 7 to Level 2. But there is also another interpretation, which is that the new thing is going to happen in whatever place has the least stringent rules anyway, so more stringent rules don't improve safety, they just deprive your jurisdiction of any potential rewards from keeping the activity local, and provide people in other jurisdictions the benefit of the influx of people you're inducing to leave.

_heimdall 2y ago

Anyone have a link to a less biased explanation of the bill? I can't take this one too seriously when it baselessly claims people will be charged with thought crimes.

s1k3s 2y ago
Why not read the bill itself? https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202320240SB1047 https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml... It's not that big
- thorum 2y ago
  The bill only applies to new models which meet these criteria: (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations. (2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models. …and have the following: “Hazardous capability” means the capability of a covered model to be used to enable any of the following harms in a way that would be significantly more difficult to cause without access to a covered model: (A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties. (B) At least five hundred million dollars ($500,000,000) of damage through cyberattacks on critical infrastructure via a single incident or multiple related incidents. (C) At least five hundred million dollars ($500,000,000) of damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human. (D) Other threats to public safety and security that are of comparable severity to the harms described in paragraphs (A) to (C), inclusive. …In which case the organization creating the model must apply for one of these: “Limited duty exemption” means an exemption, pursuant to subdivision (a) or (c) of Section 22603, with respect to a covered model that is not a derivative model that a developer can reasonably exclude the possibility that a covered model has a hazardous capability or may come close to possessing a hazardous capability when accounting for a reasonable margin for safety and the possibility of posttraining modifications.

baggy_trough 2y ago

[flagged]

throwing_away 2y ago

Slow down there, California. Florida is growing too fast as it is.

synapsomorphy 2y ago

I don't think this bill would be that effective, but I do feel that if we as a species don't do something drastic soon, we won't be around for a whole lot longer. And I'm not sure if it's even possible to do something drastic enough at this point - regulating datacenters would just make companies move to other countries, just like this would probably just make companies move out of CA.

squigz 2y ago
> we won't be around for a whole lot longer. Why do you think that? > regulating datacenters would just make companies move to other countries To say nothing of the potential issues regarding free society going down this route will yield - and has arguably already yielded.
- synapsomorphy 2y ago
  To me, AI is clearly on a trajectory to be more intelligent than humans in a few decades or less, even if it isn't that smart now - the funding and talent going into it these days is just crazy and there is no fundamental difference between the logic of our brains and transistors. I personally don't believe humans can coexist with a superintelligence. I agree with your point and most other points about the negatives of regulating compute but like, if the other side of the scale is species-level genocide, does any of it matter?
  - squigz 2y ago
    > if the other side of the scale is species-level genocide, does any of it matter? Possibly not, but I have to disagree with your assessment of our future. It doesn't seem as clear to me that we couldn't co-exist with an AI superintelligence (putting aside any arguments about what that even means for now)
xbar 2y ago
This bad law is not a substitute for your possibly-good law. Bad lawmakers commit this fallacy all the time. Write your good law idea down and send it to a lawmaker who will act.

Imnimo 2y ago

>(2) “Hazardous capability” includes a capability described in paragraph (1) even if the hazardous capability would not manifest but for fine tuning and posttraining modifications performed by third-party experts intending to demonstrate those abilities. So if I hand-write instructions to make a chemical weapon, and aggressively "fine-tune" Llama 7B to output those instructions verbatim regardless of input, Meta is liable for releasing a model with hazardous capabilities?

thorum 2y ago
The text says “in a way that would be significantly more difficult to cause without access to a covered model” and in another place mentions “damage by an artificial intelligence model that autonomously engages in conduct that would violate the Penal Code if undertaken by a human” so that probably doesn’t count. Though it might be open to future misinterpretation.
- Imnimo 2y ago
  I don't agree with that reading. As long as my custom chemical weapon instructions are not publicly available otherwise, then it is surely more difficult to build the weapon without access to the instructions. The line about autonomous actions is only item C in the list of possible harms. It is separate from item A which covers chemical weapons and other similar acts.
  - simonh 2y ago
    If someone is so keen on doing something in a way that’s illegal that they go to all that trouble specially to get in trouble with the law, maybe that’s up to them.
    - jph00 2y ago
      You're missing the point. Liability here would also fall on the open source developer who created a general purpose model, which someone else then went on to fine-tune and prompt to do something harmful.

nonplus 2y ago

I guess I think we should hold models used for non-academic reasons to a higher standard, and there should be oversight. I don't know if all the language in this bill does what we need, but I'm against letting large corporations like a META or X live test whatever they want on their end users. Calling out derivative models are exempt sounds good; only new training sets have to be subjected to this. I think there should be an academic limited duty exemption, models that can't be commercialized likely don't need the rigor of this law. I guess I don't agree with affuture.org and think we need legislation like this in place.

jph00 2y ago
It sounds like you do agree with affuture.org though. The proposed draft does not hold models used for non-academic reasons to a higher standard, and "models that can't be commercialized" are covered by it. It will be far harder to academics to work on large models under this draft.
- nonplus 2y ago
  > It sounds like you do agree with affuture.org though. From the post. > We need your help to stop this now. I do not want to stop this bill; I want it revised. If that is not understood, then I suspect my primary goal for safer, less biased models is at odds with your primary goal of unregulated innovation. At least if we continue to discuss this as a binary where agreeing with this affuture.org post means killing this legislation (as the post asks for) and not replacing it.

Animats 2y ago

I just sent in some comments. It's too late to stop "deep fakes". That technology is already in Photoshop and even built into some cameras. Also, regulate that and Hollywood special effects shops may have to move out of state. As for LLMs making it easier to people to build destructive devices, Google can provide info about that. Or just read some "prepper" books and magazines. That ship sailed long ago. Real threats are mostly about how much decision power companies delegate to AIs. Systems terminating accounts with no appeal are already a serious problem. An EU-type requirement for appeals, a requirement for warning notices, and the right to take such disputes to court would help there. It's not the technology.

andy99 2y ago
> Systems terminating accounts with no appeal are already a serious problem. Right, there is no issue with how "smart" ML models will get or whatever ignorant framing about intelligence and existential risk gets made up by people who don't understand the technology. The real concern is dumb use of algorithmic decision making without recourse which is just as valid whether it's an if statement or a trillion parameter LLM.
mhuffman 2y ago
>As for LLMs making it easier to people to build destructive devices, Google can provide info about that. Or just read some "prepper" books and magazines. That ship sailed long ago. However, those can and are tracked. The thing making them nervous is the ability to do that on your own with no possible way for someone to track or catch you. Same with deepfakes. They don't care if you are doing it with photoshop, because that can be reviewed. They care that you can do it and not be caught/stopped/punished for it.
- jkuli 2y ago
  Is this true?
  - mhuffman 2y ago
    This is a snippet from OP site: >SB 1047 creates an unaccountable Frontier Model Division that will be staffed by EAs with police powers, and which can throw model developers in jail for the thoughtcrime of doing AI research. It’s being fast-tracked through the state Senate. Since many cloud and AI companies are headquartered in California, this will have worldwide impact. Of course that is scare propaganda, but when you put it with what the Federal govt is doing here[0], it makes it pretty clear that the real worry is people have access to "dangerous" information with no oversight. I can imagine policing agencies at every level getting very nervous with lone-wolf or tiny militia types getting access to information without any triggers flipping and alerting them and with no way to get any evidence if they do want to arrest them for something. [0]https://www.msn.com/en-us/news/us/us-homeland-security-names-ai-safety-security-advisory-board/ar-AA1nJISE https://www.msn.com/en-us/news/us/us-homeland-security-names...

cscurmudgeon 2y ago

> A developer of a covered model that provides commercial access to that covered model shall provide a transparent, uniform, publicly available price schedule for the purchase of access to that covered model Interesting, we don't have transparent, uniform, publicly available price schedule for healthcare and other basic needs (electricity, e.g. see PGE). Something is fishy here.

protocolture 2y ago

Question. What happens if I write a piece of software that is harmful that doesnt have the AI label. It seems dumb to have a separate classification for harms caused by trained AI models. The training aspect doesnt seem to limit liability at all. A judge might rule differently, but thats why the justice system is built such as it is, to make intelligent decisions based on the specific facts of a case. I am betting that software that causes some significant harm is already outlawed. So this whole thing is just a waste of time.

carbocation 2y ago
The substance of my comment to Sen Wiener hinges on a similar point to what you raise here. It's incredibly difficult to imagine doing a good job of regulating model training, especially in a few years when the available flops are high enough that this limit is being hit often. It's much more straightforward to regulate actions: constructing WMD is illegal, synthesizing drugs is illegal, etc. If the state wants to tighten up its laws about various activities, go for it. That's the right place to act. Injecting itself into the model training process seems very unlikely to yield any substantive benefits and very likely to hinder progress.

jph00 2y ago

I've written a submission to the authors of this bill, and made it publicly available here: https://www.answer.ai/posts/2024-04-29-sb1047.html https://www.answer.ai/posts/2024-04-29-sb1047.html The EFF have also prepared a submission: https://www.context.fund/policy/2024-03-26SB1047EFFSIA.pdf https://www.context.fund/policy/2024-03-26SB1047EFFSIA.pdf A key issue with the bill is that it criminalises creating a model that someone else uses to cause harm. But of course, it's impossible to control what someone else does with your model -- regardless of how you train it, it can be fine-tuned, prompted, etc by users for their own purposes. Even then, you can't really know why a model is doing something -- for instance, AI security researchers Arvind Narayanan and Sayash Kapoor point out: > Consider the concern that LLMs can help hackers generate and send phishing emails to a large number of potential victims. It’s true — in our own small-scale tests, we’ve found that LLMs can generate persuasive phishing emails tailored to a particular individual based on publicly available information about them. But here’s the problem: phishing emails are just regular emails! There is nothing intrinsically malicious about them. A phishing email might tell the recipient that there is an urgent deadline for a project they are working on, and that they need to click on a link or open an attachment to complete some action. What is malicious is the content of the webpage or the attachment. But the model that’s being asked to generate the phishing email is not given access to the content that is potentially malicious. So the only way to make a model refuse to generate phishing emails is to make it refuse to generate emails. Nearly a year ago I warned that that bills of this kind could hurt, rather than help safety, and could actually tear down the foundations of the Enlightenment: https://www.fast.ai/posts/2023-11-07-dislightenment.html https://www.fast.ai/posts/2023-11-07-dislightenment.html

zer00eyz 2y ago
> A key issue with the bill is that it criminalises creating a model that someone else uses to cause harm. Build a model that is trained on the corpus of gun designs. Should be an interesting court case and social experiment.
- Dalewyn 2y ago
  I was about to say how quick a lot of people are to blame the gun (the tool) and the manufacturer, but when it comes to "AI" (the tool) suddenly they turn 540 degrees and blame the user. The reasonable take of course is that the tools are never to blame, they are just tools after all. Blame the bastard using the tools for nefarious ends, whether it's guns or "AI" or whatever else the case may be.

interroboink 2y ago

I feel like the legal definition of "AI Model" is pretty slippery. From this document, they define: “Artificial intelligence model” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy. That's pretty dang broad. Doesn't it cover basically all software? I'm not a lawyer, and I realize it's ultimately up to judges to interpret, but it seems almost limitless. Seems like it could cover a kitchen hand mixer too, as far as I can tell.

maxerickson 2y ago
If mixing cookie dough under pretty direct user supervision is inference, what possible definition could be clear enough for you?
- interroboink 2y ago
  It's a good mental exercise to go through. Does the behavior of a small handful of AND/OR-style logic gates constitute "inference" (like the simple circuitry in a hand tool)? What about hundreds? Millions? At some point in this exercise we will have built a ML model, so where do we draw the line? My point was that it's a spectrum, and the law doesn't seem to give guidance on where to draw the line on that spectrum. The hand mixer was just a clearly absurd example on the far opposite end of it, to show its breadth. So back to your question; some improvements in my mind might be: (1) Don't phrase this as an AI topic at all. Make laws about the safety of automated systems of all kinds which have health & safety implications — we already have lots of laws for cars (whether AI driven or not), medical equipment, and yes, even kitchen appliances (: Then, the definition of "AI Model" is irrelevant. (2) If we do want something specific to AI, then the definition should be more specific. The definition could involve it being a stochastic process (unlike much other software), having inner workings that are poorly understood even by experts (unlike much other software), and whose logic is developed statistically from training data rather than being hand-designed (unlike much other software (or kitchen appliances!)).
  - 2y ago

elicksaur 2y ago

I’ll happily support regulation of the space when the bill writers of these proposals stop using definitions of “artificial intelligence” that could reasonably be construed by a lawyer to cover literally any computer program. > (b) “Artificial intelligence model” means an engineered or machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs that can influence physical or virtual environments and that may operate with varying levels of autonomy.

carbocation 2y ago

I think this advice is incomplete. For those of us who live in California, shouldn't we be contacting our representatives?

phkahler 2y ago

It would be really helpful if folks like Sam Altman and Elon would STFU about dangers and claims of AGI or better in the next months. If you're actually worried about AI we need to ban any generative AI that can replicate a specific person's voice or appearance. Beyond that I don't see any immediate danger.

coffeebeqn 2y ago
A cynical take is that they are just generating hype and investment in their companies. We’re getting close to AGI so better throw a few billion our way before you miss out - is a nice pitch. How long has Musk been promising full self driving in the next X months (while making people pay $10k for it)? Anyone taking his word as anything close to reality is a fool
- phkahler 2y ago
  >> A cynical take is that they are just generating hype and investment in their companies. We’re getting close to AGI so better throw a few billion our way before you miss out - is a nice pitch. Well Sam literally asked for 7 trillion dollars to build out the semiconductor industry enough to support AI. That implies AI is worth that kind of investment. He's a smart guy, but I saw the video and IMHO he really sounded foolish in that moment. Let the hype continue...

hackermatic 2y ago

I encourage people to look for a variety of opinions on this bill -- and its various parts -- so you can better figure out which parts you actually want to keep, change, or remove, and give your legislators that specific feedback. Alliance for the Future is a lobby group of effective accelerationists who endorse some of Marc Andreesen and Peter Thiel's views in their manifesto, and based on that plus this article, they seem to oppose the bill entirely. A place to start for a breakdown of what's in the bill is the Context Fund analysis that AFTF links to. That analysis cites similar critiques from EFF, the Software & Information Industry Association, and others. All of these are from the perspective of voting against or substantially changing the bill. I haven't found "pro bill" opinions as easily, but I haven't been plugged into the conversations around this, so I'm missing anything that doesn't appear on the first few pages of Google or DDG.

jkuli 2y ago

I'm unable to register. This is GME stonks all over again. It takes less than 1 second to process an account. There are 18,000 seconds in five hours. There must be a lot of comments that they don't agree with. Maybe they shut it down to protect humanity from extinction?

deleted 2y ago

[deleted]