10 ms·

SB-1047 will stifle open-source AI and decrease safety

225 points by kmdupree 2y ago

iptq 2y ago
This entire article feels like it was written by ChatGPT. For instance, it continuously makes vague claims about the value of open source without even citing the original bill.
- jsheard 2y ago
  The eagerness to summarize points in bulleted lists is a ChatGPT-ism as well.
  - exe34 2y ago
    Hey now, I like bullet points myself, as I'm easily distracted otherwise.
- shkkmo 2y ago
  This is going to become one the the site no-nos. What does this assertion add to the discussion? Is there any argument that serves as refutation?
  - sensanaty 2y ago
    Why would I want to read some AI-generated slop? It's no different to spam, other than techbros being okay shoving it everywhere and anywhere it doesn't belong.
  - pjc50 2y ago
    If an article is AI, it's essentially spam, because there could be a thousand more similar written.
  - airstrike 2y ago
    Raising awareness so others too can flag TFA
    - shkkmo 2y ago
      The quality of the article is what should drive voting and flagging. Your personal opinion as to what level of AI tooling was used in crafting the content is a low value signal and posting that speculation without meaningfully engaging in the content is as low value as commenting to say "this article sucks". Personally, I think this would be a very strange place to find pute AI genetated content. This is a personal statement that was submitted to the state, and posted under a real name under a site the poster has a professional association with. I think that any "strangeness" in formating a wording comes from the role this text serves, as a public comment intended to affect policy.
narrator 2y ago
The politicians are gunning extremely hard for open source AI. It's crazy. "Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose a mortal threat to open societies.” https://www.wired.com/story/mortal-danger-chinas-push-into-ai/ https://www.wired.com/story/mortal-danger-chinas-push-into-a... Literally everyone out there who pursues global influence is just frothing at the mouth over AI. This is seriously tempting me to buy the 512gb Mac Studio when it comes out so I can run the big llama3 model, which will probably be banned any day now.
- Cthulhu_ 2y ago
  > The politicians are gunning extremely hard for open source AI. It's crazy. I hope someone has / can do some investigative journalism to check out their links with commercial / closed source AI; I can imagine the investors and those that benefit from companies like "Open"AI have close links with politicians. There's probably no direct links, they've become really good at obscuring those and plausible deniability.
  - kranke155 2y ago
    You'll hear about it in 10 years in an article about "why did open source Ai die? ... once blossoming along closed source AI, open source AI disappeared after XXYYZ AI ACT. Turns out, Senator X and Senator Y were both in the pocket of closed source AI (and ended up in cushy jobs in Microsoft and Xai, not a coincidence) ... "
    - observationist 2y ago
      Regardless of your perspective on Musk, X AI is currently producing open source AI with permissive licensing, and seems very likely to continue open source releases in the near future. https://github.com/xai-org https://github.com/xai-org Microsoft, Amazon, OpenAI, others are driving regulatory capture behind the scenes. The usual suspects are dropping all sorts of money on establishing control and rent seeking - actual open source AI with end user control makes it much harder for these asshats to extract money and exert influence over people, and they desperately want both. AI, like search, will be a powerful influence vector for politics and marketing.
jeffbee 2y ago
SB-1047 has real "stay in your lane" energy. Senator Weiner knows a lot about housing policy and if he spends 100% of his legislative efforts on housing policy people will celebrate that. He knows nothing whatsoever about computer science, and should just step away from the keyboard to avoid the temptation to legislate the impossible.
- deleted 2y ago
  [deleted]
changoplatanero 2y ago
I’m worried that regulations like these will create a lock in effect that benefits existing leading AI companies and makes it impossible for new entrants
- deleted 2y ago
  [deleted]
- UncleEntity 2y ago
  Some might argue that's the intended purpose.
- brutal_chaos_ 2y ago
  That is called regulatory capture, which could definitely be happening here.
  - qzx_pierri 2y ago
    Politics are depressing :/ Has the tech industry always been so cutthroat, or is this a new trend? Maybe I just didn't hear about tech companies lobbying for power in the past?
    - brutal_chaos_ 2y ago
      I can't say for sure wrt tech industry specifically, but regulatory capture is nothing new.
    - dylan604 2y ago
      What do you consider the tech industry? Do you consider Wall St firms to be tech? Do you consider bigPharma to be tech? Do you consider FAANG to be tech? Here's a link[0] with 2023 lobby spends by industry, but there is no "tech" listing specific. There's an entry for "Internet" listed, which I'm guessing is what you mean by "tech". Another chart[1] breaks down that entry. If you want to know when each company started to spend money, you could research their public filings. [0] https://www.statista.com/statistics/257364/top-lobbying-industries-in-the-us/ https://www.statista.com/statistics/257364/top-lobbying-indu... [1] https://www.statista.com/statistics/1035987/us-leading-internet-lobbying-spenders/ https://www.statista.com/statistics/1035987/us-leading-inter...
carterschonwald 2y ago
The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.
- causal 2y ago
  Important point, but I don't really have a reference for that number. Do we know what GPT-4 or other models would be in comparison?
  - srackey 2y ago
    No, it’s not a good point. It could be 10^100000000 flops and it wouldn’t fucking matter. There’s no evidence that would do anything at all. “AGI” may not even be possible with those flops. You’re talking about what would require unprecedented control over computing to ease fears over a scary robot fanfic. None of the “safety” concerns are real. A GPT4 open source would do nothing but hurt Sam Altman’s bottom line - YOU ARE BEING DUPED.
- changoplatanero 2y ago
  With algorithmic improvements and new hardware, achieving comparable performance to 10^26 flops may not be so absurd anymore.
- srackey 2y ago
  If you’re not being serious, I appreciate your sarcasm. If you are, this may be one of the worst things I have ever read. It’s fucking math people. Math. It will not create some scary golem. It will create marginally better chatbots. You are arguing for totalitarian control over computing to line Microsoft’s pockets. Shame. Shame shame shame shame shame.
- stale2002 2y ago
  Its not an absurd number, depending on how it is calculated. I've seen that number thrown around, and by some calculations it would already apply to open source models like LLama. No, we don't need to ban or regulate LLama or any existing open source models. If someone wants to be worried about GPT-6, fine. But there is no need to regulate the stuff thats already out there.
vm 2y ago
Since it’s not mentioned in the blog: >On February 7, 2024, Senator Scott Wiener introduced Senate Bill 1047 (SB-1047) – known as the Known as the Safe and Secure Innovation for Frontier Artificial Intelligence Systems Act (the Act) – into the California State Legislature. Aiming to regulate the development and use of advanced artificial intelligence (AI) models, the Act mandates developers to make certain safety determinations before training AI models, comply with various safety requirements, and report AI safety incidents. It further establishes the Frontier Model Division within the Department of Technology for oversight of these AI models and introduces civil penalties for violations of the Act. https://www.dlapiper.com/en/insights/publications/2024/02/californias-sb-1047 https://www.dlapiper.com/en/insights/publications/2024/02/ca...
- RobotToaster 2y ago
  The Scott Wiener Congressional Exploratory Cmte received $8,600 from OpenAI. https://www.opensecrets.org/orgs/openai/summary?id=D000084252 https://www.opensecrets.org/orgs/openai/summary?id=D00008425...
  - michaelt 2y ago
    It's impressive how affordable politicians are. You've got a company valued at $80 billion and you can get legislation put forward to kneecap your primary competition for a the price of a 2009 Honda Accord with 150,000 miles on the odometer? What great value for money!
    - euroderf 2y ago
      Politicians at prices so low we're practically GIVING them away ! (pace Crazy Eddie)
    - JumpCrisscross 2y ago
      > can get legislation put forward This typically works the other way. You find politicians who support you, due to personal views or electoral idiosyncrasies, and then give them money to boost them.
throwaway918274 2y ago
mmmmmmm regulatory capture, wouldn't be america without it - nothing says freedom and open markets like sawing off the rungs from the bottom of the ladder after you've already climbed to the top
- Cthulhu_ 2y ago
  Open source AI is not at the bottom of the ladder; like closed source AI, some of the brightest, well-educated and arguably well-paid people in the world are the driving force behind it. OpenAI, arguably at the top of the ladder, was supposed to be open source as well.
  - throwaway918274 2y ago
    It's not a perfect analogy, but it's still corporate interests trying to entrench their position to prevent disruption by less politically powerful players.
dbish 2y ago
Which states would you speculate will be best long term for AI startups? I would’ve guessed California but it’s looking more important to pick a state less likely to get over their skis with regulation. Washington doesn’t seem yet to be doing this, and no state tax with lots of AI engineers/researchers in Seattle/redmond which is why I’m here. Texas probably won’t add regulations and has similar pros if you’re in Austin. Anywhere else looking like it will crop up if California regulates away the industry?
- brigadier132 2y ago
  Miami has a population that is extremely ideologically opposed to laws like this.
  - dbish 2y ago
    Really doesn’t seem like Florida is that opposed to banning innovation for incumbents, not the kind of place I’d trust my startup to really given big restrictive state government over small gov https://x.com/andercot/status/1786169027007783227?s=46&t=3ZOOn0d8R-MnGsnHKqJj6Q https://x.com/andercot/status/1786169027007783227?s=46&t=3ZO...
  - dbish 2y ago
    They don’t seem to have any reason that there would be AI experts there though. No big comp sci universities or existing research labs and tech companies that you can pull talent from.
    - brigadier132 2y ago
      No but people go where money is. It sucks that California covers 80% of the West coast.
      - dbish 2y ago
        I don’t think that worked when Miami was being hyped over the last few years and I don’t expect that to work in the future. Needs a base of research and big tech to start.
ctoth 2y ago
This is a very, very bad article that reeks of LLM-generation. If you want to actually understand this bill, read Zvi's analysis: https://thezvi.substack.com/p/on-the-proposed-california-sb-1047 https://thezvi.substack.com/p/on-the-proposed-california-sb-...
- Cthulhu_ 2y ago
  > reeks of LLM-generation. > Answer.AI is a new kind of AI R&D lab which creates practical end-user products based on foundational research breakthroughs. It's very likely the company / author is dogfooding.
- Imnimo 2y ago
  I think Zvi is missing some critical points about the bill. For example: >Before initiating the commercial, public, or widespread use of a covered model that is not subject to a positive safety determination, limited duty exemption, a developer of the nonderivative version of the covered model shall do all of the following: >(1) Implement reasonable safeguards and requirements to do all of the following: >(B) Prevent an individual from being able to use the model to create a derivative model that was used to cause a critical harm. This is simply impossible. If you give me model weights, I can surely fine-tune them into doing a covered harm (e.g. provide instructions for the creation of chemical or biological weapons). This requirement is unsatisfiable, and you're not allowed to release a covered model without satisfying it.
  - ctoth 2y ago
    From Zvi's article: > The definition of covered model seems to me to be clearly intended to apply only to models that are effectively at the frontier of model capabilities. > Let’s look again at the exact definition: > (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024, or a model that could reasonably be expected to have similar performance on benchmarks commonly used to quantify the performance of state-of-the-art foundation models, as determined by industry best practices and relevant standard setting organizations. > (2) The artificial intelligence model has capability below the relevant threshold on a specific benchmark but is of otherwise similar general capability. > That seems clear as day on what it means, and what it means is this: > 1. > If your model is over 10^26 we assume it counts. > 2. > If it isn’t, but it is as good as state-of-the-art current models, it counts. > 3. > Being ‘as good as’ is a general capability thing, not hitting specific benchmarks. > Under this definition, if no one was actively gaming benchmarks, at most three existing models would plausibly qualify for this definition: GPT-4, Gemini Ultra and Claude. I am not even sure about Claude. > If the open source models are gaming the benchmarks so much that they end up looking like a handful of them are matching GPT-4 on benchmarks, then what can I say, maybe stop gaming the benchmarks? > Or point out quite reasonably that the real benchmark is user preference, and in those terms, you suck, so it is fine. Either way. > But notice that this isn’t what the bill does. The bill applies to large models and to any models that reach the same performance regardless of the compute budget required to make them. This means that the bill applies to startups as well as large corporations. > Um, no, because the open model weights models do not remotely reach the performance level of OpenAI? > Maybe some will in the future. > But this very clearly does not ‘ban all open source.’ There are zero existing open model weights models that this bans. So no, it does not seem that anything was missed.
vouaobrasil 2y ago
There is no such thing as AI safety. AI is far too dangerous. The only thing that exists with regard to AI is "distracting the population so they think AI benefits them" or "AI is too amusing so I don't want to think about the consequences".
- bgdhyfhy 2y ago
  [flagged]
verdverm 2y ago
(f) “Covered model” means an artificial intelligence model that meets either of the following criteria: (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations. (2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models. --- f.1: anyone who has taken a basic cpu arch class knows that int and float are significantly different computational effort. One could see an entity using this in court to greatly lower the threshold for qualification after the fact. i.e. lawyers play word/text games and one could say something to the effect that 1 float is 10 int ops, so the limit is 10^26 int or 10^25 float ops f.2: future proofing against better algorithms based on today's benchmarks... to the point where effort no longer matters. They seem to be drawing the threshold at today's benchmarks, whether or not they are reflective of capability. I could see a small model be trained to do poorly on these benchmarks while excelling at the problems they are concerned with, like making nuclear weapons...
- CSSer 2y ago
  Forget ints. What FP precision? That alone could skew numbers.
- verdverm 2y ago
  More exciting snippets from the bill > 22603. (a) Before initiating training of a covered model that is not a derivative model, a developer of that covered model may determine whether ... if the covered model will have lower performance on all benchmarks because I know how it will perform before training?
- exe34 2y ago
  In a way this could be what gives us AGI that runs in your pocket. If there's an upper limit on what can be used, then human ingenuity will be funnelled towards whatever does fit in. Of course, this is only in the USA. China, Russia, North Korea, Iran, etc will still be free to persue the technology.
ubj 2y ago
> The creator of a model can not ensure that a model is never used to do something harmful – any more so that the developer of a web browser, calculator, or word processor could. Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams. This matches my thoughts on why this is ultimately a bad piece of legislation. It is virtually impossible to ensure that a piece of technology will not be used for "harmful purposes". I agree that such stipulations will be just another roadblock keeping everyone except "big businesses with well funded legal teams" from working on LLMs.
- bee_rider 2y ago
  Too bad there isn’t an NRA for LLM’s. So far at least nobody has been literally murdered using an LLM, as far as I’ve heard.
- sanxiyn 2y ago
  As I understand this law does not mandate you to ensure anything. It requires you to follow best practices (to be determined), report safety incidents, etc. You are not even liable for safety incidents, you just need to report them, although it may be embarrassing. Overall, it seems highly reasonable.
  - echelon 2y ago
    > requires you to follow best practices (to be determined) Trigger happy regulation for a field that hasn't even come into full swing. It's indicative of an over-active immune system; lawmakers with nothing better to do. Pass laws against improper use and go after the malicious users. Don't ban the technology, the research, or even the applications. (Of which there will be abundant good uses. Many of which we've yet to even see or predict.) Our culture has become obsessed with regulating and limiting freedom on the very principle that it might be harmful. We should be punishing actual measurable, physical and monetary harms. Not imaginary or hypothetical ones. If California passes this, AI companies should leave California behind.
    - 2y ago
nimbius 2y ago
this feels a lot like Uber/Lyft whining about rideshare regulation, or Musk whining about self-driving regulation. Howard does a miserable job attempting to explain what, exactly, this legislation will do to impact open source development and clearly hasnt read the specifics for open-source development in the bill. things like dual use/ITAR apply explicitly to commercial products, but he attempts to conflate them with open source instead and uses inference to try and define them. they are very well defined trade concepts at the federal level. from the bill, the specific applications to open source are: Appoint and consult with an advisory committee for open-source artificial intelligence that shall do all of the following: (A) Issue guidelines for model evaluation for use by developers of open-source artificial intelligence models that do not have hazardous capabilities. (B) Advise the Frontier Model Division on the creation and feasibility of incentives, including tax credits, that could be provided to developers of open-source artificial intelligence models that are not covered models. (C) Advise the Frontier Model Division on future policies and legislation impacting open-source artificial intelligence development. nowhere does it state open source developers need "required shutdowns" or burdensome reporting for open source. The states position to regulate trade is sacrosanct and in such, the bill applies almost entirely to commercial products. it would affect Jeremys business and as a business owner, he doesnt like that. here is the actual bill https://legiscan.com/CA/text/SB1047/id/2919384 https://legiscan.com/CA/text/SB1047/id/2919384
- hellojesus 2y ago
  > nowhere does it state open source developers need "required shutdowns" or burdensome reporting for open source. Any reporting to a gov agency is burdensome. It all exists to stifle private development.
- jahewson 2y ago
  > things like dual use/ITAR apply explicitly to commercial products No, just no. Have you ever been to a University?
sanxiyn 2y ago
I disagree on "Focus on Usage, Not Development". > Regulate the use of AI in high-risk areas such as healthcare, criminal justice, and critical infrastructure, where the potential for harm is greatest This suggests, for example, image generation should be unregulated, but potential for harm of deepfake is great. In general, regulation needs to be feasible to implement, and even if it is ideal to regulate use not development, it can make a sense to regulate development due to feasibility concerns.
- root_axis 2y ago
  Deepfakes are already possible and entirely convincing with existing technology, I don't see the benefits of stifling the development of open source transformer models for fear of deep fakes.
- mattnewton 2y ago
  It’s generally not possible for California state legislature to regulate deepfakes; closing the metaphorical doors now doesn’t make sense as the metaphorical horses weren’t even contained in their barn to begin with. (I’d argue it’s not desirable for the US to try regulating the creation of these tools rather than their use at a federal level either but that’s another discussion)
- jahewson 2y ago
  That’s like banning cars because some people use them to run over other people.
casebash 2y ago
While this article makes some valid points, it basically just ignores the reasons why the law is being passed, that is the potential for open-models to enable bio-attacks, cyberattacks, election manipulation, automated personalised scams, and who knows what else. One might question why that is. Perhaps it's the case that Jeremy has an excellent response to these points which he has somehow neglected to raise. Or perhaps it's because these threats are very inconvenient for an open source developer. I'm sure he'd say that open-sourcing models means that all actors have access to defensive systems and that the good guys outnumber the bad guys and it'll all work out well. And that could be true. Or it could be false. It's not like we really know that everything would work out fine. It's not that we've run the experiment. I mean maybe it works out like that, or maybe one guy creates a virus and then it doesn't really matter how many folk on the other side, but we still get kind of screwed because we can only produce vaccines that fast. It's that's what going to happen? I don't really know, but it's at least plausible. I mean, maybe we'll automate all aspects of vaccine production and be able to respond much faster, but that's dependent on when we develop this technology vs. when AI starts significantly helping with bioweapons with someone then using it for an attack. And at that point it's all so uncertain and up in the air that it's seems rather strange for someone to suggest that it'll all be fine.
- simion314 2y ago
  You imagine some extremist could somehow use llama version 11 to print viruses from his printer ? LLMs are not intelligent, they predict text based on what it was trained, if it could somehow build new viruses, weapons then it means the internet has MANY such information so the LLC could predict something useful, so maybe those websites, scientific papers , blog posts need to be deleted because some extremist group or state sponsored group can use them directly plus Natural Intelligence plus good laboratories. But tell me how can I make my next LLM so it would help on say fighting biologic weapons, creating vaccines but refusing to make evil stuff keeping in mind that jailbreaking is always possible (scientifically proven)
- deleted 2y ago
  [deleted]
- michaelmrose 2y ago
verdverm 2y ago
Current text of the bill: https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202320240SB1047 https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...
gliched_robot 2y ago
I do not understand the taught process here. They are regulating it so fast. It's almost like regulating car before even engine is invented.
- sanxiyn 2y ago
  ChatGPT exists.
huevosabio 2y ago
Unbelievable. I like Scott Wiener for his housing policies but this bill is rampant overstepping from the government. Ironically, it will have the same effect as the NIMBY system he has fought for so long. Some nuggets... --- So we are only allowed to train what they allow us: This bill would require that a developer, before initiating training of a nonderivative covered model, comply with various requirements, including implementing the capability to promptly enact a full shutdown of the covered model until that covered model is the subject of a limited duty exemption. --- Of course it comes with a new department with powers to impose fees: This bill would also create the Frontier Model Division within the Department of Technology and would require the division to, among other things, review annual certification reports from developers received pursuant to these provisions and publicly release summarized findings based on those reports. The bill would authorize the division to assess related fees and would require deposit of the fees into the Frontier Model Division Programs Fund, which the bill would create. --- And, obviously, we must pay consultants: This bill would also require the Department of Technology to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale artificial intelligence models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform.
- sanxiyn 2y ago
  > including implementing the capability to promptly enact a full shutdown So you need to be able to turn it off. It should be easy? Still it seems to be a good thing to make sure you can turn it off.
  - jpollock 2y ago
    People saying “no, you can’t do that” is why open source software exists.
  - Dig1t 2y ago
    >until that covered model is the subject of a limited duty exemption. The second half of that sentence clarifies. It's not that you have to be able to shut down the model, everyone knows it's trivial to turn off a computer. It's that the government can force you to shut down your program until such time as they give you regulatory approval to turn it back on again.
mehulashah 2y ago
This is a well-articulated response to the SB-1047 bill. I want to underscore the undercurrent of the message — that with the proposed regulations on creators, it will accomplish the opposite of what it is intended to do. The landscape will be less open and diverse, handing the power to a few. Another point that gets buried is that AI is about the data. Without the transparency of what these models are built from, it leads to potential dangers as well as inappropriate use of materials. My bet is on the open ecosystem if it doesn’t get legislated away.
- freedomben 2y ago
  > The landscape will be less open and diverse, handing the power to a few. Proponents of the regulation approach probably wouldn't state it this way, but if I'm understanding their arguments correctly, I think they want that, because regularing a few very powerful corporations is easy. Regulating a ton of small people/startups is hard. When you genuinely believe that some of the output from LLMs is literally dangerous to some people, it's not unreasonable to decide that the "freedom" of people to run and develop models to compete is unimportant compared to protecting society from dangerous text or images.
oblib 2y ago
AI is really a giant plagiarism machine and I could not care less about it being tightly regulated.
- deleted 2y ago
  [deleted]
_akhe 2y ago
This is our sign to start boycotting OpenAI - they're behind the lobbying of this. Instead of writing your senator who is owned by OpenAI - just throw away your `OPENAI_API_KEY` and use one of the many open models like mistral or llama3. ollama run mistral It's very easy to get started, right in your Terminal. And there are cloud providers like https://replicate.com/ https://replicate.com/ and https://lightning.ai/ https://lightning.ai/ that will let you use your LLM via an API key just like you did with OpenAI if you need that. You don't need OpenAI - nobody does.
kevinsync 2y ago
Honest question -- if models (much like code) are open-sourced anonymously, what can governments and politicians do about it? This sounds like the "export-restricted cryptography" foolishness of the 90's all over again.
- michaelmrose 2y ago
  Seize your domain name. Force anyone to refuse to do business with you and or give up enough info to find you and put you in prison anywhere in the US or any country with an extradition treaty which is most of the better places to live.
  - jahewson 2y ago
    Extradition is only possible in cases where both countries have (approximately) equivalent laws and the crime is sufficiently serious.
ninjagoo 2y ago
This is worrisome for open source - this isn't some far off future limit. Llama3 training at 400 TFLOPS per GPU [1] and 6.4M GPU hours [2] puts Llama3-70B at 9.2*10^24 (so 10^25) floating point ops. Llama3-400B might be well past this threshold. [1] https://ai.meta.com/blog/meta-llama-3/ https://ai.meta.com/blog/meta-llama-3/ [2] https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct
simonw 2y ago
> Instead of regulating the development of AI models, the focus should be on regulating their applications, particularly those that pose high risks to public safety and security. Regulate the use of AI in high-risk areas such as healthcare, criminal justice, and critical infrastructure, where the potential for harm is greatest, would ensure accountability for harmful use, whilst allowing for the continued advancement of AI technology. I really like this proposed model. Are there good arguments against this?
antonioevans 2y ago
What about China and rest of the world. Their training, some are training open source models (Mistral). So the US is going to be left behind?
- ericd 2y ago
  Yeah, there's definitely no putting this back in the bag on a global scale.
- deleted 2y ago
  [deleted]
ltbarcly3 2y ago
It won't stifle anything. It will take about 2 months for all AI companies to abandon California. Forcing them out of SF will make their social lives worse but lower their costs 80%. The end result is that they will spend less time drinking $17 espresso, more time working, and will be able to hire more engineers. AI research accelerates dramatically.
sidcool 2y ago
"Note from Jeremy: This is my personal submission to the authors of bill SB-1047. It’s not an official Answer.AI statement." Then why post on answer.ai?
Nifty3929 2y ago
"This could inadvertently criminalize the activities of well-intentioned developers working on beneficial AI projects." It wouldn't be inadvertent. It's a control tactic. "Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams." ... "These requirements could disproportionately impact open-source developers who often lack the resources of larger corporations to navigate complex regulatory processes." ... "The proposed regulations create significant barriers to entry for small businesses and startups looking to innovate in the AI space." That's the idea. The government likes a small number of big businesses that they can control.
- jahewson 2y ago
  How about some liability placed on members of government for their hopeless legislating? As developers of law shouldn’t they accept responsibility for the harms they cause?
deleted 2y ago
[deleted]
bgdhyfhy 2y ago
[dead]
hendrycks 2y ago
A few things that I’m seeing folks in the comments misunderstanding about the bill (full disclosure: I’ve been one of a group of folks advising Senator Wiener on SB 1047) 1. The new Frontier Model Division is focused on receiving information and issuing guidelines. It’s not a licensing regime and isn’t investigating developers. 2. Folks aren’t automatically liable if their highly capable model is used to do bad things, even catastrophic things. The question is whether they took reasonable measures to prevent that. This bill could have used strict liability, where developers would be liable for catastrophic harms regardless of fault, but that's not what the bill does. 3. The bill requires developers to test their models and report whether they have hazardous capabilities (and the answer can obviously be yes or no). Even if the model does have hazardous capabilities, the developer can still deploy it if they take reasonable precautions, as outlined in the bill. For perjury, you would need to intentionally lie—good faith errors would not be covered. I get that models can have unforeseen capabilities, but this isn’t about that. If you are knowingly releasing something that could have demonstrably catastrophic consequences, it seems fair to have consequences for that. Some things which already require folks to certify under penalty of perjury: lobbying disclosures, companies’ financial disclosures, immigration compliance forms. 4. Overall it seems pretty reasonable that if your model can cause catastrophic harms (which is not true of current models, but maybe true of future models), then you shouldn’t be releasing models in a way that can predictably allow folks to cause those catastrophic harms. If people want a writeup of what the bill does I recommend this one by the law firm DLA Piper (https://www.dlapiper.com/en/insights/publications/2024/02/californias-sb-1047 https://www.dlapiper.com/en/insights/publications/2024/02/ca...). In my opinion this is a pretty narrow proposal focused at the most severe risks (much more narrow than, e.g., the EU AI act).
blackeyeblitzar 2y ago
The entire notion of “safety” and “ethics” in AI is simply a Trojan horse for injecting government control and censorship over speech and expression. That’s what the governments get out of it. The big AI players like OpenAI, Microsoft, Amazon, Google, etc. are incentivized to go along with it because it helps them through regulatory capture and barriers to competition. They also make some friends with powerful legislators to avoid pesky things like antitrust scrutiny. Legislation should not restrict the development or operation of fundamental AI technologies. Instead laws should only be built on the specific uses that are deemed illegal, irrespective of AI.

225 points by kmdupree 2y ago

iptq 2y ago

This entire article feels like it was written by ChatGPT. For instance, it continuously makes vague claims about the value of open source without even citing the original bill.

jsheard 2y ago
The eagerness to summarize points in bulleted lists is a ChatGPT-ism as well.
- exe34 2y ago
  Hey now, I like bullet points myself, as I'm easily distracted otherwise.
shkkmo 2y ago
This is going to become one the the site no-nos. What does this assertion add to the discussion? Is there any argument that serves as refutation?
- sensanaty 2y ago
  Why would I want to read some AI-generated slop? It's no different to spam, other than techbros being okay shoving it everywhere and anywhere it doesn't belong.
- pjc50 2y ago
  If an article is AI, it's essentially spam, because there could be a thousand more similar written.
- airstrike 2y ago
  Raising awareness so others too can flag TFA
  - shkkmo 2y ago
    The quality of the article is what should drive voting and flagging. Your personal opinion as to what level of AI tooling was used in crafting the content is a low value signal and posting that speculation without meaningfully engaging in the content is as low value as commenting to say "this article sucks". Personally, I think this would be a very strange place to find pute AI genetated content. This is a personal statement that was submitted to the state, and posted under a real name under a site the poster has a professional association with. I think that any "strangeness" in formating a wording comes from the role this text serves, as a public comment intended to affect policy.

narrator 2y ago

The politicians are gunning extremely hard for open source AI. It's crazy. "Soros argued that synergy like that between corporate and government AI projects creates a more potent threat than was posed by Cold War–era autocrats, many of whom spurned corporate innovation. “The combination of repressive regimes with IT monopolies endows those regimes with a built-in advantage over open societies,” Soros said. “They pose a mortal threat to open societies.” https://www.wired.com/story/mortal-danger-chinas-push-into-ai/ https://www.wired.com/story/mortal-danger-chinas-push-into-a... Literally everyone out there who pursues global influence is just frothing at the mouth over AI. This is seriously tempting me to buy the 512gb Mac Studio when it comes out so I can run the big llama3 model, which will probably be banned any day now.

Cthulhu_ 2y ago
> The politicians are gunning extremely hard for open source AI. It's crazy. I hope someone has / can do some investigative journalism to check out their links with commercial / closed source AI; I can imagine the investors and those that benefit from companies like "Open"AI have close links with politicians. There's probably no direct links, they've become really good at obscuring those and plausible deniability.
- kranke155 2y ago
  You'll hear about it in 10 years in an article about "why did open source Ai die? ... once blossoming along closed source AI, open source AI disappeared after XXYYZ AI ACT. Turns out, Senator X and Senator Y were both in the pocket of closed source AI (and ended up in cushy jobs in Microsoft and Xai, not a coincidence) ... "
  - observationist 2y ago
    Regardless of your perspective on Musk, X AI is currently producing open source AI with permissive licensing, and seems very likely to continue open source releases in the near future. https://github.com/xai-org https://github.com/xai-org Microsoft, Amazon, OpenAI, others are driving regulatory capture behind the scenes. The usual suspects are dropping all sorts of money on establishing control and rent seeking - actual open source AI with end user control makes it much harder for these asshats to extract money and exert influence over people, and they desperately want both. AI, like search, will be a powerful influence vector for politics and marketing.

jeffbee 2y ago

SB-1047 has real "stay in your lane" energy. Senator Weiner knows a lot about housing policy and if he spends 100% of his legislative efforts on housing policy people will celebrate that. He knows nothing whatsoever about computer science, and should just step away from the keyboard to avoid the temptation to legislate the impossible.

deleted 2y ago
[deleted]

changoplatanero 2y ago

I’m worried that regulations like these will create a lock in effect that benefits existing leading AI companies and makes it impossible for new entrants

deleted 2y ago
[deleted]
UncleEntity 2y ago
Some might argue that's the intended purpose.
brutal_chaos_ 2y ago
That is called regulatory capture, which could definitely be happening here.
- qzx_pierri 2y ago
  Politics are depressing :/ Has the tech industry always been so cutthroat, or is this a new trend? Maybe I just didn't hear about tech companies lobbying for power in the past?
  - brutal_chaos_ 2y ago
    I can't say for sure wrt tech industry specifically, but regulatory capture is nothing new.
  - dylan604 2y ago
    What do you consider the tech industry? Do you consider Wall St firms to be tech? Do you consider bigPharma to be tech? Do you consider FAANG to be tech? Here's a link[0] with 2023 lobby spends by industry, but there is no "tech" listing specific. There's an entry for "Internet" listed, which I'm guessing is what you mean by "tech". Another chart[1] breaks down that entry. If you want to know when each company started to spend money, you could research their public filings. [0] https://www.statista.com/statistics/257364/top-lobbying-industries-in-the-us/ https://www.statista.com/statistics/257364/top-lobbying-indu... [1] https://www.statista.com/statistics/1035987/us-leading-internet-lobbying-spenders/ https://www.statista.com/statistics/1035987/us-leading-inter...

carterschonwald 2y ago

The law seems to only care about models that take at least 10^26 floating point ops or models that achieve comparable benchmark performance. This is a truly absurd number! I’m ok with organizations with that sort of compute capacity being subject to regulatory oversight and reporting and liability.

causal 2y ago
Important point, but I don't really have a reference for that number. Do we know what GPT-4 or other models would be in comparison?
- srackey 2y ago
  No, it’s not a good point. It could be 10^100000000 flops and it wouldn’t fucking matter. There’s no evidence that would do anything at all. “AGI” may not even be possible with those flops. You’re talking about what would require unprecedented control over computing to ease fears over a scary robot fanfic. None of the “safety” concerns are real. A GPT4 open source would do nothing but hurt Sam Altman’s bottom line - YOU ARE BEING DUPED.
changoplatanero 2y ago
With algorithmic improvements and new hardware, achieving comparable performance to 10^26 flops may not be so absurd anymore.
srackey 2y ago
If you’re not being serious, I appreciate your sarcasm. If you are, this may be one of the worst things I have ever read. It’s fucking math people. Math. It will not create some scary golem. It will create marginally better chatbots. You are arguing for totalitarian control over computing to line Microsoft’s pockets. Shame. Shame shame shame shame shame.
stale2002 2y ago
Its not an absurd number, depending on how it is calculated. I've seen that number thrown around, and by some calculations it would already apply to open source models like LLama. No, we don't need to ban or regulate LLama or any existing open source models. If someone wants to be worried about GPT-6, fine. But there is no need to regulate the stuff thats already out there.

vm 2y ago

Since it’s not mentioned in the blog: >On February 7, 2024, Senator Scott Wiener introduced Senate Bill 1047 (SB-1047) – known as the Known as the Safe and Secure Innovation for Frontier Artificial Intelligence Systems Act (the Act) – into the California State Legislature. Aiming to regulate the development and use of advanced artificial intelligence (AI) models, the Act mandates developers to make certain safety determinations before training AI models, comply with various safety requirements, and report AI safety incidents. It further establishes the Frontier Model Division within the Department of Technology for oversight of these AI models and introduces civil penalties for violations of the Act. https://www.dlapiper.com/en/insights/publications/2024/02/californias-sb-1047 https://www.dlapiper.com/en/insights/publications/2024/02/ca...

RobotToaster 2y ago
The Scott Wiener Congressional Exploratory Cmte received $8,600 from OpenAI. https://www.opensecrets.org/orgs/openai/summary?id=D000084252 https://www.opensecrets.org/orgs/openai/summary?id=D00008425...
- michaelt 2y ago
  It's impressive how affordable politicians are. You've got a company valued at $80 billion and you can get legislation put forward to kneecap your primary competition for a the price of a 2009 Honda Accord with 150,000 miles on the odometer? What great value for money!
  - euroderf 2y ago
    Politicians at prices so low we're practically GIVING them away ! (pace Crazy Eddie)
  - JumpCrisscross 2y ago
    > can get legislation put forward This typically works the other way. You find politicians who support you, due to personal views or electoral idiosyncrasies, and then give them money to boost them.

throwaway918274 2y ago

mmmmmmm regulatory capture, wouldn't be america without it - nothing says freedom and open markets like sawing off the rungs from the bottom of the ladder after you've already climbed to the top

Cthulhu_ 2y ago
Open source AI is not at the bottom of the ladder; like closed source AI, some of the brightest, well-educated and arguably well-paid people in the world are the driving force behind it. OpenAI, arguably at the top of the ladder, was supposed to be open source as well.
- throwaway918274 2y ago
  It's not a perfect analogy, but it's still corporate interests trying to entrench their position to prevent disruption by less politically powerful players.

dbish 2y ago

Which states would you speculate will be best long term for AI startups? I would’ve guessed California but it’s looking more important to pick a state less likely to get over their skis with regulation. Washington doesn’t seem yet to be doing this, and no state tax with lots of AI engineers/researchers in Seattle/redmond which is why I’m here. Texas probably won’t add regulations and has similar pros if you’re in Austin. Anywhere else looking like it will crop up if California regulates away the industry?

brigadier132 2y ago
Miami has a population that is extremely ideologically opposed to laws like this.
- dbish 2y ago
  Really doesn’t seem like Florida is that opposed to banning innovation for incumbents, not the kind of place I’d trust my startup to really given big restrictive state government over small gov https://x.com/andercot/status/1786169027007783227?s=46&t=3ZOOn0d8R-MnGsnHKqJj6Q https://x.com/andercot/status/1786169027007783227?s=46&t=3ZO...
- dbish 2y ago
  They don’t seem to have any reason that there would be AI experts there though. No big comp sci universities or existing research labs and tech companies that you can pull talent from.
  - brigadier132 2y ago
    No but people go where money is. It sucks that California covers 80% of the West coast.
    - dbish 2y ago
      I don’t think that worked when Miami was being hyped over the last few years and I don’t expect that to work in the future. Needs a base of research and big tech to start.

ctoth 2y ago

This is a very, very bad article that reeks of LLM-generation. If you want to actually understand this bill, read Zvi's analysis: https://thezvi.substack.com/p/on-the-proposed-california-sb-1047 https://thezvi.substack.com/p/on-the-proposed-california-sb-...

Cthulhu_ 2y ago
> reeks of LLM-generation. > Answer.AI is a new kind of AI R&D lab which creates practical end-user products based on foundational research breakthroughs. It's very likely the company / author is dogfooding.
Imnimo 2y ago
I think Zvi is missing some critical points about the bill. For example: >Before initiating the commercial, public, or widespread use of a covered model that is not subject to a positive safety determination, limited duty exemption, a developer of the nonderivative version of the covered model shall do all of the following: >(1) Implement reasonable safeguards and requirements to do all of the following: >(B) Prevent an individual from being able to use the model to create a derivative model that was used to cause a critical harm. This is simply impossible. If you give me model weights, I can surely fine-tune them into doing a covered harm (e.g. provide instructions for the creation of chemical or biological weapons). This requirement is unsatisfiable, and you're not allowed to release a covered model without satisfying it.
- ctoth 2y ago
  From Zvi's article: > The definition of covered model seems to me to be clearly intended to apply only to models that are effectively at the frontier of model capabilities. > Let’s look again at the exact definition: > (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024, or a model that could reasonably be expected to have similar performance on benchmarks commonly used to quantify the performance of state-of-the-art foundation models, as determined by industry best practices and relevant standard setting organizations. > (2) The artificial intelligence model has capability below the relevant threshold on a specific benchmark but is of otherwise similar general capability. > That seems clear as day on what it means, and what it means is this: > 1. > If your model is over 10^26 we assume it counts. > 2. > If it isn’t, but it is as good as state-of-the-art current models, it counts. > 3. > Being ‘as good as’ is a general capability thing, not hitting specific benchmarks. > Under this definition, if no one was actively gaming benchmarks, at most three existing models would plausibly qualify for this definition: GPT-4, Gemini Ultra and Claude. I am not even sure about Claude. > If the open source models are gaming the benchmarks so much that they end up looking like a handful of them are matching GPT-4 on benchmarks, then what can I say, maybe stop gaming the benchmarks? > Or point out quite reasonably that the real benchmark is user preference, and in those terms, you suck, so it is fine. Either way. > But notice that this isn’t what the bill does. The bill applies to large models and to any models that reach the same performance regardless of the compute budget required to make them. This means that the bill applies to startups as well as large corporations. > Um, no, because the open model weights models do not remotely reach the performance level of OpenAI? > Maybe some will in the future. > But this very clearly does not ‘ban all open source.’ There are zero existing open model weights models that this bans. So no, it does not seem that anything was missed.

vouaobrasil 2y ago

There is no such thing as AI safety. AI is far too dangerous. The only thing that exists with regard to AI is "distracting the population so they think AI benefits them" or "AI is too amusing so I don't want to think about the consequences".

bgdhyfhy 2y ago
[flagged]

verdverm 2y ago

(f) “Covered model” means an artificial intelligence model that meets either of the following criteria: (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations. (2) The artificial intelligence model was trained using a quantity of computing power sufficiently large that it could reasonably be expected to have similar or greater performance as an artificial intelligence model trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024 as assessed using benchmarks commonly used to quantify the general performance of state-of-the-art foundation models. --- f.1: anyone who has taken a basic cpu arch class knows that int and float are significantly different computational effort. One could see an entity using this in court to greatly lower the threshold for qualification after the fact. i.e. lawyers play word/text games and one could say something to the effect that 1 float is 10 int ops, so the limit is 10^26 int or 10^25 float ops f.2: future proofing against better algorithms based on today's benchmarks... to the point where effort no longer matters. They seem to be drawing the threshold at today's benchmarks, whether or not they are reflective of capability. I could see a small model be trained to do poorly on these benchmarks while excelling at the problems they are concerned with, like making nuclear weapons...

CSSer 2y ago
Forget ints. What FP precision? That alone could skew numbers.
verdverm 2y ago
More exciting snippets from the bill > 22603. (a) Before initiating training of a covered model that is not a derivative model, a developer of that covered model may determine whether ... if the covered model will have lower performance on all benchmarks because I know how it will perform before training?
exe34 2y ago
In a way this could be what gives us AGI that runs in your pocket. If there's an upper limit on what can be used, then human ingenuity will be funnelled towards whatever does fit in. Of course, this is only in the USA. China, Russia, North Korea, Iran, etc will still be free to persue the technology.

ubj 2y ago

> The creator of a model can not ensure that a model is never used to do something harmful – any more so that the developer of a web browser, calculator, or word processor could. Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams. This matches my thoughts on why this is ultimately a bad piece of legislation. It is virtually impossible to ensure that a piece of technology will not be used for "harmful purposes". I agree that such stipulations will be just another roadblock keeping everyone except "big businesses with well funded legal teams" from working on LLMs.

bee_rider 2y ago
Too bad there isn’t an NRA for LLM’s. So far at least nobody has been literally murdered using an LLM, as far as I’ve heard.
sanxiyn 2y ago
As I understand this law does not mandate you to ensure anything. It requires you to follow best practices (to be determined), report safety incidents, etc. You are not even liable for safety incidents, you just need to report them, although it may be embarrassing. Overall, it seems highly reasonable.
- echelon 2y ago
  > requires you to follow best practices (to be determined) Trigger happy regulation for a field that hasn't even come into full swing. It's indicative of an over-active immune system; lawmakers with nothing better to do. Pass laws against improper use and go after the malicious users. Don't ban the technology, the research, or even the applications. (Of which there will be abundant good uses. Many of which we've yet to even see or predict.) Our culture has become obsessed with regulating and limiting freedom on the very principle that it might be harmful. We should be punishing actual measurable, physical and monetary harms. Not imaginary or hypothetical ones. If California passes this, AI companies should leave California behind.
  - 2y ago

nimbius 2y ago

this feels a lot like Uber/Lyft whining about rideshare regulation, or Musk whining about self-driving regulation. Howard does a miserable job attempting to explain what, exactly, this legislation will do to impact open source development and clearly hasnt read the specifics for open-source development in the bill. things like dual use/ITAR apply explicitly to commercial products, but he attempts to conflate them with open source instead and uses inference to try and define them. they are very well defined trade concepts at the federal level. from the bill, the specific applications to open source are: Appoint and consult with an advisory committee for open-source artificial intelligence that shall do all of the following: (A) Issue guidelines for model evaluation for use by developers of open-source artificial intelligence models that do not have hazardous capabilities. (B) Advise the Frontier Model Division on the creation and feasibility of incentives, including tax credits, that could be provided to developers of open-source artificial intelligence models that are not covered models. (C) Advise the Frontier Model Division on future policies and legislation impacting open-source artificial intelligence development. nowhere does it state open source developers need "required shutdowns" or burdensome reporting for open source. The states position to regulate trade is sacrosanct and in such, the bill applies almost entirely to commercial products. it would affect Jeremys business and as a business owner, he doesnt like that. here is the actual bill https://legiscan.com/CA/text/SB1047/id/2919384 https://legiscan.com/CA/text/SB1047/id/2919384

hellojesus 2y ago
> nowhere does it state open source developers need "required shutdowns" or burdensome reporting for open source. Any reporting to a gov agency is burdensome. It all exists to stifle private development.
jahewson 2y ago
> things like dual use/ITAR apply explicitly to commercial products No, just no. Have you ever been to a University?

sanxiyn 2y ago

I disagree on "Focus on Usage, Not Development". > Regulate the use of AI in high-risk areas such as healthcare, criminal justice, and critical infrastructure, where the potential for harm is greatest This suggests, for example, image generation should be unregulated, but potential for harm of deepfake is great. In general, regulation needs to be feasible to implement, and even if it is ideal to regulate use not development, it can make a sense to regulate development due to feasibility concerns.

root_axis 2y ago
Deepfakes are already possible and entirely convincing with existing technology, I don't see the benefits of stifling the development of open source transformer models for fear of deep fakes.
mattnewton 2y ago
It’s generally not possible for California state legislature to regulate deepfakes; closing the metaphorical doors now doesn’t make sense as the metaphorical horses weren’t even contained in their barn to begin with. (I’d argue it’s not desirable for the US to try regulating the creation of these tools rather than their use at a federal level either but that’s another discussion)
jahewson 2y ago
That’s like banning cars because some people use them to run over other people.

casebash 2y ago

While this article makes some valid points, it basically just ignores the reasons why the law is being passed, that is the potential for open-models to enable bio-attacks, cyberattacks, election manipulation, automated personalised scams, and who knows what else. One might question why that is. Perhaps it's the case that Jeremy has an excellent response to these points which he has somehow neglected to raise. Or perhaps it's because these threats are very inconvenient for an open source developer. I'm sure he'd say that open-sourcing models means that all actors have access to defensive systems and that the good guys outnumber the bad guys and it'll all work out well. And that could be true. Or it could be false. It's not like we really know that everything would work out fine. It's not that we've run the experiment. I mean maybe it works out like that, or maybe one guy creates a virus and then it doesn't really matter how many folk on the other side, but we still get kind of screwed because we can only produce vaccines that fast. It's that's what going to happen? I don't really know, but it's at least plausible. I mean, maybe we'll automate all aspects of vaccine production and be able to respond much faster, but that's dependent on when we develop this technology vs. when AI starts significantly helping with bioweapons with someone then using it for an attack. And at that point it's all so uncertain and up in the air that it's seems rather strange for someone to suggest that it'll all be fine.

simion314 2y ago
You imagine some extremist could somehow use llama version 11 to print viruses from his printer ? LLMs are not intelligent, they predict text based on what it was trained, if it could somehow build new viruses, weapons then it means the internet has MANY such information so the LLC could predict something useful, so maybe those websites, scientific papers , blog posts need to be deleted because some extremist group or state sponsored group can use them directly plus Natural Intelligence plus good laboratories. But tell me how can I make my next LLM so it would help on say fighting biologic weapons, creating vaccines but refusing to make evil stuff keeping in mind that jailbreaking is always possible (scientifically proven)
deleted 2y ago
[deleted]
michaelmrose 2y ago

verdverm 2y ago

Current text of the bill: https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202320240SB1047 https://leginfo.legislature.ca.gov/faces/billTextClient.xhtm...

gliched_robot 2y ago

I do not understand the taught process here. They are regulating it so fast. It's almost like regulating car before even engine is invented.

sanxiyn 2y ago
ChatGPT exists.

huevosabio 2y ago

Unbelievable. I like Scott Wiener for his housing policies but this bill is rampant overstepping from the government. Ironically, it will have the same effect as the NIMBY system he has fought for so long. Some nuggets... --- So we are only allowed to train what they allow us: This bill would require that a developer, before initiating training of a nonderivative covered model, comply with various requirements, including implementing the capability to promptly enact a full shutdown of the covered model until that covered model is the subject of a limited duty exemption. --- Of course it comes with a new department with powers to impose fees: This bill would also create the Frontier Model Division within the Department of Technology and would require the division to, among other things, review annual certification reports from developers received pursuant to these provisions and publicly release summarized findings based on those reports. The bill would authorize the division to assess related fees and would require deposit of the fees into the Frontier Model Division Programs Fund, which the bill would create. --- And, obviously, we must pay consultants: This bill would also require the Department of Technology to commission consultants, as prescribed, to create a public cloud computing cluster, to be known as CalCompute, with the primary focus of conducting research into the safe and secure deployment of large-scale artificial intelligence models and fostering equitable innovation that includes, among other things, a fully owned and hosted cloud platform.

sanxiyn 2y ago
> including implementing the capability to promptly enact a full shutdown So you need to be able to turn it off. It should be easy? Still it seems to be a good thing to make sure you can turn it off.
- jpollock 2y ago
  People saying “no, you can’t do that” is why open source software exists.
- Dig1t 2y ago
  >until that covered model is the subject of a limited duty exemption. The second half of that sentence clarifies. It's not that you have to be able to shut down the model, everyone knows it's trivial to turn off a computer. It's that the government can force you to shut down your program until such time as they give you regulatory approval to turn it back on again.

mehulashah 2y ago

This is a well-articulated response to the SB-1047 bill. I want to underscore the undercurrent of the message — that with the proposed regulations on creators, it will accomplish the opposite of what it is intended to do. The landscape will be less open and diverse, handing the power to a few. Another point that gets buried is that AI is about the data. Without the transparency of what these models are built from, it leads to potential dangers as well as inappropriate use of materials. My bet is on the open ecosystem if it doesn’t get legislated away.

freedomben 2y ago
> The landscape will be less open and diverse, handing the power to a few. Proponents of the regulation approach probably wouldn't state it this way, but if I'm understanding their arguments correctly, I think they want that, because regularing a few very powerful corporations is easy. Regulating a ton of small people/startups is hard. When you genuinely believe that some of the output from LLMs is literally dangerous to some people, it's not unreasonable to decide that the "freedom" of people to run and develop models to compete is unimportant compared to protecting society from dangerous text or images.

oblib 2y ago

AI is really a giant plagiarism machine and I could not care less about it being tightly regulated.

deleted 2y ago
[deleted]

_akhe 2y ago

This is our sign to start boycotting OpenAI - they're behind the lobbying of this. Instead of writing your senator who is owned by OpenAI - just throw away your `OPENAI_API_KEY` and use one of the many open models like mistral or llama3. ollama run mistral It's very easy to get started, right in your Terminal. And there are cloud providers like https://replicate.com/ https://replicate.com/ and https://lightning.ai/ https://lightning.ai/ that will let you use your LLM via an API key just like you did with OpenAI if you need that. You don't need OpenAI - nobody does.

kevinsync 2y ago

Honest question -- if models (much like code) are open-sourced anonymously, what can governments and politicians do about it? This sounds like the "export-restricted cryptography" foolishness of the 90's all over again.

michaelmrose 2y ago
Seize your domain name. Force anyone to refuse to do business with you and or give up enough info to find you and put you in prison anywhere in the US or any country with an extradition treaty which is most of the better places to live.
- jahewson 2y ago
  Extradition is only possible in cases where both countries have (approximately) equivalent laws and the crime is sufficiently serious.

ninjagoo 2y ago

This is worrisome for open source - this isn't some far off future limit. Llama3 training at 400 TFLOPS per GPU [1] and 6.4M GPU hours [2] puts Llama3-70B at 9.2*10^24 (so 10^25) floating point ops. Llama3-400B might be well past this threshold. [1] https://ai.meta.com/blog/meta-llama-3/ https://ai.meta.com/blog/meta-llama-3/ [2] https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct

simonw 2y ago

> Instead of regulating the development of AI models, the focus should be on regulating their applications, particularly those that pose high risks to public safety and security. Regulate the use of AI in high-risk areas such as healthcare, criminal justice, and critical infrastructure, where the potential for harm is greatest, would ensure accountability for harmful use, whilst allowing for the continued advancement of AI technology. I really like this proposed model. Are there good arguments against this?

antonioevans 2y ago

What about China and rest of the world. Their training, some are training open source models (Mistral). So the US is going to be left behind?

ericd 2y ago
Yeah, there's definitely no putting this back in the bag on a global scale.
deleted 2y ago
[deleted]

ltbarcly3 2y ago

It won't stifle anything. It will take about 2 months for all AI companies to abandon California. Forcing them out of SF will make their social lives worse but lower their costs 80%. The end result is that they will spend less time drinking $17 espresso, more time working, and will be able to hire more engineers. AI research accelerates dramatically.

sidcool 2y ago

"Note from Jeremy: This is my personal submission to the authors of bill SB-1047. It’s not an official Answer.AI statement." Then why post on answer.ai?

Nifty3929 2y ago

"This could inadvertently criminalize the activities of well-intentioned developers working on beneficial AI projects." It wouldn't be inadvertent. It's a control tactic. "Placing liability on the creators of general purpose tools like these mean that, in practice, such tools can not be created at all, except by big businesses with well funded legal teams." ... "These requirements could disproportionately impact open-source developers who often lack the resources of larger corporations to navigate complex regulatory processes." ... "The proposed regulations create significant barriers to entry for small businesses and startups looking to innovate in the AI space." That's the idea. The government likes a small number of big businesses that they can control.

jahewson 2y ago
How about some liability placed on members of government for their hopeless legislating? As developers of law shouldn’t they accept responsibility for the harms they cause?

deleted 2y ago

[deleted]

bgdhyfhy 2y ago

[dead]

hendrycks 2y ago

A few things that I’m seeing folks in the comments misunderstanding about the bill (full disclosure: I’ve been one of a group of folks advising Senator Wiener on SB 1047) 1. The new Frontier Model Division is focused on receiving information and issuing guidelines. It’s not a licensing regime and isn’t investigating developers. 2. Folks aren’t automatically liable if their highly capable model is used to do bad things, even catastrophic things. The question is whether they took reasonable measures to prevent that. This bill could have used strict liability, where developers would be liable for catastrophic harms regardless of fault, but that's not what the bill does. 3. The bill requires developers to test their models and report whether they have hazardous capabilities (and the answer can obviously be yes or no). Even if the model does have hazardous capabilities, the developer can still deploy it if they take reasonable precautions, as outlined in the bill. For perjury, you would need to intentionally lie—good faith errors would not be covered. I get that models can have unforeseen capabilities, but this isn’t about that. If you are knowingly releasing something that could have demonstrably catastrophic consequences, it seems fair to have consequences for that. Some things which already require folks to certify under penalty of perjury: lobbying disclosures, companies’ financial disclosures, immigration compliance forms. 4. Overall it seems pretty reasonable that if your model can cause catastrophic harms (which is not true of current models, but maybe true of future models), then you shouldn’t be releasing models in a way that can predictably allow folks to cause those catastrophic harms. If people want a writeup of what the bill does I recommend this one by the law firm DLA Piper (https://www.dlapiper.com/en/insights/publications/2024/02/californias-sb-1047 https://www.dlapiper.com/en/insights/publications/2024/02/ca...). In my opinion this is a pretty narrow proposal focused at the most severe risks (much more narrow than, e.g., the EU AI act).

blackeyeblitzar 2y ago

The entire notion of “safety” and “ethics” in AI is simply a Trojan horse for injecting government control and censorship over speech and expression. That’s what the governments get out of it. The big AI players like OpenAI, Microsoft, Amazon, Google, etc. are incentivized to go along with it because it helps them through regulatory capture and barriers to competition. They also make some friends with powerful legislators to avoid pesky things like antitrust scrutiny. Legislation should not restrict the development or operation of fundamental AI technologies. Instead laws should only be built on the specific uses that are deemed illegal, irrespective of AI.