5 ms·
On the Proposed California SB 1047
- sroussey 3y agoThis bill also basically bans AI in trading.
- option 3y agoHere we go, proposal to regulate technology (largest models) again. Hard No here. We should be regulating applications of AI instead (regardless of model type, size and compute).
- wolverine876 3y agoWhy? How will that yield better results for the public and for other stakeholders? How is that more fair?
- a_vanderbilt 3y agoAs is, the bill is not fair to small players. It encourages ventures to use existing models by big players to escape the regulatory burden of "safety", leading to a state-encouraged oligopoly. It also uses compute and benchmarks as the measuring stick for regulation, which can be easily gamed - and already is being gamed. Instead, the application plugged into the AI needs to be regulated. Does the application run afoul of the safety goals of the bill? Then restrict the application, not the model itself.
- Animats 3y agoYes. What needs to be regulated are systems that make decisions that affect people. The EU Data Protection Regulations already cover that, using the terms "automated system". It's not AI that's the problem. It's power over others that's the problem. The trouble is, regulating this means regulating the power of businesses to mess up people's lives. Worries about people generating porn with AI are overblown. That already works.[1] It's technically impressive, but kind of emotionless. Nobody seems to be worried about it. [1] https://www.reddit.com/r/sdnsfw/ https://www.reddit.com/r/sdnsfw/
- kelnos 3y agoAbsolutely agreed. But does AI itself not need some checks and balances on it? Reasonable or not, I think the big worry (especially among laypeople) is Terminator-style AI that wants to destroy humanity. Making a law that says AI can't be used to determine if people are allowed to take out loans doesn't stop that.
- Animats 3y ago> I think the big worry (especially among laypeople) is Terminator-style AI that wants to destroy humanity. Compared to Putin, Xi, Netanyahu, and Trump, AI is a minor problem.
- rendang 3y agoAI does not and cannot "want" anything. Throwing around these kinds of anthropomorphisms does not help clarify the discussion around this stuff.
- Animats 3y agoIf someone sets up a cost function for an AI, it wants something.
- viscanti 3y agoA lot of hand waiving about how "minor paperwork" for everyone isn't such a big deal. But very little coverage of what benefit that paperwork is accomplishing. This looks like a big bag of regulatory requirements, some which might be good and some which might be bad ideas. If it's the start of a discussion around what actually good regulations look like, then they have to start somewhere and maybe it will accomplish that. If it's an actual proposal and we have to take or leave everything, maybe they need to go back to the drawing board and figure out something better.
- wolverine876 3y agoThat's an interesting concept: There's the theory, that I tend to believe, that any successful complex system was born as a simple system, then evolved by iteration, etc. (it's from a 1970s systems theorist, iirc). Trying to birth a complex system directly is almost inevitably a mess. But does it apply to legislating? One drawback might be that political will and attention is fleeting; and interests become vested in a current state, blocking change; and it's now or not for another generation or so.
- nerdponx 3y agoIt's worth keeping in mind the well-known principle in economics that incumbent market participants, with large economies of scale, have a strong incentive to pursue regulations that increase fixed costs but don't substantially affect marginal costs, or are otherwise decreasing costs with respect to scale. This makes it harder to start a competing firm.
- wolverine876 3y agoThat is a risk, but regulation can also keep the market open to competition, preventing incumbants from abusing their market power. Antitrust law, for example, and the FTC do that.
- nerdponx 3y agoRight. But this is not a thread about the general merits of government regulation in business. This is about something specific, and this principle is essential to keep in mind when evaluating any specific new regulation.
- nvrmnd 3y agoThis and other proposed legislation is attempting to hit the ball out of the park on the first pitch. I feel it would be a lot more sensible and effective to legislate clear and present harms, such as holding developing firms liable for deep-fake technology if used for identity theft for the purpose of fraud.
- ApolloFortyNine 3y agoShould I be able to sue Honda because someone in their Civic ran into me? A users misuse of a technology shouldn't be the responsibility of the developer. You could apply this to almost every product in the world otherwise.
- thinkingtoilet 3y agoIf it was found they were reckless, absolutely. I believe this is already the case.
- ronsor 3y agoI don't. Software developers and researchers should not be liable for distributing information or code, even if it's used for something illegal, as long as they aren't explicitly promoting the illegal activity and don't have any involvement with it outside of creating the software. Not only is that consistent with previous decisions, such as those regarding copyright (i.e. torrents are fine, but making a client to torrent movies specifically isn't), but also any other decision would be a violation of the social contract with regard to open-source development.
- theamk 3y agothe original comment explicitly said "holding developing firms ..." - so this is not about software developers, it is about corporations. The moment you start to sell stuff is the moment you become liable.
- 3y ago
- SV_BubbleTime 3y ago“pro-safety” Is this a dog whistle for them too, or just for us?
- TheBlight 3y agoWon't AI companies just pack up and move?
- duxup 3y agoDo they want to do business in CA? I think so.
- wolverine876 3y agoBusinesses threaten this, but if they really want no regulation there are countries that have none at all. It turns out that a well-regulated market is a better place to do business. It also attracts smart, talented people who want to live in the best living environment - schools, roads, culture; a place filled with people who want to build a great community - not the best environment for the billionaire founder who cares only for themself and their estate. Much more wealth is generated, many more industries have been born and thrived, in places like CA and NY than in less-regulated, lower-wage, no/low-tax states. The correlation looks very strong by the eyeball method.
- sroussey 3y agoYet very few companies incorporate in California. Also, LA put in a tax regime where all the studios left the city for neighbors like Burbank and Culver City.
- wolverine876 3y ago> very few companies incorporate in California They incorporate in Deleware, where they don't locate. That's irrelevant. California's economy and business is incredible, historic.
- kelnos 3y agoIncorporation location is irrelevant to this discussion. While governance of a Delaware corporation is litigated in Delaware business courts, a Delaware corporation that is headquartered in California, or even does business in California, needs to obey California law. And there are quite a few of them, to put it mildly.
- photochemsyn 3y agoWhat's the problem with this, it seems like the best idea in this bill: > "Section 5 11547.7 is for the CalCompute public cloud computing cluster. This seems like a terrible idea, there is no reason for public involvement here, also there is no stated or allocated budget. Assuming it is small, it does not much matter." There are many solid arguments for state funding of open-source programming efforts, the only thing to look out for is that this effort doesn't follow the University of California model of 'public private partnerships' in which taxpayer funds are used to generate IP which is then exclusively licensed to private interests.
- jacobn 3y agoThis is a “sounds good, but devil is in the details” type situation, and the details typically don’t work out all that well because the initiative effectively assumes good intent from the ecosystem, when we have seen time and time again that there are enough hostile actors with enough creativity to spoil it.
- Gormo 3y ago> There are many solid arguments for state funding of open-source programming efforts Perhaps there are solid arguments for that if your goal is to put open-source projects under the de facto control of political institutions. But for those of us who prefer for FOSS to continue on in a decentralized, mostly autonomous fashion, rather than have it be overwhelmed by rent-seekers looking for state subsidies, or be used as a pawn to advance various factions' political agendas, those arguments don't seem quite so solid.
- asadotzler 3y agoNASA is a FOSS contributor. Got a problem with that?
- Gormo 3y agoNot hugely, since NASA is an organization with a very narrowly tailored scope, and it's participating in FOSS projects as a means to the end of facilitating its space-exploration mission. FOSS is incidental to NASA's purpose;: they use and modify code, and contribute their work back upstream, but aren't engaged in software development for public consumption as a primary activity. OTOH, I'd be extremely wary of a political organization founded for the specific purpose of funding/influencing the overall FOSS ecosystem.
- sroussey 3y agoJust take this bill and replace AI with “Database” and have the date be 1980. Oracle will need KYC of customers to determine if you are good and safe enough. And make sure there are no future competitors to themselves (and maybe their customers as well!).
- justrealist 3y agoSo the thing is, nobody was worried about databases. And the people who were not worried about databases, are worried about AI. It's like saying, "Oh, you're regulating biological weapons? Funny, you could say the same thing about bows and arrows!" Not all things are actually the same! At some point, there is a point where allegory breaks down and one thing can be VERY BAD where other things are FINE. You have to look at the actual contents of the thing, and a lot of very smart people think that AI is *not like the other things*. Please engage at that level!
- JumpCrisscross 3y ago> nobody was worried about databases. And the people who were not worried about databases, are worried about AI Also, Larry Ellison didn’t market Oracle as world-ending technology. Altman et al have.
- devmor 3y agoThat’s the part that drives me insane re people’s reactions to this kind of legislation. You have people saying “our technology could end the human race, please let us continue to do whatever we want with it without oversight”. Regardless of whether you believe them or not, the statement itself should be enough to provoke oversight.
- sroussey 3y agoAs if the world doesn’t run on databases. And now the government want to be able to have them shutdown. People were definitely worried about databases! All the jobs lot for people that did filing! Copying! Typing! It’s easy to look back with hindsight and so, eh no one worried. Heck, there were strikes when someone first played a record on the radio! God forbid! BTW: a lot of people consider what they work on being so much more important to the world, and thus they are more important.
- gnarbarian 3y agoKeep the government out of my GPU!
- nine_k 3y agoProbably customers from California will have more onerous requirements for them, compared to the rest of he US.
- bugglebeetle 3y ago> This seems like a terrible idea, there is no reason for public involvement here, also there is no stated or allocated budget. While I dunno where I come down yet on this legislation, it seems truly bizarre to object to the CalCompute cluster. IMO, the biggest AI threat we face is from big tech being the only ones capable of creating new models. Right now, companies like Meta are aligned with open source AI, but that could change at the drop of the hat. It makes sense to preserve state capacity to level the playing field against attempts at monopoly or other forms of overreach. Also, a lot of great work is coming out of UC system labs, so why shouldn’t we invest more in making them successful?
- sroussey 3y agoI wonder if we would have smaller llama models if compute was cheap.
- rsync 3y ago"While I dunno where I come down yet on this legislation, it seems truly bizarre to object to the CalCompute cluster." The calcompute cluster is what jumped out at me. The danger is that this will be a huge subsidy to entrenched UC/CA contractors that are in place solely to navigate - and exclude others from - state procurement mechanisms. In fact, if you were really cynical you might think that the entire purpose of this bill is the calcompute cluster and everything else is just scaffolding to enable a boondoggle. I am not asserting this - I am just pointing to a possible interpretation.
- bugglebeetle 3y agoCan you elaborate? I’m not sure I follow.
- sgslo 3y agoWho's going to build the cluster? Who's going to operate it? Who's going to decide how time/access gets allocated? Are results of compute going to be in the public domain? Given the glacial speed of government products, will the cluster be using obsolete tech by the time its finally up and running?
- Williams77 3y ago[flagged]
- ApolloFortyNine 3y agoThe Author's entire section "What is a Covered Model Here?" kind of sums up one of the main issues with the bill, even if they say themselves how clear the bill is. ''' (1) The artificial intelligence model was trained using a quantity of computing power greater than 10^26 integer or floating-point operations in 2024, or a model that could reasonably be expected to have similar performance on benchmarks commonly used to quantify the performance of state-of-the-art foundation models, as determined by industry best practices and relevant standard setting organizations. (2) The artificial intelligence model has capability below the relevant threshold on a specific benchmark but is of otherwise similar general capability. ''' They then go on to say: >Under this definition, if no one was actively gaming benchmarks, at most three existing models would plausibly qualify for this definition: GPT-4, Gemini Ultra and Claude. I am not even sure about Claude. Well how did they draw this conclusion? They decided that other models that do well are simply gaming benchmarks, and therefore can ignore the provision? >Um, no, because the open model weights models do not remotely reach the performance level of OpenAI? To me the author is simply incredibly biased and not well versed in other models. Is chatgpt4 better? Yes. Is Mistral8 comparable? I'd easily say yes. Using it just once would lead most to a similar conclusion, it's only after throwing word problems and tricks where you can really see the differences. And these kind of laws open to interpretation are always an issue because a responsible company has to err on the side of caution. If your model is often helpful, but only 80% as often as chatgpt, is that comparable? If someone released a new benchmark tomorrow, and your model actually beats out chatgpt, does your model all of a sudden apply?
- Imnimo 3y agoI'm not even sure the covered model description only applies to LLMs. Is Segment Anything a covered model? The paper self-describes as a "foundation" model, and it presents SoTA results on various benchmarks. It surely falls under subsection (b)'s incredibly broad definition of an AI model, since it "makes predictions". It sounds like you can't make a "positive safety determination" if you're trying to get SoTA performance on any benchmark, so is this bill asking Meta to implement a procedure to enact a "full shutdown" of image segmentation models? It feels like this section of the bill was written by someone who once read a LinkedIn article about AI, but isn't really sure what all the terms mean.
- wrs 3y ago>Allowing social media to grow unchecked without first understanding the risks has had disastrous consequences… Not gonna argue with that, but I’m having trouble imagining the alternative. Somehow we would have understood the risks without allowing the growth and observing the consequences? That seems unlikely, considering how surprised we all seemed when the consequences occurred. With AI we seem to have a lot more noise around imagining consequences, but in my mind there’s no reason that would correlate with completeness or accuracy of the predictions. There will be lots of very bad, and lots of very good, consequences that nobody can currently imagine.
- bluGill 3y agoWhat is important is what we can imagine today. What we discover or imagine tomorrow needs to be handled in tomorrows updates, but isn't a problem today. The courts require you to do due diligence in imagining and mitigating the problems we can foresee today. If we invest something next year you will have to redo that for products you make next year - but you don't have to retrofit that to what you did this year. Note that I said make not sell.
- kevingadd 3y agoI would argue that a more robust regulatory framework around social media could have at least prevented the worst harms. For example, more robust moderation or at least requiring companies to have responsive liaisons who speak the local language for territories they operate in could have mitigated/prevented the serious harms we've seen Facebook and Whatsapp cause in multiple countries i.e. Myanmar. This is of course assuming we'd get the kind of regulations we need, instead of useless security theater or regulations so onerous that they prevent products from developing at all. But it does seem like it would have been possible to prevent the worst of it, since many of these cases boil down to 'Facebook (or other big company) didn't care and didn't bother to try, and then bad things happened'
- corethree 3y agoJust thought I reference this: 1. A robot may not injure a human being or, through inaction, allow a human being to come to harm. 2. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. 3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law. Is SB1047 the first instance of AI regulation?
- lmeyerov 3y agoAll these seem historically bizarre, and making me have deep suspicion about the people & processes driving this legislation. I expected California to legislate early, but not like this. 1. What I expected: * AI Indemnification rules for platform providers building on ideas like net neutrality on one end, and content provider protections like DMCA and copyright on the other * Consumer AI protection rules building on ideas like the Equal Credit Opportunity Act and GPDR 2. What we got: Something about KYC and rules on cases about $500M+ incidents and deaths
- TaylorAlexander 3y agoThe author believes the CalCompute model is a terrible idea. I strongly disagree! A lack of access to appropriate infrastructure (due to cost) is one of the things that smaller researchers talk about a lot. It leads to a situation where the only types of people training large models are the types of people at large institutions. Giving more researchers an opportunity to train large models could bring more variety of thought in our exploration of AI safety and general development. As an aside, I called my local library and a real person immediately answered the phone. Public infrastructure can have unexpected benefits over commercial offerings, and I’m all for it.
- burningChrome 3y ago“If Congress at some point is able to pass a strong pro-innovation, pro-safety AI law. . . . " With AI this seems like an untenable position. We already have companies that are pushing the bounds with AI constantly and ever further, while telling us we shouldn't worry about the future dystopian implications. Trying to reign these people and companies in with government regulation at this point doesn't seem feasible.
- rKarpinski 3y ago> With AI, we have the opportunity to apply the hard lessons learned over the past two decades. Allowing social media to grow unchecked without first understanding the risks has had disastrous consequences, and we should take reasonable precautions this time around. > Critical harm is either mass casualties or 500 million in damage, or comparable. So the whole regulatory premise is based on missing the boat on social media, but how many examples are there of social media causing this kind of "critical harm"?
- kevingadd 3y agohttps://www.amnesty.org/en/latest/news/2022/09/myanmar-facebooks-systems-promoted-violence-against-rohingya-meta-owes-reparations-new-report/ https://www.amnesty.org/en/latest/news/2022/09/myanmar-faceb... there are more, but this one came to mind
- serf 3y agoscott wiener has never endorsed anything that expands personal or civil liberties in any possible way. when I read his name having to do with any law my brain just automatically requires double the amount of persuasion that it'll do any social good whatsoever.
- devmor 3y agoI find it quite disgusting how this thinkpiece is a thinly veiled advertisement to gamble on the writer’s market position.
- whalesalad 3y agoWayyyyyyy too early to be putting regulatory frameworks on this stuff. Huge waste of resources.
- siliconc0w 3y agoId pierce the corporate veil and make executives and board members personally responsible for the damages of their models. Train whatever but if you host a model and that model's output results in significant damage being done, you have to own it