5 ms·
The bill included language that required the creators of models to have various "safety" features that would severely restrict their development. It required a
by CSMastermind 2y ago
The bill included language that required the creators of models to have various "safety" features that would severely restrict their development. It required audits and other regulatory hurdles to build the models at all.
- llamaimperative 2y agoIf you spent $100MM+ on training.
- gdiamos 2y agoAdvanced technology will drop the cost of training. The flop targets in that bill would be like saying “640KB of memory is all we will ever need” and outlawing anything more. Imagine what other countries would have done to us if we allowed a monopoly like that on memory in 1980.
- llamaimperative 2y agoNo, there are two thresholds and BOTH must be met. One of those is $100MM in training costs. The other is measured in FLOPs but is already larger than GPT-4, so the “think of the small guys!” argument doesn’t make much sense.
- gdiamos 2y agoTell that to me when we get to llama 15
- llamaimperative 2y agoWhat?
- gdiamos 2y ago“But the big guys are struggling getting past 100KB, so ‘think of the small guys’ doesn’t make sense when the limit is 640KB.” How do people on a computer technology forum ignore the 10,000x improvement in computers over 30 years due to advances in computer technology? I could understand why politicians don’t get it. I should think that computer systems companies would be up in arms over SB 1047 in the same way they would be if the government was thinking of putting a cap on hard drives bigger than 1 TB. It puts a cap on flops. Isn’t the biggest company in the world in the business of selling flops?
- llamaimperative 2y agoIt would be crazy if the bill had a built-in mechanism to regularly reassess both the cost and FLOP thresholds… which it does. Inversely to your sarcastic “understanding” about politicians’ stupidity, I can’t understand how tech people seem incapable or unwilling to actually read the legislation they have such strong opinions about.
- gdiamos 2y ago[flagged]
- lucubratory 2y agoIt's troubling that you are saying things about the bill which are false, and then speculating on the motives of someone just pointing out that what you are saying is false.
- gdiamos 2y agowhy not tell us? Or point out what is actually false? You rebutted the points that the flop limit will not actually limit anyone by saying that gpt4 is out of reach of startups. Is OpenAI a startup? Is Anthropic? Is Grok? Is Perplexity? Is SSI? You ignored the counter points that advanced technology exponentially raises flop limits and changes costs. You said that the flop limit can be raised over time. So startups shouldn’t worry. You ignored the counter point that flop limits in export controls are explicitly designed to limit competition from other nations. Flop limits not being a real limit is a ridiculous argument. The intent of a flop limit is to limit, no matter how you sugar coat it.
- gdiamos 2y agoCost as a perf metric is meaningless and the history of computer benchmarks has repeatedly proven this point. There is a reason why we report time (speedup) in spec instead of $$ The price you pay depends on who you are and who is giving it to you.
- llamaimperative 2y agoThat’s why there are two thresholds.
- Vetch 2y agoCost per FLOP continues to drop on an exponential trend (and what bit flops do we mean?). Leaving aside more effective training methodologies and how that muddies everything by allowing superior to GPT4 perf using less training flops, it also means one of the thresholds soon will not make sense. With the other threshold, it creates a disincentive for models like llama-405B+, in effect enshrining an even wider gap between open and closed.
- pas 2y agoWhy? Llama is not generated by some guy in a shed. And even if it were, if said guy has such amount of compute, then it's time to use some of it to describe the model's safety profile. If it makes sense for Meta to release models, it would have made sense even with the requirement. (After all the whole point of the proposed regulation is to get some better sense of those closed models.)
- llamaimperative 2y agoAlso the bill was amended NOT to extend liability to derivative models that the training company doesn’t have effective control over.
- llamaimperative 2y agoBoth thresholds have a system to be adjusted.
- theptip 2y agoIf the danger is coming from the amount of compute invested, then cost of compute is irrelevant. A much better objection to static FLOP thresholds is that as data quality and algorithms increase, you can do a lot more with fewer FLOPs / parameters. But let’s be clear about these objections - they are saying that FLOP thresholds are going to miss some harms, not that they are too strict. The rest is arguing about exactly where the FLOP thresholds should be. (And of course these limits can be revised as we learn more.)
- pj_mukh 2y agoOr used a model someone open sourced after spending $100M+ on its training? Like if I’m a startup reliant on open-source models I realize I don’t need liability and extra safety precautions but I didn’t hear any guarantees that this wouldn't turn off Meta from releasing their models to me if my business was in California? I never heard any clarifications from the Pro groups about that
- deleted 2y ago[deleted]
- llamaimperative 2y agoThe bill was amended for training companies to have no liability for derivative models they don’t have control over. There’s no new disincentive to open sourcing models produced by this bill, AFAICT.
- wslh 2y agoAll that means that the barriers for entry for startups skyrocket.
- SonOfLilit 2y agoStartups that spend >$100mm on one training run...
- wslh 2y agoThere are startups and startups, the ones that you read on media are just a fraction of the worldwide reality.
- SonOfLilit 2y agoYou misread my intention, I think. I was pointing out that startups that don't raise enough to train a GPT-4 are excluded from the bill, that requires >$100mm expenses for a model to be covered.
- wslh 2y agoFair enough. Hope all is well there.
- SonOfLilit 2y agoNo offence taken, just a misunderstanding. I didn't downvote you.