15 ms·
Banning open weight models would be a disaster
- CamperBob2 3y agoFrom the end of the article: "If you agree, please send your comments to the DoC by March 27th, 2024." Is there a reason why this executive order / RFC received no coverage on HN (or anywhere else I'm aware of) until after the deadline had passed?
- mcpherrinm 3y agoHN discussion was @ https://news.ycombinator.com/item?id=38067314 https://news.ycombinator.com/item?id=38067314
- lolinder 3y agoI remember that discussion about the original order, but I'm in the same boat as OP—this is the first I'm hearing about an RFC about open weights. It looks like it was discussed once, about a month ago [0], but I hadn't seen any of the public comments show up on HN until today, even though they date back weeks. [0] NTIA Solicits Comments on Open-Weight AI Models (57 points, 11 comments) https://news.ycombinator.com/item?id=39494760 https://news.ycombinator.com/item?id=39494760
- CamperBob2 3y agoExactly -- I remember the 'Executive Order on ...' headline now, but I never bothered to click on it, because why would I. The second story, I never noticed at all, and neither did anybody else. One of the few cases where a bit more alarmism in the headline might not have been out of place.
- spiesd 3y agoA few come to mind... The executive order (and associated DoC/NTIA RFC) was long, and dense with both legal references and political platitudes. Not great reading, (though the EO got a good amount of discussion here). It's unfortunate, but less than surprising that it didn't make actual news outlets. It seems complicated now, and many would like to see how things play out a little more before committing the time to deciding things. (I think that's a bad idea; regardless of the revolutionary tech, it seems wise to begin governmental thought early.) It's kinda been buried in an otherwise heavy news cycle since the end of October of last year, AI and otherwise. I assume that wasn't intentional, but it seems hard to hide something so big at a better time. And though I hate to say it, I suspect: apathy. Both traditional from the bottom ("too far off, unfixable, can't do anything about it"), and from the top ("once we get too big to fuck with, we just won't give a damn about any changes they try to make anyway").
- greenavocado 3y agoA careless or reckless president might overly depend on advisers for drafting executive orders, sidelining personal oversight. This delegation risks orders that may not align with the president's intent or could lead to adverse outcomes due to unchecked biases or agendas among advisers. Without the president's close review, policies might lack comprehensive vetting, inviting legal issues, public disapproval, or impractical implementations. Excessive reliance on advisers could also push more extreme policies under reduced scrutiny. Effective governance requires the president's informed engagement to ensure executive orders reflect their vision and serve the nation's best interest. Instead we have someone who offers his "concerns" to the public while shipping billions worth of dollars of explosives to the enemies of humanity. Don't expect any concerns or input from the public to make any difference.
- foota 3y agoIs there any way to comment past the date? This seems like a horrible idea.
- thelittleone 3y agoBeyond horrible. Beyond even dystopian films. Look at the evil mega corps and governments have shown with regards to privacy and freedoms. They've been eating away at it for years, one bite at a time, not too much to cause a revolt. Are we to depend upon the good intentions of mega corp owners and politicians to wield AGI exclusively and for the benefit of society when that same AGI will render any form of public organized protest impossible.
- supriyo-biswas 3y agoYou can write to them anyway and also send a copy of the leyter to your representatives. It still sends the message, and if many people send such letters, it will force them to reconsider.
- greenavocado 3y agoOpen model weight bans will likely be struck down as First Amendment violations because, at their core, model weights are a form of expression. They embody the ideas, research, and innovations of their creators, similar to how code was deemed a form of speech protected under the First Amendment during the encryption debates of the 1990s. Just as the government's attempts to control encryption software were challenged and largely curtailed due to free speech concerns, any attempts to ban open model weights will face legal challenges arguing that such bans unjustly restrict the free exchange of ideas and information, a cornerstone of First Amendment protections. The precedent set by cases involving code and free speech strongly suggests that similar principles apply to model weights, making such bans vulnerable to being overturned on constitutional grounds.
- vintermann 3y agoThat's up to the whims of the US Supreme Court. They're in the process of legalizing their own corruption right now. I don't think you should take for granted that just because you have a solid and obvious argument, your rights will be protected by them.
- greenavocado 3y agoRemember when the world freaked out over encryption, thinking every coded message was a digital skeleton key to anarchy? Yeah, the 90s were wild with the whole PGP (Pretty Good Privacy) encryption fight. The government basically treated encryption like it was some kind of wizardry that only "good guys" should have. Fast forward to today, and it's like we're stuck on repeat with open model weights. Just like code was the battleground back then, open model weights are the new frontier. Think about it—code is just a bunch of instructions, right? Well, model weights are pretty much the same; they're the brains behind AI, telling it how to think and learn. Saying "nah, you can't share those" is like trying to put a genie back in its bottle after it's shown you it can grant wishes. The whole deal with PGP was about privacy, sending messages without worrying about prying eyes. Fast forward, and model weights are about sharing knowledge, making AI smarter and more accessible. Blocking that flow of information? It's like telling scientists they can't share their research because someone, somewhere, might do something bad with it. Code lets us communicate with machines, model weights let machines learn from us. Both are about building and sharing knowledge. When the government tried to control encryption, it wasn't just about keeping secrets; it was about who gets to have a voice and who gets to listen. With open model weights, we're talking about who gets to learn and who gets to teach. Banning or restricting access to model weights feels eerily similar to those encryption wars. It's a move that says, "We're not sure we trust you with this power." But just like with code, the answer isn't locking it away. It's about education, responsible use, and embracing the potential for good. Innovation thrives on openness. Whether it's the lines of code that secure our digital lives or the model weights that could revolutionize AI, putting up walls only slows us down. We've been down this road before. Let's not make the same mistake of thinking we can control innovation by restricting access.
- jj999 3y agoThe fight against encryption continue to this day and while https is now ubiquitous, large-scale cdns makes it somewhat a moot point and emails are still largely plaintext.
- solatic 3y ago> emails are still largely plaintext But people's private digital communications have largely moved to platforms like WhatsApp and Messenger which enjoy end-to-end encryption. Email, at least between major providers, today enjoys TLS over the wire while being sent. I'm sure there are various flaws and weaknesses and maybe even backdoors, but trying to make it sound like we lost the fight for encryption because emails are in plaintext is rather disingenuous.
- patcon 3y agoI'm open to this. If an attempt to stop frontier models is to hold, banning open weights must be on the table. I'm excited about AI, but not entirely sure that it doesn't call for the same strong regulation as nuclear proliferation did. It's about kicking the brakes until culture catches up and absorbs impacts, not stopping I'm just saying I'm open to it, and don't want them to listen to accelerationists, but rather the ppl doing the deepest work with the edge models. Many of them are humbled and worried. Generalist AI enthusiasts wanting freedom to do any and all things with paradigm-shifting intelligence infra, I'm not swayed by that so much.
- aerhardt 3y agoIf LLMs warrant “the same strong regulation as nuclear proliferation”, but this is all about “kicking the brakes until culture catches up”, should we also now loosen nuclear arms regulations, now that society largely knows what’s up? The more I use LLMs the more I’m convinced they’re harmless… The world’s not going to end because people at the margins are asking LLMs for recipes to manufacture cocaine… or a nuclear bomb.
- Ekaros 3y agoI think LLMs are not a great danger. Ignorance is their use probably is, but that really makes no difference between some random model available that does answer and home trained one. The fault is on user. And nothing can stop people from being ignorant, lazy or just plain stupid.
- Nevermark 3y agoThere is an argument that banning things for law abiders essentially means giving criminals preferential access. I don’t know that that idea always holds. But it definitely would for AI All you need is ordinary commodity computing power to keep moving Ai forward. So vast numbers of individuals, organizations, and governmental actors will. Visibly or invisibly Any blunt reactionary prohibition will only tilt things in favor of an unconstrained underground
- 3y ago
- mikkom 3y agoWow what a horrible idea. Sounds like monopoly in the making. I would be really interested to hear what "open" AI has commented about this - I guess they are lobbying for this with all of their billions. If the US government really want to do this correctly, they must also ban any API access to AI models and ban all research related to AI. How could this be even written as law? Universities and companies are probihited to publish their research? How many layer models are forbidden to be published? All neural networks?
- DarkNova6 3y agoIt's clear to me that Sam Altman has turned OpanAI into ClosedAI
- iLoveOncall 3y agoNot just "ClosedAI" but "CloseAllAI". He's been actively lobbying congress and others to ensure OpenAI are the only ones, with a few other big actors, that can make AI.
- graemep 3y agoWhich is why so many tech companies are telling us how dangerous AI is.
- smeagull 3y agoIt's not a monopoly. It's America opting out of open research.
- baq 3y agoMay I remind you of https://en.wikipedia.org/wiki/DeCSS https://en.wikipedia.org/wiki/DeCSS? If that isn't enough, it links to https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes https://en.wikipedia.org/wiki/Illegal_number#Illegal_primes which is well into ridiculous but true territory.
- rgmerk 3y agoNot saying this ban is a great idea... ...but there are plenty of useful things that we ban the general public from having access to. Opium poppies. Automatic weapons. Gas centrifuges. In fact, if you start publishing detailed designs for the last one you're likely to have people in suits who only go by first names visiting you in pretty short order.
- zzzzzzzzzz10 3y agoIn what way are automatic weapons generally useful?
- beau_g 3y agoMostly suppressive fire, but more generally, providing a steady stream of small projectiles
- moomoo11 3y agoSo you can type this comment with complete safety while being protected by tax dollars
- rgmerk 3y agoIf you're fighting a war, automatic weapons are a pretty much essential tool for doing so effectively. Yeah, war is bad. Sometimes the alternatives are worse.
- CamperBob2 3y agoAsk a Ukrainian.
- mirekrusin 3y agoEquivalent would be to ban not "opium poppies" but "chemistry", not "automatic weapons" but "automatic machines" etc. It's more like trying to ban web or emails in one country because somebody can create harmful website or send harmful emails. Sooner or later we will have to look at information through AI lenses. Before seeing anything it'll go through filtering and transformation to weed out spam, misinformation and do fact checks etc. You can't build this defense future with crippled law.
- zzzzzzzzzz10 3y agoPlease ban them, it wont change a thing but drive us underground. I have backups of all relevant models as have many others.
- gorbypark 3y agoThe problem would be that orgs like Meta would stop publishing llama 3/4/5/etc, which most open source models build upon. Without new foundational models, progress would stall pretty quickly, and procuring thousands of GPUs to train new foundational models would be difficult. In theory, since the US “controls” Nvidia/amd/tsmc, they could put up roadblocks to even doing open training outside of the US. Maybe a “SETI@Home” style distributed training system could be done on consumer GPUs…
- Chris2048 3y ago> since the US “controls” Nvidia/amd/tsmc They don't control China or Europe, and will hand their overseas monopoly to overseas competition.
- gorbypark 3y agoChina is the only place that conceivably in the near future would be able to spin up their own designs and fabs for GPUs. In fact they seem to be well on their way, thanks to the push from the GPU export restrictions already in place!
- Manabu-eo 3y agoTraining needs lots of bandwidth, at least as done today. Something easier to distribute and do in a small scale is dataset creation.
- injidup 3y ago> The biggest threats posed by AI come not from individuals, but from corporations and state-level actors. Why is this statement assumed to be true? It is far from clear that advanced weapons in the hands of irresponsible, impulsive and ideological individuals cannot cause large scale chaos. To build an egg requires the effort on the level of states or corporations but to break it requires just an individual motivated to do so.
- mirekrusin 3y agoIt works both ways with open models where _defence_ also advances in hands of majority. With closed models and research only well funded entities participate (state actors and large corps). If you want to draw parallels with advanced weapons, the difference is that weapon is also an antidote to itself.
- FeepingCreature 3y agoI don't think open models offer any defence advantage whatsoever. Compare nukes. You can't stop a bad nuke with a good nuke. If I give you a nuke, the probability that you get nuked only goes up. Afaict, nobody has even offered any explanation how open weights do anything to defend you from bad actors with LLMs.
- mirekrusin 3y agoCompare potatoes. You can stop potateo with potateo.
- CamperBob2 3y agoYou can't stop a bad nuke with a good nuke. So far, that's exactly what we've done. MAD works. Ask the Ukrainians -- or what's left of them -- what happens when you voluntarily disarm.
- injidup 3y agoAlmost free availability of weapons in the USA is a case in point that this assumption is not true. There maybe good reasons to have open models but pretending that systems that allow an individual to project power beyond their immediate vicinity is harmless is naive. An individual with a weapon can almost always cause carnage at a level greater than their own with before any first responder can deal with it. Be that a viral video of someone faked to be blaspheming or somebody with a weapon in a school. Once the shooting starts it takes phenomenal energy to put the problem back in the box.
- torginus 3y agoSo AI companies take all the world's text and knowledge for free, use openly available research, take massive private funding and generate immense economic value using that, and want to make it illegal for anyone else to do the same? I don't get the 'harm can be done by individuals' argument. Sticks and stones. Every discussion forum on the internet is moderated to some degree, and every human being has the ability to post hurtful or illegal content, yet the system works. Moderation will only get more powerful thanks to AI tools.
- dang 3y agoRelated ongoing thread: OpenAI's comment to the NTIA on open model weights - https://news.ycombinator.com/item?id=39900197 https://news.ycombinator.com/item?id=39900197 - April 2024 (41 comments)
- MrYellowP 3y ago> to prevent abuse. People never grow tired of the ever-repeating excuses to further reduce our freedoms, or to hide criminality, or just insult the already lacking collective IQ. Think of the children! We need to prevent abuse! Someone's feelings might be hurt! Someone might get harmed! Seeing real breasts might cause trauma! Or one that's not so often used, but still really effective: We're just bulldozering this place, because it's so horrible. Instead we turn it into a luxury resort for rich people. People who believe that it's because we're destroy any and all evidence are just conspiracy theorists. (in regards to Epstein's Island)
- piokoch 3y ago[flagged]
- skissane 3y agoThere are almost 200 countries in the world. Even if the US and the EU and a bunch more ban open weight models, I doubt they'll succeed in convincing every country to do so. And whichever countries decide not to follow the ban, could thereby give their own AI industries a big boost. As the world becomes ever more globalised, the potential effectiveness of these kinds of policies declines. Sure, they could try to negotiate some kind of UN convention for a coordinated global ban. But, given how fractured global diplomacy has become, I doubt the odds of something like that succeeding are particularly high.
- antupis 3y agoMoney, talent, and institutions still cluster around the US and EU where the best universities are so this would hurt seriously AI development.
- skissane 3y agoI'm sure it would cause some harm, but exactly how much will depend on the details. For example, suppose the US makes it illegal to publicly distribute open weight models (maybe above a certain size) without a government license. What happens if someone sets up a website to distribute those models outside of US jurisdiction? It may well prosecute Americans it can demonstrate have uploaded models to that website, but will it also ban Americans from downloading from it? If you go back to the encryption export controls, the US banned encryption software from being exported from the US, but (unlike some other countries like France) it never tried to ban encryption software imports. And, it sounds like they are only thinking about banning public/"open" distribution, so once an institution downloads a model from outside the US, they may well be able to redistribute it internally without the ban applying. Suppose the US bans distribution of open weight models above size N–how will that apply to fine-tunes? If you have LoRA on top of a model >N, but the LoRA itself is <N, does the ban apply to it? If the answer is "no", people in the US could still contribute to openly distributing improvements to models that they couldn't themselves openly distribute.
- perihelions 3y agoOr it would seriously hurt US and EU universities. Is Western dominance of scientific academics automatic? Did we acquire it for free—no; it's the end-product of centuries of incremental cultural development, its openness and freedoms being core to that. Imagine Google-founding era, except it was illegal for 1990's Stanford to research CS theory without a license, so they didn't. And later someone else founded it anyway, in a different country (which now became the world's preeminent tech center). Let's put to a halt to these autoimmune attacks on our own civilization. This confusion that labels Western openness as a hazardous external substance.
- entrep 3y agoThe author is a collaborator in OpenDevin [1], an attempt to replicate and improve Cognition Labs' Devin. [1] https://github.com/OpenDevin https://github.com/OpenDevin
- deleted 3y ago[deleted]
- fullspectrumdev 3y agoI’m personally of the view that most AI safety people are delusional and irrationally terrified of progress. I remain open to having my mind changed on this matter, but thus far, I’ve not seen a single good argument for restricting development of AI.
- kypro 3y agoAI centralises power, devalues human labour and at its limit is fundamentally uncontrollable. Personally I struggle to understand anyone who believes it's in humanities best interest to continue developing AI systems without significant limitations on the rate of progress. There's so many ways AI could go wrong I'd argue it's almost guaranteed that something will go wrong if we continue on this trajectory.
- Nasrudith 3y agoLiterally any productive feedback loop where more resources produce better results centralizes power, any form of productivity improvement devalues human labor. Control is better described as a delusion in the first place.
- PoignardAzur 3y agoI mean... what you're communicating right now is "I have built a strawman in my head which I thoroughly hate, can anyone please come defend it before me"? I guess I'll try anyway. What's your opinion on the orthogonality thesis, ie intelligence not being fundamentally correlated with altruism? The concept of mesa-optimization, that a process that tries to teach a system to optimize for X might instead instead make it optimize for thing-that-leads-to-X (eg evolution made us want sex because it leads to reproduction, but now we have a lot of sex with birth control). The concept of instrumental convergence, that any optimization process powerful enough will eventually optimize for "develop agency, survive longer, accumulate more power"? There's a lot of literature written about AI safety, a lot of it from very skeptical people, but it's not like there's any single insight that can be reliably transmitted to a guy at a table with a "you're all idiots, change my mind" sign.
- bradley13 3y agoGovernment is necessary, in order to organize a complex society, but government is like any other organization: made up of people, many of who are out for their own interests. The most prominent of those interests are power and money. Whenever government proposes banning a technology, one must ask: who benefits? The LLMs, even in their current state of infancy, are powerful tools. Restricting access to those tools keeps power in the hands of wealthy corporations and the government itself. That's the power aspect. The money aspect is even simpler: Don't doubt that some of those wealthy corporations are making donations to certain officials, in order to gain support for actions like this. Almost no one leaves the Congress (or almost any parliamentary body in any country) as less than a multi-millionaire. Funny, how that works...
- NumberWangMan 3y agoSure, but you still have to balance your "who benefits" against the harm of the thing they're proposing to ban. In the case of these models, you can fine-tune the model all you want to be moral and not do harmful things like scam the elderly, carry out disinformation campaigns, harass people to the point of suicide. But as soon as you release the model weights, you are giving anyone the ability to fine-tune out all of those restrictions, with orders of magnitude less cost that it took to develop the model in the first place. Regulating AI, especially as it becomes AGI and beyond, is going to be very tricky, and if everyone has the ability to create their own un-restricted, potentially sociopathic intelligences by tweaking the safe models created under careful conditions by big labs, we're in for a lot of trouble. That assumes we put the proper regulations on the big labs, and that they have the ability to make them "safe", which is hard, yes. But as AI turns into AGI and beyond, things are going to go pretty nuts, so it's important to start laying groundwork now.
- wanderingmind 3y agoMaybe some lawyer here can explain how can an administration block publishing of open weights without violating first amendment which guarantees freedom of expression for everyone
- Buttons840 3y agoAnyone know how to make a USB drive that is also a shirt?
- barryrandall 3y agoI think what you're looking for is a "book."
- guappa 3y agoTry publishing gay romance for teens and see how protected speech really is.
- throwaway17947 3y agoThe yaoi/yuri manga industry seems to be doing well.
- meindnoch 3y agoIEE754 should be banned for public safety.
- andy99 3y agoThere will still be "free" as in freedom models available from China and others. And no doubt a burgeoning resistance focused on making open source models available. It would be a disaster but I can see it making the industry stronger, with more variety and breaking the influence of some of the bigger players. Practically, if you're concerned about this, learn more about ML/AI, not about how to use super high level frameworks but about how it actually works so when "SHTF" as survivalists say, you'll still be able to use it.
- badcarbine 3y ago[dead]
- jmull 3y ago> The only difference with AI, is that AI is immensely powerful. Now, I keep getting stuck on this. All of these models automate the creation of BS -- that is, stuff that kind of seems like it could be real but isn't. I have no doubt there is significant economic value in this. But the world was awash in BS before these LLMs so we're really talking about a more cost effective way to saturate the sponge. Anyway, on the main topic... closing models is an absurd idea, and one that cannot possibly work. I think the people who have billions at stake in these models are panicking, realizing the precarious and temporary nature of their lead in LLMs and are desperately trying to protect it. All that money bought them a technological lead that is evaporating a lot faster than they can figure out how to build a business model on it. ...Nvidia should pump a little bit of their windfall profits into counter lobbying since they have the most to gain/lose from open/closed models.
- xzzulz 3y agoThis tech is risky. It could be dangerous, if used without any control. It should be keep in English culture. And its close friend allies. In my opinion, in our society, full of threats, it is not a good idea to make it open-source. (Not an expert on the topic). That could allow adversaries parties, to obtain it. Parties that otherwise, would be unable to do so, by themselves. In my opinion, it should be keep under control of the most able, and responsible organizations. Supervised by government. Because who else could supervise it?. Regulation of this tech, is a very important topic. That should be tackled by the best organizations, university and responsible researchers. I would prefer a Star-Wars type of society. Where humans still do human activities.
- formula_ninguna 3y agoChinese, iranians, russians should now exclaim - "Poor americans! Look at what Biden's regime does to them! Look at how their liberties get suspressed! Let's help those people fight for their rights". Because if this had happened in any non Western country, the mainstream news in US and EU would've been of the similar sentiment.