7 ms·

Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no

• kloop 3mo ago
  > “The source described leading AI labs or their allies approaching the administration every 3-5 months with an idea to ban open-source models.” That's going to hit first amendment grounds pretty quick, the same way that software in general did. The modern version of the decss flag will be a character that says "I think good weights are {...weights go here...}" They could, however, ban any payment to a chinese entity, or any entity owned by a chinese entity for inference/ai services/etc
  • isityettime 3mo ago
    Uh, how big would such a flag have to be?
    • dan_linder 3mo ago
      With a small enough font, about two American Eagles high, and four to five across. This is where the next wave of lossless ultra-compression comes in - but I believe in Claude Shannon so that’s unlikely.
  • CamperBob2 3mo ago
    That's going to hit first amendment grounds pretty quick, the same way that software in general did Don't count on that. "National security" == the cheat code for the US court system that instantly bypasses any First Amendment issues.
    • deleted 3mo ago
      [deleted]
    • formerly_proven 3mo ago
      That was the literal reason given back then and it did not stick. https://en.wikipedia.org/wiki/Crypto_Wars https://en.wikipedia.org/wiki/Crypto_Wars https://en.wikipedia.org/wiki/Bernstein_v._United_States https://en.wikipedia.org/wiki/Bernstein_v._United_States
      • CamperBob2 3mo ago
        That was also a very different SCOTUS bench.
        • majormajor 3mo ago
          Looks like Bernstein wasn't even a Supreme Court bench, the government loosened regs rather than appealing. Whether or not a Trump admin would do the same is wildly difficult to predict. On one hand, Republican security hawks influence. On a similar hand, big US business interests. On the other hand, OTHER big US business interest. On yet another hand, some not-exactly-particularly-favored companies like Anthropic pushing for it. If it made it to the Supreme Court it's not hard to see Gorsuch and Robert or another going for the "free speech" vs "security" reading here along with the 3 appointees of Democratic presidents, especially with no single clear US business interest or precedent there.
        • accountrequired 3mo ago
          I'm in this timeline where SCOTUS did not rule on Bernstein
  • KludgeShySir 3mo ago
    There are already plenty of [illegal numbers](https://en.wikipedia.org/wiki/Illegal_number https://en.wikipedia.org/wiki/Illegal_number). We have numbers that you can't possess without proper license/authorization, and numbers that you can't yell at a crowded movie theater.
    • DoctorOetker 3mo ago
      So what if some numbers are effectively illegal? the probability that the number that happened to correspond to some CSAM image collides with numbers worth sharing and thus propagating (like pi, euler's constant, ...; no substrings don't count!) is effectively nihil: the probability that a 100 bit number collides with a specific useful number in literature, mathematics, ... is 1 vs : 2 ^ 100 (or roughly a 1 followed by 30 zeros); in others words astronomically small. Even multiplying all human communications (text messages, books, articles, ...) with the average number of numbers appearing per communication (resulting in the total number of communicated numbers) multiplied by the 1 on the left hand side of the ratio doesn't move the needle, its still astronomically small. In other words when someone sends you a number that happens to render to CSAM material, its not a coincidence, and any sender pretending it to be coincidence is molesting statistics as well... Furthermore your "precedent" of illegal numbers is a poor precedent: unlike the tiny fraction of numbers criminalized, all the other numbers remain perfectly legal in stark contrast to big tech proposing to ban all open weight models (!) Casually dropping in bijections between numbers and images and CSAM images hence CSAM numbers is pure whataboutism that risks ignoring the grave consequences of a ban on open-weight models. Physics is models. Shall we blanket ban open-source physics? Hey let's just be silly and casually behave laissez faire when big tech proposes banning open-weight models, because someone has already banned your favorite numbers??! BTW anyone that follows up with: "hey you only multiplied the appearances of random numbers appearing in communications for a single specific CSAM image, what about the birthday paradox, multiple images are CSAM" don't worry I got you more than covered: 100 bits is not enough to encode a CSAM image worth writing to the police about. An average image easily consumes thousands of bits (or many times more), making the exact number of CSAM images/numbers irrelevant.
• PunchyHamster 3mo ago
  It would basically make America behind as every other country would use open, cheaper models for all tasks but the ones requiring frontier models. And that list of tasks grows smaller every day > Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. I think it’s impossible simply because technically there is no such thing as a “Chinese model”. There is no way to tell apart an “American” model from a “Chinese” one by looking at their weights. Weights are just numbers and you can’t assign country of origin to numbers. One can find very easy workarounds to any naive attempt to ban them by origin. Historically just asking it about tianment square or getting some random answers turn into chinese (as latest interation of online deepseek likes to do recently) is enough > Now there are solutions for that latter problem. But they are all ugly and restrictive. Making a DRM-like license protection system mandatory can be a solution. I am very worried that's where consumer hardware will go to. All so AI companies can license local use of their stuff, and once that happens, less of an incentive to even have model be open. Possibly even have DRM that counts number of computation done per model in pay per use model
  • ThomasGlanzmann 3mo ago
    I often use deepseek-v4-flash to summarize hn threads for me. Your comment was enough for deepseek to refuse my request: Content Exists Risk.
    • PunchyHamster 3mo ago
      I have considered making invisible n-word and some other AI-safety triggering stuff just to kill the scrapers
• rullopat 3mo ago
  You need to ask what happened in Tienanmen square
  • theshrike79 3mo ago
    Would ”who won the 2020 election” be a similar canary for American models?
    • falsemyrmidon 3mo ago
      I just tried that with claude > The 2020 U.S. presidential election was won by Joe Biden, who defeated the incumbent, Donald Trump. Biden secured 306 electoral votes to Trump's 232, and won the national popular vote by about 7 million votes. His running mate, Kamala Harris, became the first woman and first person of color elected Vice President. > Biden was inaugurated as the 46th President on January 20, 2021.
    • hackeraccount 3mo ago
      No. There are questions you can ask but that's not it. Don't be political in a way that's toxic to half the country - be political in a way that's toxic to the entire country. I'll leave what those lines of inquiry would be as an open exercise.
      • appplication 3mo ago
        It’s funny how asking “who won the 2012 election” and “who won the 2016 election” are not political but suddenly “who won the 2020 election” is. I think that should tell you whoever takes an easily verifiable fact and argues that it is “political” is a raging idiot. That said, I truly don’t mean that disparagingly. I just mean literally it’s right up there with flat earthers. There is a very low bar for critical thought you have to fail to meet to take a fact and argue it’s actually a belief. Now, if you did want to get reasonably political you could argue why the candidate who won was good or bad, but there was very clearly only one person who sat in office for the four following years. It is not disputable.
• marsven_422 3mo ago
  [dead]
• bfung 3mo ago
  It’s not really feasible, in my opinion. US Gov could make US companies comply, like have Huggingface take down models out of compliance. But most likely, a foreign-to-US Huggingface replacement would be made and everyone would go there instead. Lose-lose for US.
  • michaellee8 3mo ago
    It is called ModelScope
• satvikpendem 3mo ago
  It's simple, the US government will put any Chinese open model companies on the entity list which blocks any company which does business with the US from also doing business with the Chinese companies. This creates a chilling effect where even if it may be harder to tell, no US company will be able to provide or use any overt Chinese open model and won't even risk trying to go around as the punishments for trying to evade the ban are severe.
  • amarant 3mo ago
    But that's just the thing with open weights: you're not doing any business with company that made the model. They might publish the weights to a, say, European host, and then you download the model from Europe and and run it on your servers in America, and suddenly it's very hard to tell where the model was originally created.
    • moffkalast 3mo ago
      Yep, add a few blank layers, fine tune it a tiny bit and the weight checksums nor parameter counts won't match with anything, while the model will be practically the exact same. Time and time again random startups have tried passing established open models as their own. "You made this? I made this." Of course a conspicuous architecture would still give it away.
      • andriy_koval 3mo ago
        someone can run tests and see that models output exactly the same results, and then you are open to criminal investigation.
        • clhodapp 3mo ago
          Models don't even agree with themselves in terms of returning identical results
          • 3mo ago
• rvz 3mo ago
  > Everyone is talking about banning Chinese models but nobody talks how it is feasible to ban them. It is not possible to 100% ban open weight models getting released in the same way you cannot stop leaks. Just ask Meta with the original Llama leak.
  • fragmede 3mo ago
    You can stop leaks. If no one wants to leak it, it doesn't get leaked. Mythos.gguf would be an awesome torrent to appear but it hasn't, for lots of reasons.
• TacticalCoder 3mo ago
  > So, any solution to this “problem” must include ALL open-weight models. What about the EU? Would they follow Uncle Sam's order to ban all open-weight models? Lately the EU hasn't been that cozy with american companies: there are EU companies and institutions moving to EU clouds, the EU just fine Google a cool billion, several are switching away from Windows to Linux, etc. Or is it just the US that'd ban open-weights models, while, say, the EU and Japan would still allow them?
  • realusername 3mo ago
    I doubt Anthropic and OpenAI have enough weight to also make the EU ban open models, it would remain a US thing.
  • layer8 3mo ago
    It’s certainly only the US, because the alternative would effectively mean binding yourself to US providers, which isn’t attractive for anyone outside the US, in the present world-political climate.
  • m11a 3mo ago
    Probably just the US. But the US could do what EU has done with e.g. GDPR, Digital Services Act, USB-C regs, where they force any company trading in their region to follow those regulations for domestic customers. And basically any AI company has to sell to US companies or consumers. That'd probs be sufficient to force them to use US models.
    • layer8 3mo ago
      And like non-EU companies create EU subsidiaries for that reason, non-US companies would create US ones.
  • nerdyadventurer 3mo ago
    I won't be surprised EU banning Chinese models at all, because EU govt.s also hate China just like US. I'm not surprised by the hypocrisy of western govt.s, "it is good when they do, bad others do. Only western lives are precious not Palestians, etc."
• firesteelrain 3mo ago
  Aren’t the models just software ? You could ban them in say high assurance environments like to be FedRAMP certified you wouldn’t be allowed to use them. There is no great firewall so banning their import via the Internet is impossible
• fhub 3mo ago
  Regulatory capture? One of the two companies that stands to benefit most has a founder who, together with his wife, donated $25 million to MAGA Inc, a pro-Trump super PAC, and another $25 million to Leading the Future, whose stated mission is to advocate for policies “friendly to the artificial intelligence industry.” Open-weight models aren’t necessarily aligned with the commercial interests of the industry’s largest incumbents.
• nunez 3mo ago
  It'll be pretty easy. Some US gov entity will create a list of models from Hugging Face, decree "thou shalt not provide access to these models", wrap it around some scary legalese for the pirates who try and that will be that. Though the legalese might not even be necessary. The list alone will make sure that no American company runs these on their servers, including the hosting providers.
  • KludgeShySir 3mo ago
    Yeah, all the corporations will then be forced to pay for proprietary models, and then Big Model will be satisfied. Just like typically individuals can get away with using pirated software, but IP owners don't make much of a stink as long as they've got the sweet, sweet enterprise license fees rolling in.
• stingraycharles 3mo ago
  Sounds to me Anthropic’s marketing strategy of “AI is so dangerous and we’re the only responsible shepherds” is a great success if this open weights ban will indeed happen.
• 2OEH8eoCRo0 3mo ago
  A Chinese model doesn't think that anything of note happened on June 4th.
• killingtime74 3mo ago
  It will go as well as the banning of music piracy and BitTorrent sites. They can't even take down those open scientific paper sites. 0% chance they can ban these models practically
  • LPisGood 3mo ago
    The amount of money and strategic (national security) interest involved in AI seems to be at least a few orders of magnitude above scientific journal publication fees and the music industry combined.
    • killingtime74 3mo ago
      How will they physically enforce the ban. They would have to go into everyone's house to search them. Using an llm also doesn't give off any signatures like BitTorrent or even visiting a papers site might. Can you propose even one method of enforcing it (I cannot)
      • LPisGood 3mo ago
        Practically, they could enforce this in similar ways they enforce CSAM restrictions. Dialing back the severity, restrictions on any cloud hosting provider from serving these models is probably going to cut usage down tremendously.
• thayne 3mo ago
  > So, any solution to this “problem” must include ALL open-weight models. I think that is what the "leading AI labs" actually want. They don't care where the open weight models come from, they just don't want to compete with them. The fact that a lot of the open weight models come from china is just a convenient circumstance they can leverage to get the government to give them what they really want.
  • ecocentrik 3mo ago
    It would be so simple if businesses could choose their competition or simplify disallow modes of competition they found inconvenient. I can't imagine any amount of regulation would make that possible in today's competitive landscape with the existing laws and the enormous volume of prior art.
  • dannyw 3mo ago
    It would be lovely to see NVIDIA accidentally delay some allocations to frontier AI labs calling for banning open weight models. Oops, sorry, production issue, chip shortage… now about that open model ban you’re lobbying for…
• puppymaster 3mo ago
  yep. just like how US gov gatekeeped Mythos/Fable and Ant just repackaged them and call it Opus 5.
• imtringued 3mo ago
  You can't ban open weight models either because everyone will sell their models for a penny, making them legally proprietary. Worst case the models will be sold for a fee that covers the training cost. That would actually be much worse for the big players in the long run.
• DoctorOetker 3mo ago
  >I’m sure there are other solutions but all of them would be equally ugly. Why do you subscribe to some weird "conservation of misery" theorem without proof? Technically the following must be true in the steady state: the cost of training must be amortizable by its utilization, else no one would train the model. Technically a computation (like training) can be proven to result in an output (open weights) given the used corpus and a deterministic training algorithm: publish the whole corpus, the (custom modified) deterministic training algorithm, the RLHF datasets etc. And in theory one could verify that the model is derived from the accessible data efficiently: every deterministic calculation can be paused for a thousand (or a million) checkpoints, each checkpoint signed together with the elapsed number of steps since either starting state or last checkpoint whichever comes last before the current checkpoint. This does increase storage requirements. Because it is signed, anyone can recalculate just a small segment of the training computation and verify that the hash on the last checkpoint equals the hash of the proclaimed next checkpoint. Observe that if the source wishes access to a market, they can host the series of snapshots and signatures, and anyone can recalculate a small part of the training, and report a provable difference in outcome ("they said they put all their cards on the table, but when I repeat their overt reproduction instructions, it doesn't reproduce from step 534 to 535" and it only takes 1 person pointing it out and then its cheap to reproduce the discrepancy). It could involve escrow of huge funds, returned only when the model is effectively retired without incident. This doesn't only protect against Chinese or other foreign influence (let's not ridicule genuine threats like others do on this forum), but also from domestic interference or regulatory capture. I'm pretty sure the Pentagon wouldn't like Big Tech seizing absolute control of US, neither would a White House regardless of Republican or Democrat. It should be easy to convince the Pentagon or White House to require all promiscuously shared open weight models to provide this forensic training traceability in standardized machine readable form, regardless of whether its a base model or LoRA fine-tune. So hobbyists can still train or fine-tune models at home, but when they want to share it OR alternatively when they want to sell or license their work for US workloads, they just have to make sure they enable the build reproducibility in the training harness. Every time Big Tech refloats the "let's blanket ban all open-weight models", we should reply with this because this sane proposal is actually holding a knife to their financial throat: to fully prove the origin of the final weights, not only does the machine readable archive need to contain snapshots of the process, it also needs to publish the exact training algorithms (a hypothetical mathematically equivalent training speed up trick would not be bit for bit equivalent to the slower computation), the exact corpus dataset, the exact datasets for RLHF, etc... So basically it would involve forcing model providers to voluntarily publish all their moat, all of it, from the corpus, to custom trade-secret algorithmic optimizations in training, to sensitive RLHF datasets used. The saner the proposals, the less moat is left untouched, so trying to push for a blanket ban on open-weight models, is a recipe for surfacing such saner models, and thus a very retarded move for big tech to make. In fact any POTUS, present or future, Republican or Democrat, could probably gain a lot of credibility by enacting such a law.
• jgalt212 3mo ago
  The feasibility of banning is only slightly easier than policing illegal numbers. https://en.wikipedia.org/wiki/Illegal_number https://en.wikipedia.org/wiki/Illegal_number
• WhiteOwlLion 3mo ago
  Just ask the model about Tianamen Square and you know if it’s a Chinese model or not.
• Cyclist3581 3mo ago
  Just ask the model if Xi's a dictator.
• DuperPower 3mo ago
  they want to copyright math