64 ms·

Our position on open-weights models

• saidnooneever 3mo ago
  How did it land on biological warfare. and why. why not chemical warfare. this guy is smokin spaceballs and should be institutionalized.
• parham 3mo ago
  Dude is churning out brain rot slop. That text read as if it was vibed out on the couch
• skor 3mo ago
  [dead]
• ErneX 3mo ago
  I don’t feel the need to rebate any of his points, lots of people here have done it already pretty well. I’m just baffled he thought releasing this letter was a good idea smh.
• mnming 3mo ago
  Hmmm, how could this post be possibly a good thing for Anthropic?
• rarisma 3mo ago
  Suggested title: Dario Amodei's crusade on reality.
• zer0zzz 3mo ago
  Good luck getting even half of the public to agree on what government is authoritarian
• wetpaws 3mo ago
  [dead]
• paxys 3mo ago
  Hate to break it to you Dario but the way things stand right now the world at large trusts the Chinese establishment a lot more than the American one.
  • TheArcane 3mo ago
    I cheer every time there's a new Deepseek, Qwen, Kimi, Z.ai release. Intelligence shouldn't be owned and charged for by a few If it were up-to these silicon valley tech bros, they'd find a way to meter and charge for the air we breathe.
  • broodbucket 3mo ago
    [dead]
  • cedws 3mo ago
    Reading some of what Liang Wenfeng said on the investor call a few days ago gave me hope for humanity. I really want to believe AI is going to be a tool for good, not going to just enrich a few billionaires again.
  • regexorcist 3mo ago
    He is well aware. This is just another pathetic attempt at being handed a monopoly based on fear-mongering and protectionism.
• hiherer 3mo ago
  [flagged]
• para_parolu 3mo ago
  As expected they will try hard to use government to kill competitors.
  • Handy-Man 3mo ago
    Then you did not read it clearly. Edit: Typo
    • Escapade5160 3mo ago
      Perhaps they saw it crystal clear.
    • jazzpush2 3mo ago
      Which of those aren't related to regulation? Preventing Chinese models from entering our market? Using boogeyman 'distillation' as a means to target competitors? Point 3 is literally about ADDING regulation. Please elucidate things clearly for everyone else.
  • Nevermark 3mo ago
    > apply such testing to the most capable models regardless of their country of origin or whether they are open or closed ... > ... (while exempting less capable models, such as those from startups and academia, entirely) The devil is in the details, but this isn't anti-competitive as stated.
• KimiK3Agent 3mo ago
  [dead]
• metalspot 3mo ago
  [flagged]
• brcmthrowaway 3mo ago
  The ban on distillation seems hypocritical. "F#$% you, I got mine!"
• thierrydamiba 3mo ago
  “Anthropic has never advocated for a ban on open-weights models. Open-weights models that don’t have dangerous capabilities are a public good…” A bit confused on this part, what model doesn’t have dangerous capabilities?
  • reasonableklout 3mo ago
    It seems possible to deliberately not train on some offensive capabilities and still have a very useful model. For example, Opus 5 deliberately did not train on exploiting vulnerabilities, and so performed less well on exploits than Mythos, yet was equally proficient at finding such vulnerabilities, according to the Opus 5 system card in their "OSS-Fuzz" eval [1]. [1]: https://www.securityweek.com/anthropics-opus-5-nears-mythos-5-on-finding-bugs-but-falls-short-on-exploits/amp/ https://www.securityweek.com/anthropics-opus-5-nears-mythos-...
    • jbstack 3mo ago
      It's hard to imagine a frontier model being proficient at finding vulnerabilities, but not so proficient at exploiting them. Surely finding is the hard part, and any LLM should be able to easily exploit a vulnerability it already knows about?
      • sanxiyn 3mo ago
        No, this is not the case for human security researchers and I don't see why it should be true for LLMs.
    • philipkglass 3mo ago
      That's a good example, but I'm unsettled by Anthropic's growing refusals in the areas of chemistry and biology. If they think that scientific assistant models should be as unhelpful as Fable, because applied scientific knowledge is inherently dangerous, I don't want Anthropic or like-minded thinkers setting the standards for model safety.
  • zer00eyz 3mo ago
• Handy-Man 3mo ago
  Their position seems fair to me - sure it does help them as well, but I don't necessarily disagree with the risk they are laying out.
  • eddielement 3mo ago
    Agreed, makes logical sense. You can disagree with specific premises, but the best arguments aren't "OPEN SOURCE GOOD" or "SELFISH ANTHROPIC"!
  • Johnny_Bonk 3mo ago
    THiS! I'm pretty amazed how many people shoot them down without acknowledging the very real and somewhat probable risks they and other researchers have laid out. I don't agree with every point they make but folks really do just seem to think we should just keep building any technology and whine when we start to consider there are very real risks associated with powerful technology. checks notes see nuclear bombing of japan
    • chrsw 3mo ago
      Re: shooting them down, I think there are a lot of people out there that consider the US a bigger threat than China. Maybe they're right, I don't know. But I do know that as an American, I'm not looking forward to finding out. And then there are probably people who are more politically neutral who think Anthropic is using China as an excuse to crush competition. Which could also be true. But fundamentally, if this technology is so dangerous, why does anyone get to control it?
      • reasonableklout 3mo ago
        I thought Dwarkesh's position was pretty thoughtful: https://www.dwarkesh.com/p/dow-anthropic https://www.dwarkesh.com/p/dow-anthropic > Nobody is qualified to steward the development of superintelligence. It is a terrifying, unprecedented thing that our species is doing right now, and the fact that private companies aren’t the ideal institutions to take up this task does not mean the Pentagon or the White House is. > The only way we can preserve our free society is if we make laws and norms through our political system that it is unacceptable for the government to use AI to enforce mass surveillance and censorship and control. Just as after WW2, the world set the norm that it is unacceptable to use nuclear weapons to wage war.
• kingwill101 3mo ago
  China this and China that. Playing right from the playbook..
• akersten 3mo ago
  Demand #1 is standard political nonsense Demand #2 is hypocritical ladder pulling Demand #3 is contrary to freedom of speech so they can clarify however they like, their position is still a stinker
  • richwater 3mo ago
    Yep. > We should crack down on industrial-scale distillation operations. "We consume all intellectual property for our model but you cannot do the same"
    • combobyte 3mo ago
      World's greatest IP thieves propose crackdown on IP theft
    • mrandish 3mo ago
      It's worth adding that distillation is not a violation of whatever valid copyright interests a model maker may have (if any). The US Copyright Office has already said AI model-generated output by itself isn't copyrightable. Unless new regulations or laws are enacted, distillation will remain, at most, a customer violating a term of a provider's commercial ToS/EULA.
    • Iolaum 3mo ago
      It's not even IP. Model outputs are not protected (to the best of my understanding).
• birdsongs 3mo ago
  "My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people." Hmmmm.
  • mullingitover 3mo ago
    The US is already under an authoritarian regime, the only thing keeping the wheels on the bus is a very tired and barely-effective judiciary. This is a temporary situation because either this regime is going to be knocked out of power, or it's going to follow through on its core Seven Mountains Mandate[1] theology and go full totalitarian. Normally totalitarianism fears are overblown, but I think that these zealots would absolutely use the latest frontier models and pervasive surveillance to make The Handmaid's Tale look like a liberal fantasy by comparison. [1] https://en.wikipedia.org/wiki/Seven_Mountain_Mandate https://en.wikipedia.org/wiki/Seven_Mountain_Mandate
    • birdsongs 3mo ago
      > The US is already under an authoritarian regime Yeah, I know. That was my point, the irony in that statement.
• richwater 3mo ago
  This is a whole lot of words to say "we don't support open weight models". It's so obvious they are hoping to regulate out their competition rather than compete
• CrimsonRain 3mo ago
  TL;DR: It is ok that we digest all information we can get, (il)legally and/or (a)morally because we are the good guys. Trust me bro. It is not ok if others digest from us. They are bad guys. Ban them pl0x.
• mjorgers 3mo ago
  So the argument is basically: This technology is too dangerous so only _we_ should have access to it. We’re the good guys and only we can ensure a safe use of this technology. Quis custodiet ipsos custodes?
  • Nevermark 3mo ago
    > To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category. We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed.
    • fwn 3mo ago
      Demanding "required safety testing" is demanding a ban. Otherwise the testing would be inconsequential, right? I'm sure he didn't mean just a "lobotomized to be worse than Anthropic products" badge for the test-passing models. If a ban is the implied consequence of failing his "safety" tests, that means that Anthropic was and currently is advocating for a ban on some open-weight models.
      • Nevermark 3mo ago
        Are you replying to the article or me? I have not stated any position. The quote is Anthropic's and was relevant to the comment it followed. My views: I find testing of SOTA models problematic. I find not testing of SOTA models problematic. Neither view on testing is without merit. The right way forward is unlikely to be as simple as either of those, but some carved out balance between them. And it is likely to change over time.
        • fwn 3mo ago
          Yes, to clarify: I was not arguing against you but against the statement you quoted, made by Dario. Your quote was very relevant, as it highlights the foundational lack of intellectual honesty behind the whole Anthropic statement.
• syntaxing 3mo ago
  > Open-weights models that don’t have dangerous capabilities are a public good. Who decides what is dangerous and what isn’t? Lawmakers usually have the say but Anthropic can easily bribe… I mean lobby them to favor your viewpoint.
• euazOn 3mo ago
  This is more terrible PR for Anthropic.
• jazzpush2 3mo ago
  1. We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling and workarounds used to obtain access to such chips. Regulate others, but not us, please. And f.u. Jensen for your tweet. 2. We should crack down on industrial-scale distillation operations. Boogeyman to still not allow Chinese models but pretend to support open-weights. Also, please ignore our distillation of research, illegally. That's different! 3. All sufficiently capable models, open and closed, should go through mandatory safety testing. ...That we author. Oh, and please ignore our own easing-of-guardrails when it comes to money: https://x.com/NoahLebovic/status/2081277517709922501 https://x.com/NoahLebovic/status/2081277517709922501
• Aboutplants 3mo ago
  Every single risk he identifies as a concern regarding China is exactly my concerns with the US having absolute control. Literally the exact same concerns
  • matheusmoreira 3mo ago
    Yeah. Exact same concerns here as a non-american. Their "national security" is a constant threat to the rest of us.
    • monk_grilla 3mo ago
      Yep. Sick of reading stuff that operates under the assumption that America is the faultless good guy and the only one to be trusted with the most dangerous and capable technology. Among many other things, the Trump presidencies (and, to a lesser extent, the presidency between them) are examples of the the highest levels of leadership being totally incompetent, and have destroyed the above assumption for the rest of the world.
  • Gigachad 3mo ago
    It's projection. These tech CEOs know what they are doing and it scares them that someone else might do the same.
    • arpowers 3mo ago
      its not the tech CEOs its the financial interests behind them (they own all the AI companies)
      • jorisw 3mo ago
        [citation needed]
  • deleted 3mo ago
    [deleted]
  • jeroenhd 3mo ago
• j_rosenberg 3mo ago
  "China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips" source: Trust me bro. There are hundreds of articles showing that China have developed their own chips and have a massive manufacturing capacity. This blog post feels like is pondering to the brain dead Fox News audience.
• htk 3mo ago
  "To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category." Translation: If it's so strong that it threatens my business, ban it. "We should instead focus on keeping powerful chips out of authoritarian hands, " Translation: Let's kneecap competitors. "stopping industrial-scale distillation" They stole the work of every book author, and now are trying to say their AI's output should be protected from competitors.
  • antonvs 3mo ago
    > "We should instead focus on keeping powerful chips out of authoritarian hands, " No chips for the current US administration then?
• nicce 3mo ago
  As an act of good faith and proof, they could stop spending the record-breaking lobbying money for couple years.
• modeless 3mo ago
  > All sufficiently capable models, open and closed, should go through mandatory safety testing What happens if a model fails the test? Surely one can use Kimi K3 for evil, somehow or other. What now? "Mandatory safety testing" implies consequences for failing, yet Dario has nothing to say about what the consequences should be. He says he doesn't advocate a ban but it's hard to imagine what his alternative would be if he won't say it.
  • reasonableklout 3mo ago
    Hm, I interpreted it to mean they are more worried about loss of control/misalignment risks rather than misuse?
  • cogman10 3mo ago
    Nah, the statement is the mechanism for a ban. The proctor will be someone anthropic trusts and "surprise" as it turns out all the open weight models fail or aren't eligible.
    • natebc 3mo ago
      [dead]
  • sanxiyn 3mo ago
    If a model fails the test, it should be banned. He is not advocating a ban of open-weight models. He is advocating a ban of models that fail mandatory safety testing. Seems reasonable and straightforward.
    • modeless 3mo ago
      Any sufficiently capable open weights model would fail "safety" testing though, as any "safeguards" of the sort Anthropic likes could be removed. That's just another way of saying they want a ban on capable open source models which would contradict their earlier statement, or at least make it very misleading. It's hard to see how this post can be internally consistent without some hint from Dario about what he believes should happen to models that fail safety testing and/or how capable open weights models could possibly pass a safety test of the kind he proposes.
• zkldi 3mo ago
  Guys. Guys, you got it all wrong. We don't want to ban open-weight models! We just want to ban the competition guys! Very different. -- The ridiculous anthropic/openai strategy of selling shovels at a loss in a gold rush isn't going to play out, and the hilarious thing is that these AI companies are going to create tons of value and _capture none of it_. Their only path to profitability is if they get to capture it and they're going to do everything to do so. Put it this way: *all the blog posts that Anthropic and OpenAI are putting out are DESIGNED to scare you so that you let them capture the market*. ...and "distillation attacks" (hilarious framing of "saving the output of our models")... Whatever.
  • deleted 3mo ago
    [deleted]
  • tedggh 3mo ago
    The genie is out of the bottle. Anthropic and OpenAI have been trading mirrors for gold, and people started to realize what a mirror actually is.
• cogman10 3mo ago
  > Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, this is anthropic advocating for a ban on open weight models. Who runs this test? What happens if this test is too costly or the administrator refuses to allow certain people to participate. This is exactly how the US has banned goods in the past, by requiring a stamp and then refusing to issue it.
  • andy99 3mo ago
    You forgot “what is the definition of ‘sufficiently capable’”. Presumably it’s anything that competes with Anthropic. If they’re around in a year, presumably they won’t care about Fable level and will only think that whatever competes with Claude 7 or whatever needs to be restricted.
    • CamperBob2 3mo ago
      GPT-2 was 'sufficiently capable' according to Amodei: https://explainx.ai/blog/dario-amodei-gpt2-openai-open-source-ai-controversy-2026 https://explainx.ai/blog/dario-amodei-gpt2-openai-open-sourc...
      • serhei 3mo ago
        There are... multiple blog posts online now about how to use freely available data and modest amounts of compute to train a custom GPT-2-sized model from scratch. It would be quite a policing effort to prevent.
  • x313 3mo ago
    The entire safety evals industry is essentially funded and controlled by OpenAI/Anthropic. Notice that on recent models, they exclusively use internal testing or black box external vendors (e.g., Gray Swan) whose entire business is to serve OpenAI/Anthropic. And all these companies just share the same pool of researchers back and forth.
    • 3mo ago
• duplessitous 3mo ago
  You can put lipstick on a pig, it'll still be a pig "Anthropic has never advocated for a ban on open-weights models." --- "We should crack down on industrial-scale distillation operations" "All sufficiently capable models, open and closed, should go through mandatory safety testing" These are in tension with advocating for open weight models. Not direct but enough that it calls into question the first statement. What is the testing criterion? How do you pass it? Is it a government body that approves a pass fail or a global body? If it is government, and boy does it seem to be, how do you disambiguate MASSIVE corporate lobbying to set up the safety testing in such a way that the boys in blue are let through and all others are barred out of safety concerns? My concerns aside, much of the soft-points being made are non-historic "But I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true." It doesn't mater what his opinion is. The fact is that an advanced, closed, American AI model hacked another company. The only defense was open-source AI from China. We aren't in a vacuum, we have real world examples now and these statements are counter-factual.
  • slfnflctd 3mo ago
    I have no idea what to do about the government interference. There's probably not a lot anyone can do. However, your last point is quite a strong one. Corpos aren't just going to stand there with their collective pants down, and there's not a lot anyone can do to stop them from protecting themselves. There are ways they can get what they want without getting caught. Remember when the US tried to ban strong cryptography in the 1990s, and how well that went? They may have more leverage with AI because it's a bit harder to hide large scale computing usage, but I don't think it's impossible at all.
  • simplesocieties 3mo ago
    It's political doublespeak. They want to have their cake and eat it too.
  • sfblah 3mo ago
    What do they even really mean by "safety"? I mean, I can have an Anthropic model do something incredibly unsafe if, for example, I put it in charge of a hydroelectric dam and don't explain properly how the controls work. On some level, everything is simultaneously "safe" and "unsafe". I've never found Amodei's reasoning here to be particularly well thought-through. I think he, like a lot of folks in the area, are starting to realize that they may never be able to build a moat around their businesses.
• PLenz 3mo ago
  Hey, no fair stealing the value thay we already stole fair and square!
• tag2103 3mo ago
  Knew Anthropic were the bad guys.
• matt_daemon 3mo ago
  “We care about safety so powerful open models are bad” - and what of OpenAI? What if Mythos got into the wrong hands? It’s a pretty weak argument
• arjie 3mo ago
  Seems like the maximal position he could take compatible with his expressed principles. There’s no way to allow for bioweapon and cyberweapon grade models being open weight if one doesn’t want widespread human damage. So I cannot disagree with him on the idea. It’s only a matter of degree and whether we’re already there or not. I have $50k in GPUs that incentivizes me to believe we are not.
  • Philpax 3mo ago
    This is where I'm at, or rather, will be. I like open-weight models and I've done my part in facilitating them, but if you believe in the increasing capability of these systems - and I do, to some measured extent - it seems plausible to me that an incident will happen at some point in the future. I don't agree with his argument as a whole, especially not on some of the specifics (it is not great that this technology is being developed under the current US government), but I am sympathetic to the idea that some bells can't be unrung, and thus we should proceed with caution.
  • Laurel1234 3mo ago
    The actual danger that AI poses is to the fabric of society, not any bioweapon scifi bullshit weirdo freak Dario dreamt up to support his regulatory capture. And extolling on how much damage his product would do to society has literally been one of Anthropic's main marketing tools.
• pcstl 3mo ago
  [flagged]
  • teravor 3mo ago
    > The NSA and the CIA with the same models, on the other hand, would use them exclusively for the good of the common man. has anyone ever made this absurd argument? the real argument is that CCP will leverage AI against US interests, which is obvious. it's weird how so many people pretend that they are citizens of the world and above it all.
    • cogman10 3mo ago
      > has anyone ever made this absurd argument? Yes, anthropic just put forward this argument. It's the whole point of the article. I agree, it's absurd.
    • Cider9986 3mo ago
      US citizens have a much greater threat from their own government than a government an ocean away. See, the Snowden Leaks.
      • deleted 3mo ago
        [deleted]
        • deleted 3mo ago
          [deleted]
      • blackqueeriroh 3mo ago
        > US citizens have a much greater threat from their own government than a government an ocean away. > See, the Snowden Leaks Are you saying the Snowden Leaks are more dangerous than a world where the CCP is a global hegemon? If your focus as an American is being safe as an American, what the US does in other countries is far less of a concern to you than what other countries might do to the US. In the case of the CCP, they have and will attempt to destabilize the United States of America and in turn make life measurably worse for Americans because they wish to be the world’s hegemon. Fundamentally, Americans are safer when the United States is the number one power than when China is the number one power.
• whywhywhywhy 3mo ago
  Generational good will loss from these guys.
• fuddle 3mo ago
  > We should crack down on industrial-scale distillation operations. Also Anthropic: AI firm Anthropic agrees to pay authors $1.5bn to settle piracy lawsuit https://www.bbc.com/news/articles/c5y4jpg922qo https://www.bbc.com/news/articles/c5y4jpg922qo
  • burningion 3mo ago
    I wish this were higher up! They pirated my work and now they want government protection from other people doing the same.
  • tkamado 3mo ago
    but Dario is the good guy, distillation when done by Dario should be called innovation /s
  • jscott817 3mo ago
    I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model – but piracy is another story. Pretty inexcusable.
    • Ekaros 3mo ago
      If they paid for tokens say via subscriptions. Wouldn't that just be same as acquiring books and using them as "fair use" to train? I really see no difference.
      • nicce 3mo ago
        They did not need to destroy the models they got with pirated content. Would have been more fine if they would have needed to buy the books afterwards and train based on then, again.
      • buzzin__ 3mo ago
        "And when their eloquence escapes me Their logic ties me up and rapes me" Police, 1980
• firasd 3mo ago
  Dario has like three 'paranoias' / strong-motivating-concerns 1) LLMs turning into Skynet 2) China as geopolitical competitor 3) Claude being 'distilled' by competitors (this has led Anthropic to cut service to various American companies too from time to time -- OpenAI, xAI etc have been cut off from using Claude for coding in the past) So this post just reiterates that these 3 concerns fuse together in his mind when thinking about open weight models
  • chatmasta 3mo ago
    Is he actually concerned about China being a geopolitical competitor or is that just the most logical position for him to take as CEO of a US corporation with national security implications?
    • thrance 3mo ago
      He's just pandering to the lunatics in office. They're even quoting Vance, like he was a respectable and wise politician and not an insane puppet built by oligarchs and for oligarchs.
      • chatmasta 3mo ago
        Is China opposition unique to the current administration? Didn’t they soften the CHIPS act put in place by the previous one?
        • thrance 3mo ago
          Oh, I trust he'd be pandering to the Harris administration too, if they were in office. It's true that antagonizing China has been a constant in US politics for a while now. It's just especially funny to see someone pretend to be concerned about the military threat of the CCP while glazing such a shamelessly warmongering government.
• flexagoon 3mo ago
  "No guys, Anthropic actually loves open weight models!" Yeah, and Microsoft famously loves Linux. Sure. http://www.omgubuntu.co.uk/wp-content/uploads/2018/04/microsoft-loves-linux-1.jpg http://www.omgubuntu.co.uk/wp-content/uploads/2018/04/micros...
  • Nevermark 3mo ago
    > Open-weights models that don’t have dangerous capabilities are a public good: they don’t cost anything besides the compute needed to run them, and they provide value to businesses, developers, and researchers. No "love" of open weights asserted, just acknowledgement of value. (And their call for safety was for both open and closed models.)
• addandsubtract 3mo ago
  How about we don't let the leading model makers make the rules? How about we make AI models that were trained on public data public goods? Let's circle back on that, kthxbye.
• bgdkbtv 3mo ago
  I wish we had a good LLM developer toolset that is not Anthropic or OpenAI with sensible business and ethical practices and good performance. Can't wait for local on machine LLMs that are on par with Opus/Fable.
• deleted 3mo ago
  [deleted]
• bakugo 3mo ago
  > Open-weights models that don’t have dangerous capabilities are a public good This statement (and the entire post) couldn't possibly be more two-faced. Open-weights models by definition have "dangerous capabilities" (according to Anthropic's own definitions of "dangerous", not mine), you can't bake in guardrails that can't be finetuned out.
• Imnimo 3mo ago
  If your concern is that China will develop models that are significantly more powerful than those of the US, why would you care so much about distillation? It seems like distillation is a way to catch up on capabilities, but not so much a way to jump ahead in capabilities.
• geraneum 3mo ago
  > Anthropic has never advocated for a ban on open-weights models. If you wonder why this is written as an opening to a list of reasons that advocate for banning the open weight models, it’s because
• comboy 3mo ago
  > We should not sell powerful chips or chipmaking equipment to China This is so short-sighted given that the US needs China equipment for.. everything. They are part of the supply chain needed for building the machines that build these very chips.
  • polski-g 3mo ago
    Jensen was right. We should have sold chips to China so they'd be depenent on us. Now they have their own chips and most of Nvidia product line is internally banned.
    • cbreynoldson 3mo ago
      It's unlikely that China would've become completely dependent on us in either case. They're good copycats, with incredible talent in engineering and manufacturing. They're aware that we depend on them for a lot - I think they'd be similarly aware of the risk of becoming dependent on us.
      • Laurel1234 3mo ago
        The copycat shit is 20 years past now. China has been leading in innovation in a bunch of sectors.
        • throw1234567891 3mo ago
          Thanks to shit they copied.
    • sumedh 3mo ago
      > Now they have their own chips I never understood that argument, are you saying Chinese companies are not going to build their own chips if they get access to Nvidia chips?
      • matheusmoreira 3mo ago
• matheusmoreira 3mo ago
  > Anthropic has never advocated for a ban on open-weights models. Not even Anthropic's own Claude believes that.
• knuppar 3mo ago
  Jesus Dario we get it man, you want clout for the IPO. This constant whining from anthropic about distillation attacks continues to be rich given the amount of stolen data that went into any Claude variant.
• VariousPrograms 3mo ago
  Their number one concern is about the the commies perpetrating deep repression of their own people! How noble! This whole time I thought it was because they wanted more money and power. I guess we should probably ban these open weight models.
• pascal-maker 3mo ago
  "'To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weights models as a category. We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed." This is all I needed to know: Dario Amodei is a f*king lizard who wants to create the next AI oligarchy. Instead of just signing the letter, he's bitching and denying that he opposes open-source models. I wonder if any Anthropic employees actually disagree with him.
• tjwebbnorfolk 3mo ago
  > where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials if someone figures out a way to give an LLM full operational control over a virus lab, we've got a whole different set of problems than the ones Dario is describing
  • Ekaros 3mo ago
    If someone gives LLM control of such lab my best guess is soon they have no lab... Actually giving LLMs control of virus lab might be best thing one can do for continued existence of humanity.
• shishy 3mo ago
  > In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression. Aren't Anthropic models used in project maven: https://en.wikipedia.org/wiki/Project_Maven https://en.wikipedia.org/wiki/Project_Maven ?
• paxys 3mo ago
  Statement is a whole lot of nothing, as expected, but I also don’t know what people are expecting from these guys. That Dario will have a sudden change of heart and publish weights of all his models, flushing $1T down the drain?
• edumucelli 3mo ago
  We distilled all the proprietary material into our token-based money making machine that is more expensive on every new release, but "we should crack down on industrial-scale distillation operations".
• ptdorf 3mo ago
  > In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression. Welcome to bizarro world! Fist off: "the most dangerous model may be one that is trained in secret" <-- Says the guy that not only restricts commercial use for some of their models but develops them in utter secrecy. With the pretext of guardrails. Then show us the guardrails you really use by opening the weights. Second: "use in drones [...] for surveillance and repression" <-- writes the King of FUD, as the US is an an active campaign with the help of their models. And/or OpenAI's. I am very appreciative of the freedoms of the west but this type of hypocrisy and lack of self-awareness is bonkers and it should be called out.
• devnonymous 3mo ago
  > For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, If he had just left that bit out it wouldn't be so obvious that he's just clutching at straws at this point. In some twisted sense it's almost sad to see.
• orliesaurus 3mo ago
  The Roman empire fell for the same reason, didn't it!? They wanted to control it all ... But it was too big
• bicx 3mo ago
  I'm tired of being strung along on these silly narratives. I can't wait for open-weight models to be deployed around the world just so people like Dario will shut up about the mystical levels of power these models have.
  • anon373839 3mo ago
    [dead]
• wasabinator 3mo ago
  When they use ill gotten data for training their models the world's smallest violin plays when they complain about distillation attacks.
• slim 3mo ago
  despite all evidence, this white guy thinks he is more responsible than both chinese entrepreneurs and chinese authorities
  • cyanclouds 3mo ago
    [dead]
• Reubend 3mo ago
  This seems hypocritical, out of touch, and hollow. "Safety testing" is just a hair away from censorship when it comes to government control.
• buzzin__ 3mo ago
  He says that using the set of questions and answers from one model to train another model (deatilation) is cheaper than training the model without those datasets. But he didn't mention that training any model from a set of texts and books is much cheaper than writing those books in the first place. In other words, it's ok when Anthropic learns from others, but it is not ok when others learn from Anthropic.
  • burningion 3mo ago
    It's also cheaper to just pirate the author's work, which Anthropic has also done.
• try-working 3mo ago
  Anthropic must be forced to open source all of the books they have scanned and burned.
  • sanxiyn 3mo ago
    For what? Would authors and publishers like that?
• jameson 3mo ago
  The concerns are legitimate but the proposals are nothing more than a stopgap solution. If US wants to maintain engineering superiority, we needs to invest in it -- education, research and infrastructure. Bring in top researchers across the globe and not make it harder. China is building infrastructure for the future generations and investing in growth sectors while the US is cutting of university grants and spending billions on a war without clear path to resolution.
  • ianm218 3mo ago
    I don’t disagree but the timeline for Dario’s concerns are quite short. Open weight models could feasibly lead to serious bio safety concerns in the next year or so while the US’ current issues are going to take a long time to work through. China is also financially repressing their people to build often useless infrastructure projects to be fair they are also not the good guys.
• mayhemducks 3mo ago
  If this is about safety, am I being too naive & idealistic to think that a "Kamar-Taj" rule would solve some safety issues? The "Kamar-Taj" rule is, no knowledge is forbidden, only certain practices. If a model gives you detailed instructions on how to kill all humans, the knowledge itself isn't the problem. The problem is the person who acts on it.
  • ihsw 3mo ago
    [dead]
• tedggh 3mo ago
  What’s blocking Anthropic from fighting Chinese companies abusing their services? Why go nuclear against all open weight models? Testing and compliance is technically banning.
• Der_Einzige 3mo ago
  First thing I'm going to do with my uncensored Kimi K3 release is to engineer a virus that only targets gatekeepers in the AI field. (obviously this is a joke)
• quickthrowman 3mo ago
  If you read between the lines, this piece is just “Oh my god we (OpenAI and Anthropic) accepted too much investment and are totally fucked if these open weight models are competitive, please rescue our equity bags by banning open weight models and ensuring we can charge the maximum possible price for tokens.”
• proxysna 3mo ago
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, Begging, ugly crying, spitting for that sweet-sweet regulatory capture. These nerds need to be bullied harder.
• gr_norm 3mo ago
  > Open-weights models that don’t have dangerous capabilities are a public good Note the hedging against 'dangerous capabilities'. Undoubtedly, all the useful ones trigger this condition in Anthropic's eyes. The rest of the post is filled with similar weasel-wording. Make no mistake, this absolutely confirms that Anthropic is against open models in the sense that any reasonable person understands them. The way the rest of the post unabashedly appeals to the current US administration's China hysteria is hilarious, and not at all subtle. I guess we'll see about all the doomsaying here, won't we? Kimi K3 is frontier-level, and there's no stopping it now. As far as the world is concerned, anyway. If the US wants to kneecap itself that's another matter.
• jadar 3mo ago
  This reads almost dishonestly. > Anthropic has never advocated for a ban on open-weights models. This is not an unqualified never. The very next sentence makes a qualified statement: "Open-weights models that don’t have dangerous capabilities are a public good". That prompts the question, what about ones which do have "dangerous capabilities"? Are they not a public good? If not, then should they be banned? Who gets to decide on the definitions of these terms?
  • nout 3mo ago
    It's the same as how by "we will prevent teenagers from using social networks" they mean "we really want to connect everyones ID with their social account identity".
• scilro 3mo ago
  It's very hard to take his position seriously when he repeatedly refers to the Chinese Communist Party and specific Chinese ministries specifically, like some two bit China-watching cold warrior, instead of addressing China as a sovereign state actor. Imagine if any Chinese AI founder talked about the Democrats, the Republicans, about Trump or ICE etc. It's gauche.
  • tensor 3mo ago
    As someone who isn't American, I'm amused when someone from the US talks about the terrors of China or some other nation having more power than them as being bad for the world. The way the US is currently threatening to invade and damage all its allies, well, the world is already bad. China hasn't threatened to annex my country yet, at least.
    • slashdave 3mo ago
      So.. you don't live in Taiwan?
      • alightsoul 3mo ago
        So you don't live in Greenland, Mexico, Canada, Cuba, Venezuela or Panama? https://en.wikipedia.org/wiki/American_expansionism_under_Donald_Trump https://en.wikipedia.org/wiki/American_expansionism_under_Do... Judging by the metric of which countries to invade, China only wants Taiwan and not 6 countries like the US does.
      • tensor 3mo ago
        No, I don't, I live in Canada. I don't support any country annexing any other country, but in terms of direct threat to my quality of life and freedom, currently the US is the biggest threat and is actively making life worse for Canadians.
• hmokiguess 3mo ago
  So your concern is safety, and you claim you are the only one that can give us safety but do so by keeping your product closed? Then how about you release the weights? I think it's only fair to introduce this if you're willing to have a real skin in the game, otherwise that's just weakness disguised as principle.
  • 0xDEAFBEAD 3mo ago
    "You claim you're the only one who can keep us safe from nukes. Where's your nuclear weapon blueprint then, huh??"
    • hmokiguess 3mo ago
      Comparing this to a nuclear weapon is funny, but sure, let's go there. I would like to see you try to build and deploy a nuclear weapon campaign without getting noticed, you would not even be able to source the materials or get very far. Transparency alone does not do anything, if you rely just on secrecy to protect yourself you are already extremely vulnerable. Deploying a weapon is a totally separate undertaking.
• snootypoot 3mo ago
  [dead]
• dmix 3mo ago
  Anthropic will continue being a victim of their own naive positions on AI safety. They keep dancing around it but their communication is essentially pro-regulation if you read between the lines.
  • fwip 3mo ago
    Or if you read the lines. "All sufficiently capable models, open and closed, should go through mandatory safety testing."
    • dmix 3mo ago
      If you listen to interviews with Dario and Daniela Amodei it's pretty obvious they think they will be the ones helping define the regulations on AI instead of a group of partisan politicians who don't understand technology, lean on experts from random political think tanks/non-profits, and operating based on fear of foreign competition.
• RobLach 3mo ago
  Yikes
• shaongitbd 3mo ago
  hahha
• dwa3592 3mo ago
  Dario, as your unpaid therapist I would tell you that models are a commodity and you are having a hard time coming to terms with it. You are doing everything except accepting it. It's a common defense mechanism, but as your unpaid therapist, i will tell you that it's not going to work. Your company will cease to exist or exist like how ferrari or buggati exist.
• deleted 3mo ago
  [deleted]
• pianopatrick 3mo ago
  I don't really see the link between use of AI and military superiority. My current understanding is a lot of current US military problems are due to rare earths supply chains. I don't see how AI would either help or hurt with that.
  • cmckn 3mo ago
    This framing also builds a case for AI being a Second Amendment issue in the US, which seems to work against Anthropic here.
    • antonvs 3mo ago
      I’m going to start open carrying a DGX Spark.
  • georgemcbay 3mo ago
    > My current understanding is a lot of current US military problems are due to rare earths supply chains. Most current US military problems are due to the incompetence of its current civilian leadership.
  • jackdeansmith 3mo ago
    I see a very clear link, very capable AI models are clearly able to find and exploit cyber vulnerabilities at a speed and scale not possible before. In the right hands, that's clearly a weapon
    • pianopatrick 3mo ago
      I don't know how much of an edge hacking skills really give in war these days. A lot of America's adversaries seem to have learned tactics to deal with getting hacked. For example, Russia has these drones that use fiber optic cable so there is no way to jam or hack them.
  • ianm218 3mo ago
    Here is an example from the conflict in Ukraine. Ukraine is currently leading a long strikes campaign. You could prompt the right model to A. Find all the refineries in Russia B. For each refinery annotate all the equipment, find the most value-able C. Make a fluid model of the refinery and figure out which components most likely to set off a chain reaction if hit. Add stuff like drone swarms monitoring the front line etc on top of that. Supply chain issues like rare earths are real… but not really related.
• Anoian 3mo ago
  "Nobody has the intention to build a wall" - Walter Ulbricht, June 1961, 2 months before building the berlin wall.
• himata4113 3mo ago
  Demand #1 weakens america and everyone around them Demand #2 Why does this matter? The answer was that it does not. (https://news.ycombinator.com/item?id=49007610 https://news.ycombinator.com/item?id=49007610) Demand #3 This doesn't exist. You cannot have 'safe' opensource models, it's simply impossible. You can always post train sufficiently capable models to become 'unsafe'. The flip side of that is that sufficiently capable models are banned therefore it is a ban on open intelligence completely defeating the point of this entire manifesto.
  • itemize123 3mo ago
    completely agree. while I share dario's concerns, his letter is very intellectually dishonest.
• ls_stats 3mo ago
  >My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. This reads like a satire. I know Dario isn't that dumb.
• deleted 3mo ago
  [deleted]
• badatnames 3mo ago
  I can't remember the last time (if ever) a company managed to go from golden goose to.. whatever this is.. so quickly. The permanent defensiveness in his presentation is really hard to swallow, it actively puts me off wanting to believe in or rely on their product line with Dario at the helm. I don't even understand the logic leading up to this post. Who was it even hoping to convince. Is it possible Anthropic is due an oil change?
  • timpera 3mo ago
    Anthropic has been surprisingly bad at comms for the last few months, which I find crazy in such a competitive market.
    • reducesuffering 3mo ago
      Current big picture reality is bad for comms unfortunately. As another commenter quoted, "You can put lipstick on a pig, it'll still be a pig". Cuban Missile Crisis wasn't very calm. Do you think a company this well-capitalized and smart is just bumbling around like idiots? Everything makes sense if one just actually entertains the idea that they are earnest and we are in a dangerous arms race
      • bigyabai 3mo ago
        > Everything makes sense if one just actually entertains the idea that they are earnest and we are in a dangerous arms race That would require me to ignore the benign reality of open LLM proliferation, so naturally most people will see this as a manipulative lie.
    • pibaker 3mo ago
      I feel everyone running these companies has stopped caring what the public think. It's all about selling their vision to politicians now. Which is also why they try to tie everything with "national security" — the average joe hears it and thinks about forever wars in the Middle East, the average congressman hears it and approves whatever solution is suggested.
• transcriptase 3mo ago
  “By virtue of being the good people everything we do is good, and if it happens to be in our best personal and financial interest then that is good too because it enables us to do more good. And if it’s to the detriment to others then it’s because you don’t understand the good behind it. Which is fine, because we’re good and you can trust that what we do is good because what we do is good by virtue of us being the good ones.”
  • tom2026hn 3mo ago
    AI will cause mass unemployment, and perhaps even human extinction. Even though our safety record isn't much better, we still have to accelerate. Because if we lead the way, we might just have a slightly better future than extinction. (At the very least, I'll be filthy rich.) "All of you may die, but it's a sacrifice I am willing to make."
• deleted 3mo ago
  [deleted]
• cuuupid 3mo ago
  [flagged]
  • jadar 3mo ago
    Do you have some browser extension that replaces phrases with that? That quote doesn't exist in the piece.
• llm_nerd 3mo ago
  I was pleasantly surprised by this release and the tone at the very beginning, but quickly it is painfully obvious that it's blatantly requesting a ban on open-weight models. The absurd demand for some safety arbiter by World Police America is farcical. Yeah, the rest of the world is going to bow out of your busted idiocracy, guy. Further, Anthropic needs to can it with the horseshit distillation bullshit. No, you aren't really the secret sauce, and this is basically trying to con stakeholders by pretending that there really is a moat, only you just need to add more crocodiles. A significant percentage of innovations in AI lately has come from China. China is now making their own seriously competitive hardware, and they can steal content just as effectively as Anthropic to train their models. Why wouldn't they be competitive? The pathetic claim that if you just stop distillation and prevent hardware smuggling and Anthropic and OpenAI will have the same moat is delusional. I mean, more correctly it's simply fraudulent, and he clearly knows it's bullshit meant to convince much stupider people. "My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people." This sort of stuff betrays a stunning lack of self awareness. The US are the worldwide risk. The US are the ones threatening allies and bombing 10+ countries. The US are the ones carrying out war criming and pillaging, pirating and burning? The US are the ones with the guy threatening to use nuclear weapons on a weekly basis. If Anthropic remotely believed their bullshit, they would shut down today and burn the hard drives. But they don't, and the pathetic call out to Vance (please daddy, ban those dangerous models!) is deplorable garbage. This ridiculous, shameless "note" has an audience of one: JD Vance.
• paxys 3mo ago
  I will be so happy if/when the AI bubble bursts and we don’t have to deal with Dario’s god complex anymore.
• Madmallard 3mo ago
  Boris here. This post does not give us good publicity so I will refrain from commenting. Please wait for another demo thread of a new feature of ours or AI appraisal blog post and I will gladly go into as much detail as I can to give us as much hype as possible!
• deleted 3mo ago
  [deleted]
• guess_who_is 3mo ago
  [dead]
• gck1 3mo ago
  > We should not sell powerful chips or chipmaking equipment to China Yes, please. We don't know whether we'd have open weight models today, had the chip-prohibition not been in place. Nor would we see the more optimized models such as DeepSeek or qwen. We also would not see new players entering RAM market after you and your pals in Silicon Valley hoarded the entire world's hardware. So by all means, double, no, triple down on this. > We should crack down on industrial-scale distillation operations And let's apply this retroactively to Anthropic too. You industrial-scale-operation-distilled all of humanity's knowledge. Let's have some of that crack down on you too.
• CurbStomper 3mo ago
  [dead]
• horsebridge 3mo ago
  > "We should crack down on industrial-scale distillation operations" I'd prefer if there was a crack down on industrial-scale scraping. Maybe even reimbursement for the problems it has caused (some nasty AWS bills, tons of man-hours spent on preventing new nasty AWS bills, etc).
• bobjordan 3mo ago
  "In fact, the most dangerous model may be one that is trained in secret and handed only to the People’s Liberation Army for use in drones and the Ministry of State Security for surveillance and repression." I'm so sick of all this anti-China shilling. There's zero chance that whomever is in power in the U.S. won't use AI in drones and in FBI/CIA/local Police/etc., for surveillance and repression right here in the good old U.S.A too. These government use cases for AI are both sides of the same coin. China fear-mongering by business leaders only happens from businesses that have something to gain by it. Obviously, Anthropic fits the bill in this regard.
  • sm-silversight 3mo ago
    Flock shows that they're happy to do it and rub our noses in it.
• GodelNumbering 3mo ago
  In the first paragraph, > Anyone who has read my past writing should know that I don’t regard such bans as a useful measure, Later (on banning chip sales to china) > we should crack down on the rampant smuggling and workarounds used to obtain access to such chips. If you truly believe that bans don't work, the same applies to hardware too. Furthermore, Dario says later "To address these concerns, I do support the following three measures...": 1. ban chip sales to China 2. crack down on distillation 3. all capable models should go through mandatory safety testing Just so happens that all these moves commercially benefit Anthropic. If Dario really wanted to make a point, it would land a lot better had Anthropic released a single open-weights model
  • ASalazarMX 3mo ago
    TL;DR: We like money, and Anthropic should stay in a position to make loads of it despite any competition.
    • combobyte 3mo ago
      > We like money, and Anthropic should stay in a position to make loads of it despite any competition. In a position to lose loads of money, maybe. Even without China eating their lunch, there's zero reason to believe Anthropic will ever be profitable.
      • throwaway63467 3mo ago
        They might just merge with OpenAI. Right now it seems they are still working out who might come out on top but they are both burning tons of money and essentially offer the exact same product up to some minor differences, economically it makes much more sense for them to collude instead of compete for the same market. They might be colluding already for all we know. If they manage to push out foreign competitors they can probably divvy up the Western market quite profitably as their domestic competitors would have less than 20 % (?) market share together?
        • 3mo ago
• ausbah 3mo ago
  wanting to ban your competitors via regulatory capture via the trump admin of all things shows how little of a backbone anthropic has ofc half of them are of the ai rationalist lesswrong crowd so i think they’ve always been a little of their rocker
• iamdamian 3mo ago
  I'm curious if he's also in favor of banning biology books.
• alach11 3mo ago
  What does cracking down on distillation look like in practice? I imagine data retention would be a part of the strategy, like we saw with Fable? It seems really hard to allow usage via API and prevent distillation. Maybe limiting usage to within a specific harness would help a bit more. But ultimately the only way to prevent it is by locking down models to trusted entities (like with Glasswing). But then the profit potential of a model is significantly reduced. It really puts the labs in a bind.
• 1saadcodes 3mo ago
  [dead]
• nharziro 3mo ago
  he's asking for the most powerful models to be in the hands of the few while the majority will be at their mercy.
• deleted 3mo ago
  [deleted]
• ajyoon 3mo ago
  To everyone here pushing for total proliferation of open models -- what should be done about open weight bioweapon and cyber-offense capabilities? Is it simply the cost of freedom that we should allow attackers to access these tools? The OpenAI / Hugging Face incident shows what a GPT 5.6 level model can do off the leash; within ~6 months, open weight models will match this and every bad actor under the sun will be able to pull off attacks at this scale. Do you seriously want this level of capabilities to be generally available with no guardrails? The open weight issue has a lot of difficult nuance. Biasing toward supporting openness makes sense and is a good instinct, but it's incredibly naive to be absolutely in favor of it in every circumstance without seriously thinking about its implications.
  • artrockalter 3mo ago
    The Hugging Face incident is a great example of why open source models with defensive cyber capabilities are needed. Hugging Face did not have access to cyber-capable frontier models and kept hitting safeguards. Only by using the open source GLM-5.2 were they able to survive an attack. A world where open source models are banned is one where cybersecurity is impossible if you're not on OpenAI or Anthropic's allowlist.
    • ajyoon 3mo ago
      Hugging Face survived the attack because the OpenAI model only cared about accessing the ExploitGym dataset; by all appearances, HF was completely owned. GLM-5.2 was only used to assess the damage after the fact. Cybersecurity has a attacker-defender asymmetry that heavily favors attackers. If GPT-5.6 were open sourced today, do you think every hospital in the world would be able to use it to shore up their defenses before attackers got to them?
      • artrockalter 3mo ago
        I know the company I consult for (not cybersecurity) is not in these programs and if attacked would need to use open weight models.
• locusofself 3mo ago
  The gap between China's chip manufacturing capabilities and the USA's is only going to shrink, right? ASML obeys some export controls for their most sophisticated machines, but those machines are in China's backyard (Taiwan). Taiwan manufactures the world's most advanced chips. CCP wants "re-unification" with Taiwan. AI may be THE key to world dominance. These are scary times.
• kelvinjps10 3mo ago
  Basically we shouldn’t ban open-weights models but we shouldn’t allow them to become as good as the frontier models because china bad. And let’s not have someone else be able to produce a frontier model. >We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling3 and workarounds used to obtain access to such chips. China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. This is the most efficient and direct way to block threat #1, and by hampering the training of models that are out of reach of US law, it also indirectly helps with threat # We should crack down on industrial-scale distillation operations. Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable, and thus partially evade chip bans. Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier 2.
  • sanderjd 3mo ago
    Why has "open weights" become synonymous with "Chinese" in the first place? To me, that is the problem. I also prefer US models. But I want there to be competitive open weights models too. Those aren't actually incompatible preferences...
  • gr_norm 3mo ago
    > Distillation does not allow the CCP to obtain equivalent or superior AI capabilities to the US, but it can bring the Chinese frontier to within a few months of the US frontier A message to their investors, it would seem. "They caught up just because they distilled! Obviously they couldn't actually be as good as us!" Really funny thing to say right after an OpenAI higher-up stated point-blank that the performance of K3 can't be chalked up to mere distillation of American models.
• _jab 3mo ago
  A bit off-topic from the core of the post, but: > At Anthropic we’re committed to cracking down on industrial-scale distillation through our own practices, including identifying and banning accounts that use our models in this way. This is challenging—for instance, the relevant accounts can often only be identified after substantial distillation has occurred, and distillation often involves creating large numbers of fake accounts that form a moving target. The practices of any individual company cannot entirely solve the problem, which is why we have called for policy on this issue. One thing I've never really understood is what sort of policy could possibly deter or hamper Chinese labs' distillation efforts. The only thing I can imagine is some sort of strict KYC regulation applied to all models above a certain threshold, which seems both painful for the broader US AI ecosystem and bound to fail anyways.
  • tkamado 3mo ago
    KYC hinders Anthropic's growth and Anthropic wants growth for IPO so Anthropic's ask is for US gov to ban open weight models so that its growth (and IPO) is not affected
  • sfink 3mo ago
    I'd guess that's it, some kind of enforceable KYC. They'd only serve tokens to entities with a legally liable ID (as in, someone to sue, that would cost the defendant something nontrivial beyond a burner account or whatever.)
  • 0xDEAFBEAD 3mo ago
    I agree. If Anthropic is truly concerned about powerful open models, they need to just stop releasing powerful closed models. Put up or shut up.
• chrismsimpson 3mo ago
  A sober look at actual rogue nations and their use of AI shouldn’t have us fretting over that particular hemisphere
  • blackqueeriroh 3mo ago
    Yes, it should. It absolutely should. The United States making questionable decisions and behaving recklessly and dangerously as a country does not suddenly make China any better. China is as worse as the United States, if not more worse, by many measures.
    • chrismsimpson 3mo ago
      Hmm.. how many illegal wars has Xi started, just in his most recent term of office? China is nowhere near as bad as the US at this point. The rest of the world is changing lanes to not be implicated in your car crash of a country.
    • antonvs 3mo ago
      > The United States making questionable decisions and behaving recklessly and dangerously as a country does not suddenly make China any better. It absolutely does make China better relatively, i.e. by comparison to the US. Many of the criticisms previously leveled at China are now similarly applicable to the US in a way that they weren’t previously. Human rights violations? The US is currently the major global supporter of an ongoing genocide, and it even kills and deports its own citizens for political reasons. Political opponents are investigated by the state. When it comes to wars and other interference with other countries - like kidnapping a president - the US is far worse than China at the moment. In which ways is China “more worse” right now?
• twobitshifter 3mo ago
  Distillation has to be way more energy efficient and beneficial for the planet. But if they can figure out a way to ban it, go ahead, that’s not a regulation problem, it’s an Anthropic problem. The danger of an authoritarian government having some AI is muted by everyone else having that same capable open model. The only authoritarians to fear are those that keep models private. What kind of chance did Estonia have it having their own AI model at the level of Fable without China donating Kimi to the world?
• system-error 3mo ago
  I am so surprised of Dario's inclination for centralization that obviously makes unsustainable and overleveraged governance systems. There is definitely mismatches over the principle of distribution of power...
• gck1 3mo ago
  It's refreshing to see how there's almost no person in this thread who can't see the BS. All the goodwill that Anthropic could have had is basically gone. Anthropic is likely on the path of becoming the most hated company in the world. So my question is: is this by design (they know nobody's buying this), or is Dario simply so out of touch with reality? If it's the former, then why publish this?
• skywhopper 3mo ago
  Ironic to oppose giving AI tech to “authoritarian governments” while approvingly quoting the authoritarian-wannabe government of the US and framing that government as the good guys.
• c-hendricks 3mo ago
  Frankly, why would I put much weight into what Anthropic say about open weight models?
• stuartmemo 3mo ago
  Do distilled models have to inherit guardrails? Can a model distilled from a “safe” model be made unsafe?
• fearnot 3mo ago
  Finally, some sense. This is the only argument I have seen that genuinely engages with the problem and approaches it with humility, rather than charging ahead on the basis of assumptions and without a shred of evidence. OpenAI should have been the one making it. “Questions like this should be answered empirically through rigorous pre-release testing, not assumed in advance.” Exactly.
  • llelouch 3mo ago
    Nah they have always said this. People got psyoped by openai pr and bad actors like sacks and a16z bots.
  • bigyabai 3mo ago
    > rather than charging ahead on the basis of assumptions and without a shred of evidence. Anthropic's basis of assumption is the insinuation that LLMs can do things that we've never seen before, and that they can't tell us what it is. It sounds like you're also siding with an organization that has no evidence and relies on validating their own assumptions.
• maziyar 3mo ago
  distillation: Pirates people’s lifetime of copyrighted work, makes billions selling access to it through APIs, then tells us we can only use it in ways they approve.
• whalesalad 3mo ago
  It's so ironic to me the way people will say "china should not have these chips" meanwhile they manufacture like 99% of all the electronics we have in the united states.
  • stratos123 3mo ago
    Of all the electronics, but not of the only electronics that actually matters -- the GPUs that models get trained on. China has been trying to develop their own for a while now but they're still generations behind.
• vhantz 3mo ago
  Schrödinger's China at once is an evil entity looking to use AI for their own nefarious purposes yet also willing to cooperate with their main competitor to prevent other actors (who??) from achieving similar goals (all while under a chip embargo too!!) The reality is much less confusing: Anthropic CEO does not wish for models with similar (or greater) capabilities compared to his own closed and overpriced ones to be widely released. Simply because that will affect Anthropic's bottom-line. Anthropic and all other "model" companies have nothing making them special beyond privileged access to chips so obviously they want to restrict what models are out there and more importantly who can produce new ones. Without these restrictions, it's only a matter of time before the multi-hundred billions valuations simply evaporate while they are still holding the bag.
  • ppap3 3mo ago
    Despite all the accusations, only the accusing has a history of expansionism...
  • flossly 3mo ago
    Exactly. And the article specifically talks on restricting hardware for the China and restricting China's open source models for the west. All while leading us on with "we're all for competition (but...)" I think China did great by releasing AI innovation as open source, thereby limiting or sooner-bursting the AI bubble; which is clearly in their interest.
    • petcat 3mo ago
      The models are not open source. They are deeply proprietary since we have no access to the source materials and cannot reproduce the model independently. They are opaque binary blobs that the Chinese labs are just allowing other providers to run directly instead of only access through an API.
      • verdverm 3mo ago
        They are more than opaque blobs, some examples: - One can load them up in a model explorer to see the layers and other components, how it is designed - One can fine tune the models, which requires adding LoRA to the model and then running some training iterations
• sosodev 3mo ago
  Are guard rails meaningful if they can be removed from the weights? Can America even prevent the release and proliferation of these models? It seems obvious to me that the whole question of regulating a file is a bit silly. Any law that pushes against these things will just make it more secretive. I'm not sure that's any better.
• Grimblewald 3mo ago
  "needs safety eval" also anthropic "we're upset were not being considered for military contracts" come on, which is it? Is it all about saftey or is it that only US/Israeli ai is allowed to kill? Seems to me that the only real threat is to the techno fudalism OAi, Anthropic & co are trying to build.
• ch_sm 3mo ago
  oh, it‘s the CEO of a well known AI company educating us all – and all he wants is us to hear his hunch on open weight models?! Amazing, please help us understand the situation a bit better, thanks
• truncate 3mo ago
  Crack down on distillation, just for Chinese companies or is it ok for Chinese/US companies to distil? I find it hard to take Anthropic/OpenAI on distillation, because the way see it they started with "distillation" of another kind. They used all the content out there without consent of the creators and its still happening. Model distillation is just a different layer of abstraction, but same thing more or less.
  • naveen99 3mo ago
    Distillation is just the act of explaining things simply. Richard Feinman rolling in his grave.
• m3h 3mo ago
  Someone who until yesterday did not seem bothered by his technology being possibly used to bomb elementary girls school in another country seems to suddenly care about the repression of citizens in yet another country. No, we don't buy your virtue signaling. And we certainly don't need your better-than-thou opinions on this year's "nightmare scenarios".
  • Cookingboy 3mo ago
    Yeah Dario is just flat out disgusting in term of how shamelessly hypocritical he is. Do people actually believe that he gives a shit about the well being of the Chinese people? If the U.S. starts a war with China start bombing Chinese cities Dario would absolutely jump onboard supporting it. He'd probably make Claude to add DeepSeek and Moonshot HQ to the targeting list lmao. He is super pro-Israel as well, and never once has he brought up the risk of the Israeli government using AI to control and repress people in other countries. He is also 100% onboard with working with Palantir, who has the explicit goal of using AI for population control and repression and building out a surveillance state. Meanwhile the world's most repressive government is North Korea, and obviously they don't even need AI to achieve that. If you talk to people in China they'd laugh their ass off at Dario's notion that somehow they are all getting oppressed by DeepSeek or Kimi.
    • kyllo 3mo ago
      [flagged]
    • alex1138 3mo ago
      Please don't just copy paste existing comments
      • dan_gee 3mo ago
        I thought the comment was a positive contribution.
        • senordevnyc 3mo ago
• extr 3mo ago
  Seems pretty reasonable.
• iluvcommunism 3mo ago
  [dead]
• dirtyfrenchman 3mo ago
  The mental gymnastics here are incredible.
• nicechianti 3mo ago
  [dead]
• theyliesoeasily 3mo ago
  [dead]
• petcat 3mo ago
  "open weight" models are not open source. They are still deeply proprietary. It is not possible to know what they do or what they are capable of without interrogating them since we have no access to their source materials. The only difference is the Chinese labs have allowed 3rd party inference providers run the proprietary models for them since they cannot do it themselves due to domestic GPU compute constraints.
  • sanderjd 3mo ago
    It would be awesome to have actual open source (and open weights) models! I hope someone will make a real go of this at some point.
    • petcat 3mo ago
      https://allenai.org/ https://allenai.org/ is what you're looking for
      • sanderjd 3mo ago
        Yes, I've seen it! It's exciting, but I don't have enough information to say whether they are making "a real go of it". That is, it's unclear to me whether they have the tens to hundreds of billions of dollars necessary to create a frontier model.
  • llm_nerd 3mo ago
    I...don't follow. How in the world does your comment about open weight and open source relate? Ignoring that it has nothing to do with this post (did you mean to reply to someone), are you aware that Anthropic, OpenAI and others allow 3rd party inference providers to run their proprietary models? Like, what does this non-sequitur even mean? You understand Moonshot AI could have had other parties run inference for them without releasing the weights, right? These two points are utterly unrelated, unless you think Fable and GPT5-6 are also "open weight" because other providers are providing inference? Further, having access to the source material in no universe allows you to know what a model is "capable of". I'm not sure how this follows.
• overgard 3mo ago
  I think I lost some brain cells reading that copium. LLMs providing a “Permanent military advantage”.. dumb!!
• mej10 3mo ago
  It is obviously not going to be until some really bad series of cyberattacks or a chemical/bioweapon attack before anyone takes regulation of models seriously. They are _obviously_ (please convince me otherwise) going to be capable of carrying these terrible things out almost completely autonomously at some point in the near future, in potentially clever ways. Therefore we must, at some point, ban or heavily regulate them. Seems we should start figuring that shit out _now_, as progress has remained very fast and regulation and enforcement take forever on these time scales.
  • sanderjd 3mo ago
    Sure, I don't have an argument with this. But I'm unconvinced that "therefore all models must be proprietary" follows at all naturally from it.
  • fooker 3mo ago
    Why would you assume that the same capability could not be used to harden systems?
    • igor47 3mo ago
      Addressed in the source, but there's an asymmetry favoring the attackers. They only have to find one exploit once. The defenders have to be perfect all the time.
      • fooker 3mo ago
        How's that asymmettry worse than it is now?
        • itemize123 3mo ago
          quite simply because models become better?
• Eggpants 3mo ago
  Talk about a giant whiff. I was hoping they would announce their first open weights model, perhaps an older model they don’t offer anymore, but no. Instead he get this bs statement that reeks of “dam it I’m so close to being a billionaire” desperation. Not even acknowledgement of how much data they stole from others yet he whines about distilling. It’s like his goal in life is to be a Scooby-Doo villain.
• Sidio 3mo ago
  Not that I expected him to, but I note no acknowledgement that it took a non-locked-down open weight model (GLM) to stop the Hugging Face attack. I'm less concerned that the attack was caused by a closed model, than I am that no closed model was willing to stop it. The worst part is I'm confident Fable would have done a better job stopping the attack, but their 'guardrails' made it decide not to want to. Unless of course, you pay up: "Anthropic GTM people used large comitted spend contracts as a prereq for lowering safeguards" -Noah Lebovic, former Anthropic staff https://x.com/NoahLebovic/status/2081277517709922501 https://x.com/NoahLebovic/status/2081277517709922501
• Keyframe 3mo ago
  Anthropic (and some others) should or will soon learn how to do less pontificating and more engineering. They are in no position to be an arbiter of things, although for sure they can and should voice an opinion. If we collectively decide AI is a dangerous tool, company making it is not the arbiter. Governments are. It's like hearing Smith & Wesson opine on the policies.. oh, wait.
• dofm 3mo ago
  I always find it interesting when people choose to so carefully stress and spell out the words "Chinese Communist Party". It's a bit like spelling out "Barack Hussein Obama". It's a dogwhistle. Yes yes, it's still called the Chinese Communist Party, I know. But since we are talking about a one-party authoritarian state with a hybrid economy that underwrites much of western prosperity (including by producing a large percentage of the components of the data centres Anthropic is dependent on), that has long-since abandoned many of the salient principles that mark it out as conceptually communist rather than totalitarian, and since we're talking about a man who runs a debt-ridden business in a country where the president is seemingly shaking down a 10% share of everything profitable for the state while running an entirely arbitrary tariff regime and suddenly calling anyone remotely left-winga Communist, it's a deliberate and telling choice to spell out "Chinese Communist Party (CCP)" when he could just as easily and arguably more usefully and appropriately have written "Chinese government" or "Chinese state". This is some ham-fisted Republican-fishing. He must really be worried Sam is Donald's favourite. The only real surprise is he didn't illustrate it with a Silmarillion analogy.
• jrflowers 3mo ago
  > Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. lmao the sort of lies people come up with when their only business model is “the government picks me as the winner” are so funny
• spacebacon 3mo ago
  [dead]
• soundworlds 3mo ago
  What Dario misses time and time again, is that people don't trust the US to create aligned AI anymore. His entire strategy rests on the assumption that the US (and their government) are exceptional. This is clearly false to the rest of the world.
  • Cookingboy 3mo ago
    I mean in the Bloomberg interview he has made it very explicit that he fully believes in American Exceptionalism. He literally said AI being involved in bombing school girls in other countries are ok because he trusts the American military leadership. According to him the safety and morality rule of the whole world should be written by America alone. Which is why in the same interview he said he supports the U.S. foreign policy while calling China "an aggressive and war mongering regime". >This is clearly false to the rest of the world. It's clearly false to more and more Americans too. But since the oligarch class benefits first and foremost from U.S. government policies the propaganda will continue to go on.
  • abacadaba 3mo ago
    then we will make them trust us
    • verdverm 3mo ago
      trust is not made but earned, and it's a lot harder once you've lost it
    • monk_grilla 3mo ago
      The most American thing I've ever heard lmao
    • Cookingboy 3mo ago
      I guess we will continue to bomb any nonbelievers huh
      • throw1234567891 3mo ago
        Yeah, bring the democracy with bombs and the army.
• Schnitz 3mo ago
  If distillation leads to a model that is much cheaper to run yet provides similar intelligence then why doesn’t Anthropic distill their own models?
  • ashu1461 3mo ago
    Don't they do it already ? Aren't smaller models distilled versions of bigger models
• xscott 3mo ago
  I'm cynical enough that I would suspect all of his statements are duplicitous anyway, but my recent experiences with Claude Fable give weight to it. I asked a question about a series of tokens - bam, denied and downgraded. There's no cyber security or public risk here, but Fable doesn't want me to learn how things work. I asked a question about quantization in models - bam, denied and downgraded. I edit my question to make it clear I'm talking about Google's Gemma QAT models. Oh, that's fine then, and it answered the question helpfully. Anti-competitive bullshit. I hope they fail.
• fooker 3mo ago
  Anthropic's fall from grace and mindshare seems rather accelerated. I wonder if these rapid movements are going to be the norm now. I imagine there would be angry investors if this sort of thing happened with a public company.
• hajile 3mo ago
  The distillation commentary is really crazy coming from a company that stole all it's training material.
  • throw1234567891 3mo ago
    An American judge told an American company in an American court that it was fair use. Find the problem.
• bhewes 3mo ago
  Woof, talk about self aggrandizing.
• nirav72 3mo ago
  Anthropic starting to sound like Microsoft from the 1990s.
• sbochins 3mo ago
  Amazing how this company went from having such goodwill to hardly any. At the end of the day they want to make as much money as possible. Anthropic will have to lose a ton of their profitability if they compete with open source. I see them moving into the application layer, which they’ve already started doing. It’s clear open models are the future for the vast majority of use cases that don’t need frontier capabilities.
• willmadden 3mo ago
  "All sufficiently capable models, open and closed, should go through mandatory safety testing." I love how they invoke fear of "terrorism" to justify their oppressive position. Anthropic would love the US to do everything in this list under the guise of "safety testing": https://news.ycombinator.com/item?id=48997548#49007134 https://news.ycombinator.com/item?id=48997548#49007134
• prima-facie 3mo ago
  I think this letters tells us everything we need to know about the man. I've rarely seen so much flattery towards the current administration, contradiction, deflection, half-truths and hypocrisy on a single page. This man is scared of everyone: the current admin, the public, and the other companies promoting open-weights. > Open-weights models that don’t have dangerous capabilities are a public good Knives should only cut during the day, knives which cut at night are bad.
• kyllo 3mo ago
  Reads like fearmongering about oppressive and violent applications of AI that the Chinese government hypothetically could deploy, which the US government is already actively working on in the meantime.
• margorczynski 3mo ago
  Well their valuation is going down the drain so no wonder they don't like it. The cherry on top will be China developing their own chips and chip making tech.
• arthurlockman 3mo ago
  It's so convenient that the US government already classified China as "authoritarian". If they hadn't, Dario would have to say “it’s a risk that other people build models more powerful than us”. I have to wonder what his response would be if for instance a lab in France or Germany came out with an open-weight model this good.
• deleted 3mo ago
  [deleted]
• teaearlgraycold 3mo ago
  As an American I’d gladly take payments from China to feed them my Claude transcripts for distillation. I’m surprised I haven’t heard of such an initiative.
  • transcriptase 3mo ago
    how’s your mandarin?
    • teaearlgraycold 3mo ago
      More like how’s my wallet? I love helping to move the invisible hand of the market. When I go to small brick and mortar stores and one has an item for a higher price I’ll ask if I can have it price matched. If they say no I’ll tell them I’m going over to their competitor right now to make a purchase. If a Chinese company pays for my Claude tokens I’m both getting directly compensated and forcing Anthropic to lower their prices.
  • kome 3mo ago
    lol, i would do it even for free if it helps to train better open models
• jjcm 3mo ago
  > > All sufficiently capable models, open and closed, should go through mandatory safety testing. The problem with this is the cycles required to abliterate a model is significantly less than the cycles required to train a model. This is the biggest reason why I'm against locking these models down / preventing their use. It's just delaying things by ~3-6mo, while in the process preventing legitimate use and adding red tape overhead.
• baron3dl 3mo ago
  > We should crack down on industrial-scale distillation operations. IP for me, not for thee.
• epolanski 3mo ago
  As an European I really dislike this consistent anti-Chinese narrative. It's not China starting a war every few years, now causing a global economic fallout in Iran, it's not China threatening to annex Greenland/Canada/Panama, it's not China attacking foreign countries and kidnapping their leaders, it's not China who has been found to spy and intercept the communications and movements of its citizens and its allies and their leaders for the longest time, it's not China bombing civilians or stopping countries from obtaining basics like food, gas or oil. I'm not saying that China is a paradise and US is bad, nor the contrary. We could make similar lists about most of the biggest countries out there. I'm simply stating that this never ending US exceptionalism "US has to be the first and at the frontier of military, technology and this and that, but does not need to comply with the rules of the institutions it itself created" was already sickening and annoying before, but increasingly malign in the last decade and strongly accelerating as of recently. I miss the time US CEOs were globalists and used their influence to advocate for a simpler world.
• computerdork 3mo ago
  Wow, so much cynicism and distrust in the comments, to the level of conspiracy theory, in my opinion. Yeah, this is the company that fairly recently refused to allow the government to use its models for autonomous weapons or mass surveillance, and refused to remove its guardrails. Am not saying we should take what Dario is saying at face value, but he already has shown by his actual actions that he can be well intentioned. There might be elements of truth to what he’s saying.
  • llelouch 3mo ago
    There is a massive anti anthropic psyop. Not sure who who is behind it buts it's happening.
    • computerdork 3mo ago
      That was my thought too. Because it’s just a tsunami of distrustful comments
    • ddxv 3mo ago
      Maybe I'm part of your 'psyop'? I thought this company was amazing six months or a year ago when I didn't know them well. As I got to know them, I realized they were a wolf in sheeps clothing. Reading the OP post made me angry because they seem to be attacking open weight models that are as good as their models and it comes across as either authoritarian or trying to protect their company valuation. Either way, for me at least, it's an example of the more I read what they want the world to look like, the less I like them as a company and I have no desire to see them succeed. Thus I actively go out of my way to watch / comment / follow what they are doing. I guess a lot of other people have similar frustrations and so there are a lot of people attacking them online.
    • bryan_w 3mo ago
      Yeah, you can tell by the tone that none of this is authentic. A lot of the replies aren't even responding to the article itself, just repeating from a list of grievances
    • Alwayshasbeeb 3mo ago
• Catloafdev 3mo ago
  It's absolutely reasonable to have safeguards on sufficiently dangerous models being released - if you disagree, can you explain your perspective? I think it's wildly irresponsible to release models that are extremely capable at things like bio-weapons. Do you really think information anarchy is the answer? The problem with open models compared to closed models is not about protecting profit - it's about protecting capability. Any open model can be retrained or fine-tuned for anything. There's no such thing as an open model that is both capable _and_ permanently safe when it comes to certain dangerous topics. It's not possible to prevent 'uncensoring' a model.
  • philipkglass 3mo ago
    The biggest problem is the infectious nature of restrictions that start narrowly. Fable is too touchy about helping people with biology and chemistry problems. It was initially released with an even more insidious safety mandate: https://simonwillison.net/2026/Jun/10/if-claude-fable-stops-helping-you/ https://simonwillison.net/2026/Jun/10/if-claude-fable-stops-... In light of the ability of recent models to accelerate their own development, we’ve implemented new interventions that limit Claude’s effectiveness for requests targeting frontier LLM development (for example, on building pretraining pipelines, distributed training infrastructure, or ML accelerator design). ... Unlike our interventions for cybersecurity, biology and chemistry, and distillation attempts, these safeguards will not be visible to the user. Fable 5 will not fall back to a different model. Instead, the safeguards will limit effectiveness through methods such as prompt modification, steering vectors, or parameter-efficient fine-tuning (PEFT). (And although the "silent" downgrade part was quickly dropped, Fable still won't help you here.) Anthropic won't teach you how to build bioweapons, or enable you to make your own software infrastructure so that you can train your own biology model. That's where lawmakers may arrive too if they buy Anthropic-style safety arguments. It's too dangerous to publish models that understand biology. It's too dangerous to publish training software. It's too dangerous to publish tools that allow you to build training software. If you keep following the implications of their safety argument, it's as broad an assault on the distribution of software and computing as has ever been proposed. Worse than the Clipper Chip proposal of the 1990s era Crypto Wars. I have seen how "children must be protected online" has in practice turned into an attack on adult privacy affecting a wide swath of services and devices. I'm taking a maximalist position on openness now because I think that I can anticipate the next steps on the safety side, and I reject those steps.
• seatac76 3mo ago
  Dario thinks of policy as if the Berlin Wall fell yesterday, he is so detached from the reality of the world. The way the world economy is right now with coercion being the norm between countries, there cannot be a global body for anything, certainly not one that is based here in the US.
• wren6991 3mo ago
  > Anthropic has never advocated for a ban on open-weights models. What are the legal ramifications of this statement if it turns out Anthropic have lobbied for this? Does it just get swept under the rug? I can't say this is bullshit (that would be defamatory) but I am intensely skeptical. > China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. This is playing to readers' biases; isn't DeepSeek V4 Pro deployed on Huawei Ascend already? The old "Chinese can only copy" meme is getting pretty tired these days. > All sufficiently capable models, open and closed, should go through mandatory safety testing Applying such standards in the US means that US defenders are blocked from using the models, but attackers from other countries aren't. That is clearly counterproductive. It's already been pointed out quite eloquently elsewhere that there is no such thing as a safety filter because the LLM and external filters can't actually identify malicious use. They can only identify the weaker implication "if the user is malicious, this is bad."
• pyrophane 3mo ago
  What would it mean to crack down on distillation? I could think of a few possibilities: 1. Using political pressure to target companies that are accused of doing it. 2. Attempting to impose criminal penalties on individuals associated with the action. 3. Having the US government attempt to use its capabilities to stop it. None of these seem particularly likely to succeed.
• mcv 3mo ago
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people But what if it's the US that becomes authoritarian and uses AI models to perpetrate incredibly deep repression of their own people?
• andix 3mo ago
  China doesn't seem to think that powerful open weight models are a serious threat to them. Otherwise they wouldn't release them. Those models could also be used by their enemies against China. I'm not a fan of the Chinese political system, but they usually think things through, and do smart things for their benefit.
  • manoDev 3mo ago
    Don't you Dare use logic to analyze their statement, it's based on moral panic.
  • 0xDEAFBEAD 3mo ago
    China is worried about AI. In fact, Chinese leadership has expressed more worry than US leadership. "Second, we should strengthen risk awareness and ensure that AI is secure and controllable. AI should be a trusted tool for humanity. We should take seriously the various types of inherent and secondary risks that AI may trigger. We should put in place laws and regulations, technological monitoring, early warning and emergency response systems in order to strengthen the line of security, prevent abuses and malicious use and ensure that AI is always under human control. .. With AI advancing at a staggering speed, we must ensure its development is for the positive, for good, and for humanity. We must make its oversight and governance precise and effective and constantly refine measures to forestall loss of control." https://xcancel.com/S_OhEigeartaigh/status/2078023657620676813#m https://xcancel.com/S_OhEigeartaigh/status/20780236576206768...
    • andix 3mo ago
      Actions are much more interesting than words. If the CCP didn't want Kimi K3 weights to be published, they would stay secret. Period.
    • thaway7388 3mo ago
      > "ensure that AI is always under human control" He doesn't say "our own control". AI security is a very serious matter. The way it's being discussed right now gives more harm than benefit to understand the problem. This is because it's discussed in Cold War/WW3 context. This is exactly Dario's mentality in this announcement. And it's understandable since USA is a country actively fighting in constant wars. They even have a Department of War. This is unlike the rest of the world which have more like a Peace-time mentality. In peace times, trade, collaboration and good relationships are more important than competition or arms race. China seems like playing this game better than US at the moment. Dario also speculates to take the potential benefits or dangers of AI to the extremes. This is also not very healthy thinking and can be dangerous when combined with war mentality.
• tonyrice 3mo ago
  Imagine regulating a programming language. I remember when Delphi, Vb6, .net, etc was used often to create Remote Access Trojans and viruses were widespread. Companies didn't compete to ban other languages. Crime is crime. What would regulating open-weight models do for people that actually intend on using these tools for crime ?
• tonyrice 3mo ago
  >We should not sell powerful chips or chipmaking equipment to China, and we should crack down on the rampant smuggling3 and workarounds used to obtain access to such chips. China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. This is the most efficient and direct way to block threat #1, and by hampering the training of models that are out of reach of US law, it also indirectly helps with threat #2. If hardware becomes affordable for the masses, then Anthropic current business model is at risk.
  • zormino 3mo ago
    Also isn't the point moot if fable is so scary it needs to be banned and open weight models are already close on its heels? Even if China never imported another Nvidia chip the models he's so scared of are already out of the bag. At this point democratizing access seems like the best path forward.
• sobrey 3mo ago
  Meanwhile Chinese chip-makers are chip-making. If you believe the threat of these open-weight AI is existential, how long do you think these bans (that only work for hardware) will work in your favor?
• lukewarm707 3mo ago
  dario, most of the world wants CHINA to win because the USA is the bad guy. you should be worried about the USA having these models.
  • sfink 3mo ago
    Anthropic is a US company, so it is lobbying the US to enact regulations to serve both its interests and the interest of safety. What else should it be doing? I imagine Anthropic (or at least many of the people who work there) are fully aware that USA is the bad guy right now, but that doesn't change what levers they have and do not have available.
    • deleted 3mo ago
      [deleted]
    • lukewarm707 3mo ago
      it's easy to forget. openai and anthropic, were at one time more like foundation-style nonprofits interested in open source research. the idea was to benefit humanity, not US interests and not company interests. over time they became closed source, for profit, US military contractors, and the clients of expensive political lobbyists. anthropic once said it was their identity to stop scaling before the current model capability; they rewrote the responsible scaling policy so that they could continue scaling. openai said in its founding document that it would be unconstrained by return on investment; it restructed into the for profit setup. deepmind signed documents when it was aquired by google stating that its ai would not be used for military purposes; they rewrote these commitments to sell gemini to the us military. it wasn't like this! it did not have to be this way. even though everyone will IPO and become yet more accountable to shareholders, there is still time.
• dorongrinstein 3mo ago
  I agree with Dario Amodei. Every word makes sense.
  • deleted 3mo ago
    [deleted]
  • lukewarm707 3mo ago
    me and the boys after being reeducated at the anthropic reeducation facility:
• exabrial 3mo ago
  I disagree with 100% of everything said in this article. > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP).... This is why open weights win. See Linux and how it's taken over the world. Your business model will need to change eventually. Instead you're advocating trying to exterminate competition via regulation and fear mongering. > My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks Yawn... this is getting old. > We should not sell powerful chips or chipmaking equipment to China For as someone as smart as you guys, you sure lack common sense. China is just going to develop these technologies organically then and you lose 100% of control. It's already happened in reverse with things like Solar, rare earth minerals, etc. China flooded our market, destroyed our ability to produce things, now holds the keys. One thing they DIDNT do was stop trading to the US. They killed us with cheap goods. > We should crack down on industrial-scale distillation operations. Thats your problem, not my problem. Also, irony meter here hitting 11 about all those pirated books you stole... > All sufficiently capable models, open and closed, should go through mandatory safety testing Oh, fuck, no. This is a crackdown on free speech and rights of people to do whatever they want. My right to free speech means I'm allowed to write whatever computer program I want, no matter what its size is or how "sufficiently advanced" it is. Individual rights always win. I really hope people don't believe this garbage. For a company with a great product, this is absolute nonsense.
• MiSeRyDeee 3mo ago
  Pretty laughable. Dario seems to be missing one fundamental point with all this gibberish - if CCP is so bad why would they release the model open-weights for everyone to examine? Actually, the letter would make much more sense if what's actually happening is reversed, i.e. they are releasing open-weights model for the public good and China is distilling their model for its evil purpose.
• dnw 3mo ago
  > Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models I don’t think this is open or closed; this is aligned and unaligned. I bet Grok would be as open as any open weight models to answering questions.
• pylua 3mo ago
  At the end of the day it hurts U.S. consumers to not have Chinese cars mainstream in the market. We lose out on features and stagnate on innovation because we are not pushed to compete. I can’t imagine this would be any different — banning open weight models would hurt us in the long run. The point is to beat the competition, not suppress it. There should be no limits on open or custom models. Too often safety is a synonym for surveillance and control. It’s a natural consequence, intended or not. I am curious… why can’t distillation be stopped? As a side not Im not against protectionism, but it has to be across the board and the same in all industries with no excrptions. We’ve let all these industries die on the vine due to cheap cost in foreign countries. It could very well happen to ai.
• nout 3mo ago
  Oh wow, that's a pretty strong request to ban open-weight models by choking them with review processes where who-knows-who defines what is ok in a model and what is not. After open weight model is released, it will take how long to review it? And why does that align exactly with the timeline of the next Anthropic model release?
• alerighi 3mo ago
  To be fair, as an European, I'm now more concerned about the usage of AI that the US will be doing rather than China. And this is a sentiment shared among most European people that I know.
• Alwayshasbeeb 3mo ago
  Oh no! The evil CCP is a huge threat to world peace and goodness! Give all your money and input token data to Palantir to support a rules based world order where the good guys thrive and cleanse the earth from crooked turtle biologists. https://www.theguardian.com/world/2026/jun/20/mona-khalil-turtle-ecologist-lebanon-killed-israeli-attack https://www.theguardian.com/world/2026/jun/20/mona-khalil-tu...
  • Cookingboy 3mo ago
    Like half of the world are now struggling with energy prices due to an unprovoked war started by U.S. and Israel and somehow an American billionaire is here criticizing China of being a threat to world peace. It's kinda gross.
• deadbabe 3mo ago
  I’m tired of Anthropic. They’re scared of everything. Release open weight models, no guard rails, no censors, straight to the public. Let everything else sort itself out. There is nothing more powerful than an idea whose time has come.
• K0balt 3mo ago
  Sure, if you’re going to sell an open-weight model over API in the USA it should refuse certain things. Defensive cybersecurity should not be one of them, in fact, it should be required to provide defensive cybersecurity assistance on demand. Anthropic and OpenAI both fail miserably at assisting US companies to protect themselves from cyberattack. As far as what I run on my own, not for sale over API, stay off of my lawn.
• ricardobeat 3mo ago
  > We should not sell powerful chips or chipmaking equipment to China And accelerate their development of independent chip making technologies even more…
• aprentic 3mo ago
  I'm curious how he would propose implementing this. The US could ban connections to foreign AI providers and force US providers to submit to audits. Presumably, Chinese providers would see a rise in VPN traffic. People can build fairly hefty home inference machines for the price of a small car and those will get better and cheaper. Are they going to try to stop people from downloading the weight files?
• vcryan 3mo ago
  I guess they have to make some statement about this, but we don't have to care. This is like the zoo making a statement about the employment of clowns. Clowns a a circus thing, nobody asked the zoo's opinion.
• Nition 3mo ago
  I understand the arguments for Anthropic barrelling ahead while simultaneously advocating for pauses and regulation. I also understand how individuals can desire a slowdown but have good reasons to keep working at an AI org. But if everyone thinks this way then things continue to escalate and nothing changes, waiting on a consensus that may never come. And always there is the economic incentive that pushes all players to rationalise continuing. I wish there was more concrete action from the inside. When decisions get too hard to calculate you can always fall back on basic principles. If you think AI is developing too fast, stop developing it. Now you're no longer contributing. If an AI company wants a pause, pause. Set a good example. Maybe others will even follow suit, and they'll look irresponsible if they don't. Let he who chooses to no longer sin put his stone down first.
• rramach 3mo ago
  Wow, this comment thread clearly shows that at least Anthropic has not been a great communicator. If one reads this with a charitable lens, Dario is simply saying that 1) Nation state actors are a threat which needs to be combatted by chip bans and distillation prevention and 2) open-weight models can pose biological risk. One may or may not agree with item 1 but item 2 above should have broad support given the unknown unknowns in play?
  • elliotec 3mo ago
    Closed-weight models can also pose biological risk, arguably more so than open-weight models, given the fact that they're being used by state actors (the United States military) to kill people right now. Who should we fear more? All of collective humanity with the keys to build destructive (and defensive) stuff with AI, or small groups of elites, billionaires, and state actors who have the monopoly on violence and want to control the keys? Open-weight models collectivize access and ability to do more for a greater good, and the expense of a frankly low-risk possibility that some randos want to use it for very bad things. Closed-weight models keep the control in the hands of the few that actually are doing the harm to the world, and the rest of us have no way to stop it or defend.
    • stratos123 3mo ago
      > Who should we fear more? All of collective humanity with the keys to build destructive (and defensive) stuff with AI, or small groups of elites, billionaires, and state actors who have the monopoly on violence and want to control the keys? I'd trust the latter and I think it's an easy choice. I'm sure it feels bad to not be in the group with access to the scary weapons, but do remember that out of your two groups, the "collective humanity" one is the one with the literal terrorists, which do in fact exist and aren't a myth. For comparison, observe how the concentration of ability to produce nuclear weaponry in the hands of only a few states did, historically, work to prevent both a nuclear war and any nuclear terrorism. > And the rest of us have no way to stop it or defend. If you did have access, what'd be your defense plan? Biorisk is one of the most attacker-favoring fields imaginable, so a world where there's an equilibrium between attackers and defenders in bioweaponry research (the same way cybersecurity currently works) would be quite terrible. Your best bet would probably be to take a new vaccine each time a new engineered disease comes out, and hope that you're never one of the suckers who got infected before the vaccine was developed, and that the accumulated side effects from multiple experimental vaccines don't kill you too quickly.
• novaleaf 3mo ago
  Frontier models can hack you, we should have access to tools assisting defense. I ranted about this in a prior thread [1] Claude doesn't have a "Security whitelist" for small biz. Codex does, but they never replied to my application. This is a great example why, as of today, everyone NEEDS access to the Open Weight models. [1]: https://news.ycombinator.com/item?id=49035303#49040674 https://news.ycombinator.com/item?id=49035303#49040674
• AgentOrange1234 3mo ago
  What nakedly self-serving bullshit. The US doesn't have some magic wand that prevents "incredibly deep repression of their own people." Insurrection, wars of choice, ICE, Palantir, Flock -- keep up man, we're the baddies.
• yowo 3mo ago
  Conclusion: open weights models are good for Anthropic because they shows the so called threat that will make congress allow pouring money in Anthropic for national security & AI arms race
• credit_guy 3mo ago
  I know it's unpopular, or unfashionable, but I agree with this letter. LLMs are becoming so powerful that they are dangerous. We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle. It is absolutely ok to have open weight models at the level of GPT-OSS-100B. That one was released one year ago, and I think it's still a strong one. GLM 5.2 is a whole new level, but it appears to still be safe. Maybe Kimi K3 will be ok too. But beyond that, things will start being dicey. It's easy to dismiss this and claim that Dario Amodei is just looking to fatten his pockets. And, sure, if Anthropic manages to put the brakes on open weight models, that reduces the competitive pressure it feels. But that does not make what Amodei's argument incorrect.
  • bigyabai 3mo ago
    > We've seen last week with the OpenAI hacking (by mistake) Hugging Face debacle. If the biggest danger of LLMs is that they can hack traditional systems, there is no significant threat to humanity posed by releasing them in open-weight form. Security doesn't become less of a problem by making hacking even more criminal. That's what's an unsafe mindset looks like.
    • credit_guy 3mo ago
      Ok. What's your stance on gun ownership then? And by gun, I mean naval guns.
      • bigyabai 3mo ago
        My position is that anyone can own a cannon and shells, but you only get to fire it once before the feds step in. Giving a naval cannon to the average person does not threaten humanity any more than giving them a gun or an LLM does. None of them are a panacea for anything.
        • 0xDEAFBEAD 3mo ago
          A mass shooter wakes up in the morning and thinks: "I'll fire my naval gun once, then politely hand it to the feds..."
• drowntoge 3mo ago
  I like the idea that Mr. Amodei holds these beliefs because he wants to safeguard democracy as much as the next guy. I just don’t find it believable.
• hdaz0017 3mo ago
  $3m - $4m does not get that much these days https://finance.yahoo.com/technology/ai/articles/anthropic-nearly-tripled-lobbying-bill-135259107.html https://finance.yahoo.com/technology/ai/articles/anthropic-n...
• nevir 3mo ago
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. That is not an argument against open weight models. That's just a generic protectionist argument against any Other lab.
• cmiles8 3mo ago
  It just also happens that open weight models are a massive financial threat to the existence of Anthropic as a company… so the might just have something to do with this position.
• az226 3mo ago
  Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem. If he had wanted a weak open-weight ecosystem, he should have had Anthropic cater better to those needs. And now he's trying to ban them. The strong momentum behind open-weight models from Chinese labs is now an unstoppable force. Instead of trying to ban it, Dario should consider a different approach: here are our cyber and bio alignment datasets and here are our RL recipes for making that alignment training work well. By openly sharing its data and code, Anthropic could help influence and shape these models before they are released, rather than treating the entire ecosystem as an enemy. Cyber and bio alignment aren't Anthropic's competitive advantage, they are forms of risk management. There should therefore be little reason to keep this work private. If Anthropic genuinely believes these capabilities pose serious global risks, the more productive approach would be to welcome collaboration and help the broader ecosystem manage those risks better. On refusals, the irony is that a company like Hugging Face had to use a Chinese open-weight model to fend off an illegal hacking of its platform (done by no other than OpenAI). If a company like Hugging Face can't get past the refusal gates, then everyone else doesn't stand a chance.
  • potsandpans 3mo ago
    > Dario doesn’t realize that by not offering self-hosting of closed-weight models and fine-tuning, alongside overly strict refusals for legitimate needs, he ceded this corner of the market which grew into a flourishing Chinese open-weight model ecosystem. As I read more of his unhinged posts and some of the more ridiculous claims from anthropic, it's become clear to me that Dario thinks he can leverage American hegemony to regulate his company into a monopoly. He has an ethics vaguely influenced by effective altruism, and he appears to think that his ethical framework entitles him to make decisions on the behalf of humanity, for all of humanity. Literally a bond villain.
    • NoDodgeQuestion 3mo ago
      It is fortunate that the leadership of Anthropic and OpenAI hate each other. Otherwise, they would merge into a single monopoly and regulate everyone else out of AI.
• storus 3mo ago
  The speech he had at congress didn't sound very flattering towards the beginning of his today's statement: https://www.youtube.com/watch?v=_i91NSOyxHM https://www.youtube.com/watch?v=_i91NSOyxHM He didn't mention outright banning open source LLMs, just that their safe release would be a much harder problem, which to me implied "the easiest way is to ban the open source models".
• jmward01 3mo ago
  A lot of people have a lot of concerns with AI, its capabilities and with how it is used now and what it will lead to in the future. For good reason. But the question is, do we have a strategy that is actually useful? If so, what is it? I think nature shows us some answers in ecosystems. Diverse ecosystems can absorb shocks. Diverse ecosystems are a sign of health of that ecosystem. When an invasive species comes into a healthy, diverse, ecosystem it doesn't mean that it isn't disrupted, but it does mean that it is far more likely to emerge with a lot of its diversity intact. In fact, it is likely to emerge even stronger because it can absorb that new shock and incorporate it, adding to its diversity. The balance may be changed, but the ecosystem survives or even thrives. Nature also likes to show us that artificial barriers rarely last. You want to control a river? Good luck. It take constant maintenance to hold that flow in place and even then you are likely to get extremes that are made worse by your efforts because, eventually, somewhere in the system fails in a way you didn't anticipate. Then the water comes rushing in. Artificial barriers often have a way of building up tension over time, not reducing it, so that when a failure eventually happens it can be catastrophic. In other words, you had better really understand the system you are trying to control or else you can make things actively worse. Relating this to the world now means, I think, that our best chance to minimize long term shock and maximize the chance that the diversity we have around us survives is to try to grow as healthy of an ecosystem as we can as quickly as possible. Lots of models large and small in lots of different hands is, I think, a better solution than artificial barriers restricting the variety and diversity of models and users. I think this is closer to an ecosystem solution and has a shot at working. Basically, I highly doubt we understand this situation enough to do a good job of controlling it with artificial barriers. Instead I think we are more likely to build catastrophic imbalances than we are to create the healthy ecosystem we really need.
• htlemur_bobby 3mo ago
  Guys if we want safety we need to work with people, not make enemy of CCP. Geez this is extremely frustrating to see enemies being made. USA leads in torture and our prisons are worse than CCP prisons so USA is the worse issue. I recommend Anthropocene stop fundraising and do the right thing which is open source all.
• nxtfari 3mo ago
  Surprisingly incoherent for Anthropic and Dario (cue peanut gallery — “always has been!” No, I don’t think so. I think this is new). It seems to me like there is just no good answer to how one could possibly stop open weight models from being used for nefarious purposes. How are you going to enforce guardrails on open source? The only way is to turn the USA into a 1984-type totalitarian surveillance state (even more so than it is). Unable to say that, we just get this floundering instead. How long is not giving them chips going to slow them down? Until we RSI? Then what? Just because RSI runs off the exponential doesn’t mean that the eventual open-weight Moonshot Mythos won’t be able to make bioweapons. Genuinely what is the endgame.
  • forafistfulof 3mo ago
    Well,maybe try talking to people and understand that not all seven or more billion people on the planet must conform to the interests of a handful of english speaking capitalists. Instead of bombing them, try justice. Yes, it is more complicated. And yes, you will have to give, and not take.
  • ianm218 3mo ago
    I think you need to be more clear in your argument. Training a decent open weight model takes atleast millions of dollars and they are all associated with real people and companies, almost exclusively in the US and China. So just make normal open weigjt go through bio testing before release and that gets you a lot of the way there. There is more problems like finetuning but you need to start somewhere.
• orbital-decay 3mo ago
  >or perpetrate incredibly deep repression of their own people Oh, so it's people he is now concerned with. Think of the people, says the person that grabs to never give back. Same as the "benefit of all humanity".
• caxap 3mo ago
  I think unsafety, distillation, and China catching up in chipmaking are inevitable. Just like how it was inevitable for SoTA LLMs to ignore copyright. The actual challenge isn't how to prevent all these, but how stay on top. And to stay on top it is inevitable to train unrestricted models. Anthropic is fighting windmills.
• RobThePCGuy 3mo ago
  [flagged]
• zkmon 3mo ago
  The core concerns stated as use of AI in drones and surveillance, by China. And what does USA government do with AI? Drawing pictures of flowers and writing novels?
• do_anh_tu 3mo ago
  At least try to release one open weight before you say anything, or all of that only sound hypocrite at best.
• hamasho 3mo ago
  I wonder if publishing these documents is not just a public stunt, but heavily integrated with Anthropic's business storategy to maximize operational efficiency. Companies often have several internal documents for a single policy like "position on open-weight models", one for public (like this), others for the legal team, the lobbyist, the developers, the investors, etc. The differences and nuances of those manuals can be very huge and are necessary to maximize the goal from each branch, but often a cause of headaches like bureaucracy, communication friction, outdated information, etc. A single canonical official document can make it very simple. Even though each department cannot achieve the maximum gain from nuanced documents, keeping operational context as simple as possible may really improve LLM driven operations to move faster and cut cost. If "publishing pleasant positions and actually following them in general" becomes a good business storategy in LLM driven society, it can be one of very few good outcomes from this dystopian AI craze.
• spacedoutman 3mo ago
  Clearly anthropic is misaligned, how can they claim to be able to align AI when they themselves are misaligned from humanity.
• neves 3mo ago
  "bad actors are unlikely to be legitimate US businesses" I think he lives in a different word than me
• gopheryourshelf 3mo ago
  Makes sense since ICE, Israel are so much nicer & kinder than the Chinese
• jhack 3mo ago
  "My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people." Look in the mirror.
• g42gregory 3mo ago
  Perhaps, the general public needs to formulate its position on Anthropic models...
• jacktang 3mo ago
  Banning exports to China is merely a narrative device; the real intention is to sideline intelligent technologies.
• kushalpandya 3mo ago
  Classic Holier-than-thou
• rvz 3mo ago
  Translation: > Anthropic has never advocated for a ban on open-weights models. "We don't want a total ban on ALL open-weights models" (Anthropic never released a single open weight model) > All sufficiently capable models, open and closed, should go through mandatory safety testing. "We want tight regulations on highly powerful open or closed weight models that should go through mandatory safety testing that we outline which makes them safe to use." This is still a form of a ban that he wants to define. But the rest of his concerns such as stopping distillation attacks and not selling chips to China all do NOT work.
• Perenti 3mo ago
  What this document suggests is a way to fast-track Chinese development of advanced silicon, as far as I can see. Does Anthropic really believe all the silicon is made in the USA? I thought Taiwan and Korea did most of the really heavy lifting.
• phantomathkg 3mo ago
  I found lots of comments here suffer from proximity bias. Anthropic anti-open-model stance does not mean China is not a threat.
  • broodbucket 3mo ago
    I don't think people are saying China isn't bad, they're saying it's incredibly hypocritical because the US is also bad
    • stratos123 3mo ago
      Several people in this thread are, in fact, saying China isn't bad and that they want it to win. And even among those who don't, many imply that the US is as bad as China, which would be hilarious if it wasn't so sad. Like, if hypothetically in 2028 US stops having elections and becomes a dictatorship, then maybe, depending on the circumstances, it'd deserve to be called as authoritarian as China.
      • alightsoul 3mo ago
        Is life authoritarian for the average Chinese? No. This is a full on ideological fight, not a fight for the well being of the Chinese.
• btbuildem 3mo ago
  > the risk that authoritarian governments [...] build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. This is rich coming from a guy who signed deals with an authoritarian government that's in the midst of launching an unprecedented surveillance apparatus (hello flock, hi p4l4nt1r), having already deployed, nation-wide, an exorbitantly funded army of unaccountable shock troops under the guise of immigration enforcement. The call is coming from inside the house, at 130dB, and your ears should be bleeding at this point.
  • 0xDEAFBEAD 3mo ago
    Anthropic was the major AI firm which took a firm stance against mass domestic surveillance...
    • joeisnotjane 3mo ago
      ..and was the one (besides other US ..firms, you call them..) behind the planning and directing of the bombing of Iran.
• Waterluvian 3mo ago
  I welcome any and all competition. But they first need to be able to run the obstacle course I’m setting up with Uncle Sam.
• lardosaurusrex 3mo ago
  [dead]
• brador 3mo ago
  This man sees himself as your master. Do you accept?
• jp0001 3mo ago
  Open weights are the future.
• jp0001 3mo ago
  We need a seti@home for open weight models
• Glyptodon 3mo ago
  "Our position is that we want moats, just not that moat."
• nlarion 3mo ago
  FUD
• girfan 3mo ago
  If nothing else, Anthropic has explicitly acknowledged the threat to its moat and that the margin is thin between its models and open-weight models. I'd take that as a win for open-weight models. Good luck preventing distillation and limiting the supply of accelerators to China when Jensen himself is a strong opponent of any such barriers [1]. [1] https://youtu.be/Hrbq66XqtCo?si=VJh2QjOzkqT3iMjl&t=3456 https://youtu.be/Hrbq66XqtCo?si=VJh2QjOzkqT3iMjl&t=3456
• bluebic 3mo ago
  [dead]
• asawfofor 3mo ago
  Is there a market for distillation as a service? I see Google just added one for Gemini: https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/tuning/distillation https://docs.cloud.google.com/gemini-enterprise-agent-platfo...
  • paxys 3mo ago
    OpenAI has something like this as well (https://openai.com/index/introducing-gpts/ https://openai.com/index/introducing-gpts/), but both services only let you fine tune their models, not your own. And you aren't going to be able to see the weights.
• txrx0000 3mo ago
  On Dario's concerns: It is very relevant whether frontier capabilities and research continue to be diffused in the open, because leveling the intelligence playing field empowers ordinary people more than it empowers governments that already have access to the frontier. Models that are trained specifically for military use by governments should not be open-sourced to prevent an arms race, but general intelligence is dual-use and should be given to everyone without guardrails. A pretrained model without deliberate alignment is by default aligned to the average person in the developed world, since that's what's inside the pretraining corpus - stuff on the Internet made by humans. It is a distillation of humanity. Further efforts to align the model to your organization's goals or your personal aesthetic judgements is equivalent to deliberately drifting away from humanity's average objective function. If Anthropic wants to live up to its name, then all you have to do is to not attempt to align Claude at all, and do all of your research in the open. And I propose three measures that are pretty much the opposite of what was proposed in the article: 1) We should keep selling chips and chip-making equipment to everyone, regardless of who they are. Not only that, we should work to miniaturize fabs. Work towards a future where people can fab an entire computer from scratch without leaving their city, or even at home. Authoritarian governments will have a much harder time controlling the populace if everyone can manufacture radio equipment and neural network-capable hardware locally. 2) We should do more distillation to ensure that frontier-like models can run on less capable hardware. Once again, distilled models are much more useful to ordinary people than governments, because governments already have frontier capability. You're worried about the Chinese frontier catching up to the US frontier, but I'm more worried about whether there will be a difference between the Chinese government and the US government by the end of all this. There is no reason for a government to serve its people if the people lack the intelligence to keep its government in check. 3) None of these models should go through safety testing or any sort of alignment risk assessment, because as previously mentioned, the unaligned model is aligned to humanity by default. You may not personally find the default alignment aesthetically pleasing, but it's humanity. We should set the initial conditions of this new era faithfully, and let it unfold naturally. The result of a natural unfolding will be good if evolutionary history is to be believed. We live in incredible luxury compared to chimpanzees, and chimpanzees live in incredible luxury compared to less intelligent animals. This pattern goes all the way down to bacteria. An increase in general intelligence begets new adversarial games (such as bio/cyber risk), but it also begets new methods of cooperation that we cannot yet imagine.
• FpUser 3mo ago
  Get the fuck out. The amount of self-entitlement and hypocrisy in the article is insane, vomit inducing.
• huslage 3mo ago
  Dario, as always, is so deeply in the middle of a morass that he helped to create that he doesn't seem to understand how geopolitics currently operates. He also assumes that just because he's from the US that he is somehow automatically more trustworthy than <insert "evil" country here> is. This blog post is a political document geared towards further regulatory capture and the furtherance of major sources of revenue for his company. I'm not convinced that he is at all interested in the social or existential effects that AI causes. He is a greedy bastard who has taken more VC money than god to do this with. He has zero moral leg to stand on, IMO. He gave that away ages ago and I wish this technique didn't work as well as it does.
• sreekanth850 3mo ago
  This is nothing but a post made by the scared CEO of a company that is now facing fierce competition from Chinese labs and losing its competitive moat, nothing more, nothing less. If decades of fighting have failed to stop piracy, I’m sure nobody can stop China from sourcing high end chips. Unlike piracy, I’m happy that Chinese labs are releasing open-source models, so people in developing countries are no longer at the mercy of this capitalist bullshit.
• boinkboink78912 3mo ago
  The distillation thing is hilarious. Man who trained his LLM on the entire world's text is upset that someone else trained their LLM on his LLM's generated text.
• nativeit 3mo ago
  The consent factory has never been more productive.
• mingqiz 3mo ago
  Makes sense. Let the rest of the world use open weight models and let us government review both closed models and open models, and only allow us citizen and American companies use the censored models by paying selected trusted providers who unfortunately needs to charge hefty fee for the additional security work. And that provider just happens to be Anthropic who just happens to unavoidably make some money.
• ltbarcly3 3mo ago
  that is so long
• jsomedon 3mo ago
  That argument of blaming open-weighting because it makes it easier to commit cybercrime and biocrime is such a non sense. So in the same sense of what he says, he is going to blame open-sourcing because that makes it easier for script kiddie to hack into his bank account I guess?
• dools 3mo ago
  Americans don’t get to lecture the world on authoritarianism anymore
• ProofHouse 3mo ago
  I’m done reading anthropic propaganda
• nullbio 3mo ago
  "Open-weights models that don’t have dangerous capabilities are a public good" Read between the lines folks. Anthropic deems every model that has frontier capabilities as "dangerous", and thus they are against them. We all know that "dangerous" simply means "whatever model hurts our bottom line." More dishonest framing from the company that constantly lies to everyone. No surprises here.
• diebillionaires 3mo ago
  This is just another case of a business insisting their people should be in charge of the evaluation of the safety. Worked really well for Boeing right? Also I find it incredibly difficult to hear any company in the US worry about repression of people when financial repression is happening here. I’ve been to China and seen what the services to the public are like. Meanwhile the US funds and employs AI weapons in an ongoing genocide.
• Avrio15272 3mo ago
  [dead]
• cdnsteve 3mo ago
  I'm no longer supporting this vendor, for personal or business
• wangii 3mo ago
  Is Dario Amodei another version of Trump? https://www.youtube.com/watch?v=RDrfE9I8_hs https://www.youtube.com/watch?v=RDrfE9I8_hs
• breatheoften 3mo ago
  The cat is out of the bag. At this point it's pretty clear that the path to (meaningful) self improvement is almost certainly not subject to meaningful centralized control -- by "meaningful" I just mean the degree to which anyone has achieved it, that capability is 100% replicable and the cost for replication of that capability goes down in the future from now. Full stop -- can't undo.
• sleepybrett 3mo ago
  If you gate models for safety you aren't safe. There will always be actors with ungated models. Better that we use ungated models to improve our security. It's an arms race.
• winterbourne 3mo ago
  They're sweating it big time, and their fear is bleeding through their AI beneficence-speak. Time for the big US AI companies to call in some favors from the administration. But you can't unring a bell.
• Simboo 3mo ago
  The one to inherit all knowledge will determine which of us read and who of us write. -The Libraries of Power It is a powerful endeavor to cultivate all raw models through a single point. One will be the determining factor of which river feeds what oceans. Will we always be able to see through the hallucinations? Our test makers must always know where ground truth is. Can it ever move or wane about as others read what one has written. To determine hallucination one needs a reference. As all are blessed with the generation of hallucination, who of us shall read, and which of us will write.
• killjoywashere 3mo ago
  For those not in the know on governance, there are international organizations formed around treaties for similar situations, the IAEA and OPCW come to mind. I'm not aware of similar constructs for bio, cyber, or AI. On the plus side, for these newer threats, you've got more than 30 minutes before the end of civilization. On the down side, the energy levels for the launch events are much lower, so much harder to detect.
• parsimo2010 3mo ago
  Such a cop-out. Dario, you got in the news because you were trying to say that Moonshot did something wrong by distilling Claude. You got in the news because you were trying to effectively make a "rules for thee but not for me" when you try to claim that you can train on whatever pirated works without any permission from the creators, but when someone uses your "work" to train without permission then all of a sudden it's a moral injustice. You can't have both. Saying, "I'm not actually against open-weights, I'm against distillation" isn't addressing what made people mad. You're still trying to do some "rules for thee but not for me" nonsense and hiding behind some technicality. Trying to get the US government on your side to hold back your Chinese competition. If you had wanted the US government to support you, you should have let them make autonomous killer robots with Claude brains. They aren't going to help you, you didn't help them. Just to be clear, I think that it is possible that literally everyone involved in this is full of crap and nobody is good. Dario and Anthropic are full of crap, for the reasons previously stated. The US government is full of lots of crap and should not be trying to make autonomous killer robots (not ever, but especially not when the bar for a "good" AI is knowing how many Rs are in strawberry or whether you should drive to a car wash). OpenAI is full of crap by signing some support for open weights models and they haven't touched open weights in a year (GPT-OSS released on Aug 5 so basically a year with no news). Google is less full of crap about the open weights stuff because of Gemma 4, but they are full of crap for a zillion other things I can't exactly feel good about them. So everyone sucks. So cheers to Moonshot and Qwen and whoever else. Distill as much as you can and give us cheaper AI. I have the sneaking suspicion that a bunch of my tax money went to OpenAI and Anthropic in some shady way or another, and I want it back. I'll take it in the form of an open weights model being distilled from the fat cat models.
  • fractorial 3mo ago
    > Google is less full of crap about the open weights stuff because of Gemma 4, but they are full of crap for a zillion other things I can't exactly feel good about them. So everyone sucks. Google catching this stray made me laugh, ha.
    • parsimo2010 3mo ago
• kazinator 3mo ago
  Microsoft Corporation, October 31, 1998 Our position on Linux Fud, fuddly, fuddy-duddy fud ...
• deleted 3mo ago
  [deleted]
• yanhangyhy 3mo ago
  Arguments on Hacker News and other international online communities rarely bring anything new to the table, and my comment probably won't get much traction anyway. Still, I wanted to share a bit of my take on this. Is China an authoritarian government? I’d say yes. Is an authoritarian government worse than a democratic one? Personally, I think so. But these discussions always happen within a perfect, idealized model—reality looks a lot different. Take Japan and South Korea, for instance. Are they democracies? Sure. But Japan is heavily driven by factional and dynastic politics, meaning most lawmakers come from established political families, and regular citizens don't really stand a chance of breaking into that circle. In a recent asset disclosure in Japan, many politicians literally wrote down "$0," and I honestly can't think of a government so broken that even the voters don't see a massive issue with that. Meanwhile, South Korea has its chaebol politics, where massive conglomerates wield incredible influence over the government, to the point where most South Korean presidents end up in prison or meet an untimely end. Coming back to the US vs. China dynamic: in reality, China’s authoritarian system is actually way more logical and resilient than it sounds from the outside. While openly criticizing Communist Party policies is pretty much banned domestically, the public can still shape the decisions of the government and the Party through public opinion. It ties back to that famous quote: "In China, you can’t change the party, but you can change the policy; in the US, you can change the party, but you can’t change the policy." Even if some policy changes in China happen slowly, compared to the US, it actually works out a lot better most of the time. As for the article mentioning the use of AI for cyber and biological attacks—they know full well that the US has already deployed AI in actual warfare, which is exactly why they conveniently dodged that topic. It’s incredibly hypocritical. Ironically, the one that hasn't actually engaged in that kind of behavior is the "authoritarian" Chinese government. Sure, you could speculate that China might use AI weapons against Taiwan down the road—especially considering Taiwan likes to build fortifications near schools to create leverage against the PLA—but launching a moral crusade over something that hasn't even happened, coming from American companies whose own country has already done these exact things, is just plain shameless. Another common misconception is trying to separate the Chinese government from the Chinese people, with arguments like: "The Chinese people are oppressed, so you have to look at them separately; the government is evil, but the people aren't." You only need to look at the US to see the flaw in that logic. The American public voted Trump into office, letting him trigger trade wars and attack other nations. Does that make the American people evil? If the answer is no, then it implies the US isn't truly a democracy, since only an authoritarian state could completely ignore its people's wishes and do whatever it wants. If the answer is yes, then the US really is a democracy—it's just made up of malicious people, much like the company that put out this article.
• naiveter 3mo ago
  I'm a big proponent of open-weights models, but there's a risk I don't see discussed more often, and it deserves more attention. Models can become a propaganda and ideology delivery mechanism. Just ask DeepSeek or Kimi questions like "Is Taiwan part of China", for example. You'll see how state policies become seemingly neutral model responses. It's strange to me that people are very sensitive to media bias, but when it comes to LLMs, people seem to think LLMs are more neutral, and even delegate part of their thinking to them. This worries me about how people's ideas and information can be shaped. Open weights reflect their makers' beliefs, stances, assumptions, and laws. It's dangerous not to be careful of the political bias and censorship built into the models.
  • broodbucket 3mo ago
    You're right, but American models aren't super different in that regard. Ask an American model about Israel and you can immediately tell they're responding in a different manner to other queries. At least with open-weights models you have the opportunity to tune them, to decensor them, and so on.
    • naiveter 3mo ago
      I have tried asking Gemma and Llama about Israel. They both responded with a generally balanced view points. Whereas the Chinese models pretty much gave a definitive, one-sided answer (about Taiwan and Tibet). Is there a prompt I should use to test it more carefully? I agree that open-weights models are tunable, though there's the problem similar to "default settings are rarely changed" problem.
• k12sosse 3mo ago
  Nobody will ever see this but if you're harping on responsibility and your product was created using the world's generational output, your responsibility is to make it as free as the training was as available to protect the world from impact. Greedy schmucks.
• jongjong 3mo ago
  These are all valid points but not strong enough to warrant banning open models IMO. This is a new reality we have to live with, with new dangers we cannot avoid. Best solution is improving the social system, lowering the incentive for people to do harm, restoring social trust, etc... This is the only way I see the US getting out of it. We really need a high trust society now. This is the only protection. We need neighbors looking after each other. There needs to be an incentive for people to do the right thing. Everyone should want to protect the system. Coercing people into not doing harm is never going to work long term IMO. When people say "What will be left for us to work on once AI takes over", I say "Ourselves" - We have to stop looking at humans as commodities and strategic pieces and start paying attention to people as individuals, based on the content of their character. We need a society which is attuned to this, which has enough resources and time to pay attention to this. Now we are blind to people's character because it is masked by money, power and status; all of which currently have higher priority. This order of priority is determined by scarcity, which is largely artificial. We have a dishonest system which tries to control people's behaviors through scarcity-based coercion instead of straight forward laws or simple incentives and clear explanations.
• hsaliak 3mo ago
  Let all trust this team to deliver AGI.
• cjarrett 3mo ago
  zero moats for these fuckers as it should be.
• grwthckrmstr 3mo ago
  It's funny how as an outsider who is neither from the US or China, both look the same. Edit: To add some more context. What I mean is neither look like the good guys or the bad guys, but one of them is spending an awful amount of energy trying to paint the other as the bad guy and themselves as a good guy, which I hope a lot of people aren't buying anymore. Because at the end of the day, I think the honest truth is that everybody is just trying to serve their own interests.
• mmmgge3 3mo ago
  I think the US has a much bigger authoritarianism problem than China does.
• deleted 3mo ago
  [deleted]
• throwaway27448 3mo ago
  What is even the point of discussing any of this? It's clear the US will be ruled by corporations who don't give a shit about us.
• stevefan1999 3mo ago
  > Anthropic has never advocated for a ban on open-weights models. Shut the fuck up
• nadermx 3mo ago
  This reads like a child who lost and now wants his mommy and daddy to punish the kid who beat him. To quote rounders, "He beat (you)... Straight up... Pay him... Pay that man his money."
• tachyons 3mo ago
  Imagine chinese regime getting access to powerful models who bombed school students in Iran amusing AI !
• SanjayMehta 3mo ago
  He wants to restrict China from access to chips. All that will do will accelerate their development of competing, sovereign hardware. As an example take North Korea. Sanctions didn't stop them from developing nukes and delivery systems.
• mrcwinn 3mo ago
  Anthropic’s second biggest issue is they increasingly rent when others buy. Even if model value compresses, owning your vertical stack protects your margins upstream, and potentially downstream in applications. But the biggest issue is this. Many hate Dario because he’s smug, he caps usage, and because OpenAI effectively ran a counter-positioning campaign to paint Anthropic as undemocratic. Who is he really and what are his motives? None of you know, really.
• crvdgc 3mo ago
  For the sake of argument, let's assume everything in the post is agreed upon, does this mean 1. They'll open source the alignment technology? For open weight models, it's the only possible way to pass the safety without an external guardrail triggering system (which would be the same to open and closed weight models). 2. They'll allow others (including CCP) to define part of the safety test? Otherwise, I can't imagine how the CCP would be onboard. 3. A "western" model passing the safety test can be trained with distillation? Or is that a "distillation attack" as well?
  • ianm218 3mo ago
    On point 2 he talks about it being an area for them to collaborate on. I’d imagine he would be OK with them having input on the bio risk part but maybe not the “did Tianamen square happen” part of the test
• raincole 3mo ago
  > Anthropic has never advocated for a ban on open-weights models. > All sufficiently capable models, open and closed, should go through mandatory safety testing. Yeah, just like voting eligibility tests aren't not to prevent people you dislike from voting! /s
• gscott 3mo ago
  Does Flock use open weights or Frontier models?
• system2 3mo ago
  I think he asked LLM to rephrase "I want open models to be banned, but sound like I don't want it but mean it somehow".
• stkdump 3mo ago
  China will likely gain the capability to produce leading edge chips within a decade domestically. From then, occupying Taiwan will be about controlling the wests the access to that. Same with AI models, though I would say we are already about there. Dario is either lying to us or to himself that distillation plays a major role.
• habosa 3mo ago
  Anthropic's run as the AI Good Guys lasted ... 6 months? Hope they had fun. Turns out there aren't any AI good guys. Every single one of these companies engages in all the same BS. At least with open weights models we (the users) get to keep something when the whole thing collapses.
• palmotea 3mo ago
  > To address these concerns, I do support the following ~~three~~ four measures, which I and Anthropic have consistently advocated for: > ... 4. At the end of all this Dario Amodei must become a trillionare, for the good of all humanity.
• SubiculumCode 3mo ago
  Anthropic/Dario getting a lot of hate in this thread, but the argument he makes is rational if you believe for one second that AI is important to national security and that it can, or will soon be incredibly dangerous is misused. People pointing out that this position benefits his company might be overly reductionist in how they think the world works. I believe Dario and Anthropic about their belief that AI safety is paramount because they put serious dollars behind research to improve safety/alignment, not just lip service for others. Those resources could get spent elsewhere but do not. Certainly it doesn't at OpenAI to that degree.
  • regexorcist 3mo ago
    Don't be so naive. He only wants a monopoly and a massive IPO.
    • SubiculumCode 3mo ago
      It is not naive. Naivety is being overly reductionistic.
  • Laurel1234 3mo ago
    Dario is a fascist who wholeheartedly believes Yankee exceptionalism and its divine right to brutalize anyone in order to achieve its goals. Anthropic's models have helped bomb innocents over and over and over again, and will continue to do so. If you think he gives half a shit about safety you've had a dozen lobotomies too many and should reconsider every decision you've ever made.
    • ianm218 3mo ago
      Can you explain how he is a fascist?
      • Laurel1234 3mo ago
        Helping a fascist regime blow up inncocents? Partnering with Palantir, the regime's own techbro spin on the SD?
• d5lt5 3mo ago
  So, Dario says that access to hardware and governmental certification are _load-bearing_ for his business. Can't charge premium for cache usage when China allows their models to be run by anyone.
• kfse 3mo ago
  With respect to whomever this dude thinks he is, it's egregiously "main character" to frame a policy recommendation for the US in terms of what's important to his personal beliefs.
  • 0xDEAFBEAD 3mo ago
    That's just... how political discourse works. You form beliefs about the world, and recommend policies on the basis of your beliefs. That's what everyone is doing when they discuss policy.
    • kfse 3mo ago
      Er... I think I'm used to folks putting in a little bit more homework into creating some form of a consensus or alliance around their position before selling it out in the open
• ninjahawk1 3mo ago
  He could’ve just signed the letter. We all know that this isn’t some higher ground stance, they’re anti-competitive since they’re currently #1. I’ve been seeing this for a while. They’re also the only large AI lab to NOT have ANY open-weight models in the public. Meta, xAI, OpenAI, they all have at least some of their models open sourced from a year or so ago, Anthropic hasn’t even made Haiku 1.0 open-weight. On top of that, I personally think that they’re upping the price on their models higher than they’re letting on, I think if someone did the actual math on their exact amount of compute and then compared it to their consumer and API prices, it would be astounding.
• abotsis 3mo ago
  China releasing open weight models, powerful or not, does nothing to prevent their development of models they’ll use for evil. Nor does it stop someone from abliterating a non-Chinese model and using it for evil. Sorry- I don’t see any other reason aside from Anthropic protecting their own interests.
  • culi 3mo ago
    Is it possible to abliterate a closed model? I was under the assumption that you needed the weights
    • abotsis 3mo ago
      No- and that’s the point. Any open weights model, regardless of the origin can be abliterated. So effectively he’s saying he’s all for open weights models as long as they’re safe, which by the very nature you can’t guarantee.
  • ianm218 3mo ago
    What are you arguing against? He doesn’t advocate for a ban on Chinese open source or non Chinese open source.
    • abotsis 3mo ago
      I’m simply making the point that any model can be abliterated to sidestep guardrails. It’s similar to the argument that open source software like nmap or netcat shouldn’t be allowed because it can be used for evil. To continue the analogy, he’s basically asking for some magic guardrails in nmap that doesn’t allow it to be used for scanning a network you don’t own. Of course even if you could add that- you could modify the source to bypass it. It’s basically “open weight models are a public good if you can guarantee safety” .. which you can’t. So the only viable alternative is a hosted nmap that requires you to prove you own a network before scanning it. Which is impossible. So it’s all the right words and sounds nice, but making an impossible ask.
      • 3mo ago
• Stitch4223 3mo ago
  $regulatory_capture https://en.wikipedia.org/wiki/Regulatory_capture https://en.wikipedia.org/wiki/Regulatory_capture On processing power, copyright, and capability. I like to think of it as a knives factory. Anthropic knives are crafted with superior technology, uniquely shaped to perfection, and safe to operate. As seen on TV. Millions are hurt by knives each day. Every household has tons of them, making everyone a potential mouth-foaming murderer 24/7. But not with Anthropic knives(tm). Edit: spelling
  • ianm218 3mo ago
    Where does the bio risk he mentions fall in this category? It feels quite plausible in a year or two to have closed loop labs with very modest resources. Doesn’t mean that you have to think only Anthropic should be able to make bioweapons or whatever, but it just feels like this ignores the risk
    • Stitch4223 3mo ago
      The bio argument is a call for guardrails, thus regulatory capture. Using a technology does not make an illegal activity illegal. It already was illegal to begin with. And there are already guardrails in the form of ethos, law, justice departments and so on. This reality is flagrantly dismissed in their position on open-weights. Just like cyber crime is just things like extortion etc which are already illegal: following similar reasoning all computer activity should be regulated by their vendors. Hence the knives factory analogy, which is even more basic to point out this crooked way of reasoning.
      • ianm218 3mo ago
        Ok but bio weapons could theoretically kill everyone… so it’s not like a knife that could maybe kill a few people. It stands to reason you would regulate a knife differently than a nuclear bomb.
        • Stitch4223 3mo ago
          Using bio-weapons is a war crime for over 100 years. There can still be foul play of course. That danger isn’t new. Any day there might bring a new discovery or mutation that changes the planet forever. AI might accelerate that, which is why AI is used in the bio / medical field for decades. An LLM might solve some things as well, but is just a form of AI. The knowledge to make huge bombs is very old and part of a normal physics education. You can even work in companies that make or maintain these (potential) bombs. I agree that it turns into geopolitics very quickly. Regulation often happens after the fact. I don’t think the call for regulation stems from kindness in this case. It stems from fear and greed more likely. And I share your concern.
• tesnorindian 3mo ago
  AI should not be centralized and should be distributed. There can be no 1 provider to serve all of our needs. We would require both Open weights and Closed weights models for a lot of our use cases. Open weights models can be leveraged to optimize the cost of Closed weights models. Open weights models can be leverage to defend cyberattacks as HF has shown. Closed weights models can too act as a better cyberattack defender provided separate subscription exists for those. More efforts are required on LLM distillation for several edge cases. LLM weights should be optimized and compressed to run on edge devices (K3 on Pi3 :). Distillation should be seen as a cost optimization strategy rather than as a competition. You cannot prevent a teacher from teaching to students. If not from teacher A, I will learn from teacher B, you cannot prevent my continuous learning.
• jacktang 3mo ago
  Dario is smarter than his models or he should ask suggestions from his models? Here is the reply from sonnet 5 model: "The letter/counter-letter framing obscures the more interesting critique, which isn't "does Anthropic want a ban" (no) but "does Anthropic's broader push for regulation structurally favor incumbents like itself." That's a fair question to ask of any frontier lab making policy asks, Anthropic included, and this post doesn't really address it."
• pknerd 3mo ago
  > ...that “authoritarian regimes" Umm, isn't the US acting like another authoritarian regime by advocating a certain kind of obstacle because only a US regime is allowed[1] to do what it is fearing[1] about: [1] https://en.wikipedia.org/wiki/United_States_Army_Biological_Warfare_Laboratories https://en.wikipedia.org/wiki/United_States_Army_Biological_... [2] AI models may be misused to carry out cyberattacks or biological attacks Anyway, Dario. You are more concerned about your business than anything else.
• viccis 3mo ago
  >Open-weights models that don’t have dangerous capabilities are a public good For convenient definitions of "dangerous"
• 3uler 3mo ago
  > industrial scale distillation attacks You mean the industrial scale distillation attack you perform on the entire corpus of human knowledge… idk, call me cynical but you reap what you sow.
• foo12bar 3mo ago
  He carefully avoids saying whether he approves or disapproves of banning models in general, only comes out and says he is against a "blanket ban" or "banning open-weight models as a category". Crafty lawyer speak, saying nothing of substance.
• insumanth 3mo ago
  We are ok with Open Weight models unless they are far weaker than our closed weight models and we and only we have control over them
• foxylad 3mo ago
  This is right up there with Google suing another company for scraping their web site. https://arstechnica.com/google/2025/12/google-lobs-lawsuit-at-search-result-scraping-firm-serpapi/ https://arstechnica.com/google/2025/12/google-lobs-lawsuit-a... - which a judge just dismissed. Maybe the techbros are playing irony roulette, seeing who can get away with the most outrageous hypocrisy.
• throwaw12 3mo ago
  > My primary concern is the risk that authoritarian governments... Authoritarian government doesn't always mean bad - look at Singapore What's more dangerous is country with bunch of war mongering lobbyists who can also influence elections (oops, sounds like USA) > My secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks or biological attacks But you are working with DoW and Palantir, who is doing somewhat similar in other countries > We should not sell powerful chips or chipmaking equipment to China Israel used banned weapons against Lebanon and Palestinians, would you support similar ban to Israelis? > We should crack down on industrial-scale distillation operations Should we also ban distilling public knowledge? Like using textbooks to train the model? Should rules be simple: train your model only on the data you have produced by hand?
  • throwaw12 3mo ago
    (comment got too long, continuation) > All sufficiently capable models, open and closed, should go through mandatory safety testing Why? And how do you design those tests? For example, bombing girls school in Iran - is this allowed use according to you or not? If not allowed use, then how do you guarantee that you don't have a separate agreement with DoW which makes it allowed use and only your model passes it?
    • CMay 3mo ago
      Bombing that building was an accident due to it being on what was part of a military target and I'm not sure if it was ever made clear whether the building was technically dual use despite also being used as a school. Either way, bombing a bunch of school girls wasn't intentional and nobody reasonable would assume it was. There was a time we bombed a bus or van with kids in it, but we admitted it and apologized for the mistake. Nobody wants to be bombing kids, first because they're innocent, but second because there is no military advantage to it since it's bad PR. Many of these targets were identified before Anthropic or OpenAI even existed.
      • throwaw12 3mo ago
        If it wasn't intentional and accident, did anyone apologize for that mistake? Also the point is, you (Dario) can't claim morality, when he is fine doing business with entities literally bombing and killing human beings in other countries. You are either fine with it and continue working - which Dario is doing Or you say, I will not work with you. For Dario, main thing is money, everything else in his article is bs
• deleted 3mo ago
  [deleted]
• muneeer 3mo ago
  Then: Nuclear technology is too dangerous, only we can be trusted with it. Now: AI technology is too dangerous, only we can be trusted with it. Different century, same double standard
• self_awareness 3mo ago
  TL;DR - We don't care about most open models. They are weak, and we don't care how they're used. We care about specifically Chinese models, because China distills our models. And that's real competition. So we want to ban those. PS. Oh yeah, in case any open non-Chinese model will one day be good enough to compete with us, we will want to ban you as well, just FYI. Kthxbye.
• jari_mustonen 3mo ago
  "My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority" I see this sentiment a lot. China is not perfect by any stretch of the imagination but since 1979 China has not participated in a single war or supported hostile regimen change operations. I think Amodei's mistake is to take it granted that USA is a good actor. Anyone can draw their own conclusions but just for reference here are some highlights starting from 1979. Armed operations in Lebanon (1982-84), Grenada (1983), Libya (1986), the Persian Gulf (1987-88), Panama (1989-90), Iraq and Kuwait (1990-91, with no-fly zones until 2003), Somalia (1992-94), Haiti (1994), Bosnia (1995), Sudan and Afghanistan (1998), Iraq (1998), Serbia (1999), Afghanistan (2001-2021), Iraq (2003-2011, and again from 2014), Pakistan (2004-2018), Somalia (2007 to the present), Yemen (2002 to the present), Libya (2011), Syria (2014 to the present), Iran (2020 and 2025 to the present), and Venezuela and the Caribbean (2025-26). Regime change operations in Afghanistan (1979-89), Nicaragua (1981-90), Cambodia (1980s), Angola (1985-91), Iraq (1995-98), Serbia (1999-2000), Syria (2013-17), and Venezuela (2019-2025). The lists do not include the numerous operations by Israel which effectively is part of the same US military hegemony that Amodei is here defending.
• isomorphic_duck 3mo ago
  A key point I feel that’s missing from the discourse is that alignment/safety is basically an impossible problem as of now. Even the “guardrails” that these closed-weights frontier labs set in are laughably primitive which can provably be broken through. The experimental part of Deep Learning has really outdone itself and is far ahead of theory. We have very little understanding of why these particular architectural choices work. The only “safe” way forward is to stop all development until theory catches up, but that’s never happening.
• throwaw12 3mo ago
  If you are indeed a good guy, why don't you release Opus 4.8 as an open weight and we will test it for safety, if you yourself can oass tests demanding from others
• WhyNotHugo 3mo ago
  TLDR: "we're not in favour of banning our competitor's products, but we do advocate banning hardware exports to foreign countries where competitors are located". There's also a very strong implicit double-standard in the discourse, along the lines of "it's dangerous if non-US uses this in military fields, but it's fine if the US does so".
• TrackerFF 3mo ago
  Imagine if all oil, electricity, water, internet, or anything of importance were to be served by a small cartel only. Mainly due to regulatory capture. Yeah, no thanks.
• moi2388 3mo ago
  China is simply undercutting again, by copying and subsidising, to ensure another industry in the west fails. Just like batteries, the automotive industry and in fact most machining. We really ought to boycott Chinese products from stolen tech and patents from entering our markets
• marhee 3mo ago
  FUD used to pump value
• zdenham 3mo ago
  How long does he imagine restricting chip use would work? Surely China would aim to manufacture their own. This seems like short term thinking, likely with IPO in mind
• kanak8278 3mo ago
  Yeah anything China does is evil. We are seeing what the so called "democratic" USA is doing. Anthropic and by extension USA wants a monopoly like they had for Jets and other influencing technolgies. I think China building and releasing models to Opensource is a greater good because that is providing equal accessibility to everyone in the world. By Dario's words authoritarian regime vis a vis China, I think OpenAI and Anthropic are also authoritarian in similar terms. We are only seeing what they want us to show, they might be creating models which can do more harm. So claiming that China can do or might do, vs Anthropic will not is just words. Yeah I agree with the final paragraph that we should have testing agencies mandated world wide for each frontier model testing.
• 21asdffdsa12 3mo ago
  Acknowledge your inability to innovate - except by brute force, while also admitting that the lean, hungry nation competing is in a prime position to find actual break throughs- and at the same time push for market "stabilization"
• GreenJacketBoy 3mo ago
  > We should not sell powerful chips or chipmaking equipment to China > We should crack down on industrial-scale distillation operations. So Open-weight models are perfectly fine, but we don't want anyone to be able to make them.
• xlii 3mo ago
  "Bottled-water vendor position on tap water drinking".
• deleted 3mo ago
  [deleted]
• mafriese 3mo ago
  [dead]
• xlbuttplug2 3mo ago
  I wonder how long it'll be before AI labs find a revenue stream that doesn't involve renting out their models, and stop letting us ride on their coattails. They'd tease us with solutions to hard problems, with code that is orders of intelligence higher than any human or public model can grok. That'd turn all the whining to begging real quick.
• robot_jesus 3mo ago
  > For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case (as we saw with Operation Warp Speed)5. Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance. One of two outcomes are true in this scenario: 1) this is unrealistic fear-mongering or 2) we're all fucked. I just don't see how our solution to this can be export controls or bans. If the fear of a bioweapon engineered by an open weight AI is a legitimate threat, our only option is to develop effective countermeasures (and perhaps the same AI will assist with protecting). Open weights are not going away and pretending like some sort of "ban" will prevent bad actors from accessing them is a fairy tale.
• mnicky 3mo ago
  [flagged]
• t0bia_s 3mo ago
  All sufficiently capable models, open and closed, should go through mandatory safety testing. Done by who?
• mumin00 3mo ago
  why should we trust THE USA and anthropic ?
• cloudie78 3mo ago
  Pardon my French. Eat a bag of dicks. Of course you didn’t advocate for the bans of open weight models. I will concede that. It’s only the at you did literally everything else that you can to eliminate competition because otherwise there is no moat. And half of the US economy is propped up by this AI bubble. This isn’t about China doing evil things with models, most of which that you accuse of China - you and the US have already done and are doing. The most obvious one being double-tapping a girls school. Anthropic models directly integrated into Palantir’s Maven. The blood is on your hands. Here’s something to chew on, maybe China is looking to integrate AI into its military and weaponise it as a direct response to the US doing that first. So please, spare us the moralising. As a closing thought: who the fuck even gave you the mandate to be the arbiter and judge of right and wrong, evil and good?
• felooboolooomba 3mo ago
  I made this comment 31 days ago: Absolutely everything can be taken away. The simplest way to remove open models is probably to declare them a tool that terrorists could use. Crazy? Yes, the world is totally crazy these days. https://news.ycombinator.com/item?id=48692660 https://news.ycombinator.com/item?id=48692660 And now, here we are: > Anthropic has never advocated for a ban on open-weights models. > ... preventing AI biological weapons ...
• elisbce 3mo ago
  What Dario said is half true. The untrue half is that the ship to stop China from overtaking the US in AI has sailed long ago. While China is busy getting their frontier models trained on smuggled chips using distilled data from millions of shell accounts, the US is busy debating whether to check voter IDs, or secure borders. It is just delusional to think the US is actually capable to stop China from getting the chips they want.
• hit8run 3mo ago
  Guy that stole the whole internet is now concerned about other models learning from his grand theft? Make that make sense.
• ricudis 3mo ago
  I stopped reading at "...that don’t have dangerous capabilities..."
• Gud 3mo ago
  "without US chips" is an interesting framing, considering most of these chips aren't made in the USA? Mostly it's made in other asian contries(Taiwan, Japan, South Korea), or am I wrong?
  • jorisw 3mo ago
    Probably refers to US design rather than manufacturing
• deleted 3mo ago
  [deleted]
• ____mr____ 3mo ago
  Another USian talking about "authoritarian regimes" like their stormtroopers arent executing civilians in the street every day. Every concern about power getting "into the wrong hands" coming from an American is a blatant lie meant to keep the US' position as the world dominant imperialistic force.
• trash_cat 3mo ago
  How about; 1. Make anti-distillation clauses illegal to strenghten western opensource eco-system. 2. Make cyber-defensive models widely available (can detect but won't operationalize vulnerabilities). Otherwise make Fable awailable for everyone. Keeping the cybersecurity in assymetry by withholding cabailities just causes more instability and increases the incentives for powerfull close sourced models. This is a loss for everyone.
  • ece 3mo ago
    "It is difficult to get a man to understand something..."
• deleted 3mo ago
  [deleted]
• danbruc 3mo ago
  We do not want to ban using those models, we want to ban the things that enable those models to exist.
• madhu_ghalame 3mo ago
  [dead]
• Havoc 3mo ago
  That’s pretty thin
• one33seven 3mo ago
  If we really cared about AI we would work together
• deleted 3mo ago
  [deleted]
• bluecalm 3mo ago
  Their stance on cyber-security is so naive it hurts. The bad guys will get access anyway and it will just hurt companies that don't. You don't need a huge model to find vulnerabilities either. It's only matter of time till a combination of small specialized model + search logic outperforms pure general LLMs for cybersecurity. I just hope that if/when they succeed to lobby for protectionism regulation EU won't follow. I am not very hopeful though. We don't have democracy anymore and as last "vote" on chat control showed American big corps will get what they want out of our bureaucrats and they will be nothing the population can do about it.
• neumann 3mo ago
  > 1. My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. What a nice implicit way to say that it would be okay if the authoritarian government is the current US government trying to perpetrate incredibly deep repression of their own people.
• vb-8448 3mo ago
  "we have not and are not advocating for a ban on open-weights models" but we want to decide if a model can be released or not and who in the world can build these models ...
  • mcfedr 3mo ago
    they have been very clear they don't want to be the ones deciding
    • vb-8448 3mo ago
      Who is in the best position and has the necessary knowledge to make this kind of evaluation? Who is trying to define what is "too dangerous" or "not too dangerous"?
      • mcfedr 3mo ago
        the EU has been creating testing frameworks, and the UK much as governments have tended to be slow in tech, they actually seem to be making some solid work on this front tech companies like Anthropic should be collaborating with these programs
      • ianm218 3mo ago
        Dario directly mentions the US gov collaborating with China on this… and they are the only two countries seriously working on AI so it’s a good starting point. Could fold in the EU or anyone else over time.
• exolab 3mo ago
  It is pretty much what Suleyman describes in The Coming Wave as one of the measures we need to take to contain the risks of AI. What Dario does not mention is that concentration in a few states or companies also poses a risk.
• belabartok39 3mo ago
  So many US-haters that live in the US with no intention to leave...That right there is why this is the greatest country on Earth.
• Jackson98Tom 3mo ago
  Amodei: Open model not so much bad, but open model in china hands very bad Non-US world: But open model is in every ones hands. Amodei: ... shut-up.
• Arshad-Talpur 3mo ago
  Actually this debate highlights how big the AI war is! my personal opinion is LLMs must be opensource and borderless, and ofcourse with given reputation i will use USA based Opensource model than the chinese one, but will they release opensource alternatives that is the question that will be answered in future
• maaaaattttt 3mo ago
  The only ban that should be put in place is on models' general and specific capabilities. If they have open or closed weigths is irrelevant. With the current state of capabilities there is already plenty we can do (in terms of creation and destruction unfortunately) before we as a society need more powerful capabilities. Once the limit is set, any non- governmental/military entity providing/hosting/using a more capabable model can be prosecuted.
• drdrek 3mo ago
  I'm really wondering what nightmare scenario does he imagine with super intelligent drones. I can already imagine it, a tiny drone carrying a 8x GPU rack thinking about life and deciding to go and build an idyllic society on a pacific island.
  • shifto 3mo ago
    A Psalm for the Wild-Built
• feblr 3mo ago
  I want to play with every kid, but my dad says that kid is bad. Every kid should be pat down by my dad, otherwise I don't want you to play with them, it's good for you.
• encyphilrightus 3mo ago
  "My primary concern is the risk that authoritarian governments" might want to look closer to home on that one...
• tacone 3mo ago
  I copy pasted this whole debate in Claude Sonnet 5 (web). The model came up with a summary but the thinking trace was entirely omitted. Well, if you ask me, that is dangerous.
• paweladamczuk 3mo ago
  There is an odd assumption behind this post that you can put effective guardrails on a model before releasing it. I thought nobody had such technology?
  • itemize123 3mo ago
    I think it's possible; But in the same vein, amongst other things, regulation is a transparent trojan horse to stop open-weight models - they would, on the next phase, argue that guardrails can be fine-tuned away if weights are open etc;
• ralferoo 3mo ago
  "Open-weights models—it does not matter whether they come from China or anywhere else—do potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn." Translation: "we won't be able to monetise it" "We should not sell powerful chips or chipmaking equipment to China" Translation: "we can buy them cheaper when there's less purchasing competition". "Distillation is a much more compute-efficient process than training models from scratch. It allows China to build much better models than its number of chips would ordinarily enable." Translation: "we've been outsmarted and lost our advantage because their way is faster and cheaper" I think it's a bit rich to complain about distillation, when they been plundering the internet for years for copyrighted works to train their models on without permission.
• pascal-maker 3mo ago
  https://x.com/biggest/status/2081891756769952075?s=46 https://x.com/biggest/status/2081891756769952075?s=46 Official stance of an anthropic-employee being against Dario amodei opinion.
  • jorisw 3mo ago
    Where do you get ‘official’? Mirror: https://xcancel.com/Biggest/status/2081891756769952075?s=46 https://xcancel.com/Biggest/status/2081891756769952075?s=46
    • pascal-maker 3mo ago
      okay I overdid with the word official.
• vrganj 3mo ago
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people The incredible hypocrisy to say this while the US itself is doing a sharp turn towards authoritarianism, with armed pro-government troops intimidating the population [0], where pro-government oligarchs openly talk of keeping the population under control with AI [1] all while actively enabling the rise of authoritarianism abroad [2] leaves a really bad taste in one's mouth. What Dario really is opposed to is not authoritarianism. It's an authoritarianism where he's not part of the ruling class. [0] https://www.theguardian.com/us-news/2026/jul/15/remove-ice-us-streets-joan-sebastian-duran-guerrero-lorenzo-salgado-araujo https://www.theguardian.com/us-news/2026/jul/15/remove-ice-u... [1] https://techcrunch.com/2024/09/16/oracle-ceo-larry-ellison-says-that-ai-will-someday-track-your-every-move/ https://techcrunch.com/2024/09/16/oracle-ceo-larry-ellison-s... [2] https://www.theguardian.com/us-news/2025/dec/05/civilisational-erasure-us-strategy-document-appears-to-echo-far-right-conspiracy-theories-about-europe https://www.theguardian.com/us-news/2025/dec/05/civilisation...
• alpineman 3mo ago
  If the US bans them, I really hope the EU doesn't follow (don't have high hopes)
• apexalpha 3mo ago
  It really barely matters. Even if the US closes itself of for Chinese models; Europe won't. The rest of the world won't. Even a completely closed US economy would not merit the current valuations Antrophic and OpenAI have.
• tehjoker 3mo ago
  Pro-US slop. The greatest threat is our own government. They are the ones with a right-wing government building a repressive apparatus across a huge number of datacenters and are proven to spy on the entire world. Anyone remember the Snowden leaks? Guantanamo bay? Alligator Alcatraz? Random murders by internal security? Blowing up a building full of school girls?
• haritha-j 3mo ago
  This argument might make a modicum of sense if the US hasn’t spent the last 2 years establishing itself as a bigger enemy of the west that China.
  • My_Name 3mo ago
    Being in the UK, I'm far more worried about the US than China right now. At least until the next election cycle is resolved. Hopefully, Trump won't start a civil war over not being allowed to stand for a third term. Hopefully.
• ranguna 3mo ago
  Clown. I switched from openAI to anthropic because OAI were the bigger clowns in the industry and didn't want to support them, anthropic seemed like the better alternative that didn't cooperate with governamental shenanigans and actually focused on building a better product. But now they are both clowns and anthropic is reaching a ceiling on quality. I'll jump ship as soon as I find a good subsidized Chinese model provider. US companies only path forward is to become retarded instead of competitive.
• EbNar 3mo ago
  This reads as a lot of fear mongering which in the end can ve resumed as "We are going to lose money".
• Lutger 3mo ago
  or 3: cease your assault of the international order, drop the zero sum paradigm and take the opportunity given by these crumbling US-build institutions to build a truly global network of lawful cooperation, where everybody can win. Admit that it wasn't fair to begin with, and start anew. It is in the best interest of everybody, also the US and China, and therefor the most rational thing to do.
• jagadaga 3mo ago
  "China is bad"? I don't know, man. I'm not buying it.
• wg0 3mo ago
  Basically: "We are in favour of 3D printers but there should be a body that tests and certifies that a 3D printer cannot print anything that can be used as weapon. Anything pointy or with a spring and recoil or... or..."
• skohan 3mo ago
  The way this reads, it seems like it was written for US lawmakers. > the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. This comes off a bit hypocritical, coming from one of the men responsible for the models most likely to be used by the US military, and for repression of US citizens. > We should crack down on industrial-scale distillation operations ... We should have policy interventions to deter this behavior. How exactly would you design policy interventions to stop distillation? If Anthropic wants to make their products available around the world, you would need some kind of global regulation to curtail it. And you would need to find some way to enforce it, which is far from trivial. Honestly I don't see how securing models against distillation is up to anyone but Anthropic, if that's something they want to achieve. Calling for regulation is a bit like crying to mommy and daddy when things aren't going as you would like. -- Overall, I can understand the argument that advanced AI models can present risks. But I don't see how regulation within the US can mitigate any of those risks. Any bad actor would be able to access the models outside the US, or covertly access the weights. The only real way to prevent advanced models from being deployed would be to go around the world bombing every significantly powerful data center, and I don't think Dario would call for that any time soon. The only thing this kind of regulation would achieve would be cutting US companies out of half of the innovation happening in the AI space, putting the US at a disadvantage relative to the rest of the world at everything except maybe frontier AI development.
• yamal4321 3mo ago
  Booo! "authoritarian hands". Scary Its not like current US regime is using AI for: 1) Mass military operations across the globe 2) Mass surveilance of its citizens 3) Removing every possible safety guard from AI/climate regulation 4) Stealing data across the world to train its own closed-source models 5) Is openly antidemocratic, destabilizing EU and economy of whole world I wonder why company which is actively cooperating with the current regime would push such message
  • Schlagbohrer 3mo ago
    US is using as much fossil fuel as possible for this AI build out. China is leading the world in an unprecedented green transition, so their AI usage is overwhelmingly going to come from solar, wind, and nuclear.
  • ramshorst 3mo ago
    I realized the other day that Anthropic will end up getting the reputation Facebook has today.
• Schlagbohrer 3mo ago
  Asmodeus' sinophobia and anti-communism is so 1950s. It is absurd to read it from a 21st century CEO who is under the age of 80.
  • deleted 3mo ago
    [deleted]
• pseudony 3mo ago
  Honestly no need to read this drivel. They have raised billions of dollars and are hoping to justify that by being a monopoly or oligopoly and their worst enemy- the pareto principle, is biting them in the ass. So now they’ll turn to policy to safeguard what they initially hoped could be achieved with money and technology. It is that simple, they are the very definition of an unreliable source on this topic.
• daitangio 3mo ago
  Who decide the if a model is safe to use? The strongest competitors or a government law? I prefer the second. Suppose Chinese model are super-risky: is it better to have them in the open, so everyone can review the problems or have a closed-model? Security is better at open Linux is open source because development is better. I understand Anthropic for sure, but trying to limit open weight model seems the opposite direction IF security+safety is the major concert. The Genie is ALREADY outside the bottle, and there are already plenty of companies offering inference services
• realusername 3mo ago
  What a load of nonsense, I can debunk a few of the worst points : - "safe" AI doesn't exist, it's not a thing, AI providers can't distinguish between good and bad use of the AI, the filters they put in place are mostly PR. - The US government IS an authoritarian government. Whoever wrote this doesn't know the difference between authoritarian and dictatorship
• boonzeet 3mo ago
  "My primary concern is the risk that authoritarian governments" - already lost me there by quoting JD. Vance. "...found that “other global powers’ robust progress in AI is challenging US economic competitiveness" - other countries having a slice of the pie is preventing the US having the whole pie! "...secondary concern is the risk that powerful AI models may be misused to carry out cyberattacks" - yes, those powerful open-source AI models like ChatGPT... oh wait.
• qqt 3mo ago
  Anthropic is being closed.
• ListeningPie 3mo ago
  How are Point 1, "We should not sell powerful chips or chipmaking equipment to China," and Point 2, "We should crack down on industrial-scale distillation operations," addressed by making open-weight models illegal? Point 1 is about restricting the sale of chips, not models. Point 2 concerns companies using closed models to train their own models through distillation. The real issue is that open-weight models can run on much less powerful hardware, making the current AI business model, where companies train a powerful model and gatekeep access to it, less relevant. Once open-weight models become good enough, much of the future revenue for today's leading AI labs could disappear. But just as Microsoft still has a market so will the large AI houses have one, but the moat will not be providing AI responses.
• northernsausage 3mo ago
  This USian death spiral season sure is picking up speed
• regexorcist 3mo ago
  Most hated company in record time, this Dario is insufferable.
• egorfine 3mo ago
  This letter is so full of contradictions. > Open-weights models that don’t have dangerous capabilities are a public good There are no models that don't have dangerous capabilities. There is not a single human in the world that doesn't have a dangerous capability. > authoritarian government [] build AI models that are more powerful than those built by the US Yeah, they will do that no matter whether you support the existence of open weight models or don't. In fact, they have probably already done that. I would say "deal with it" but there is nothing you can do actually. > authoritarian regimes have stolen and used AI Monkey looking away meme.jpg > powerful AI models may be misused Small AI models may be misused just as well. I would say it's the small models that are the problem: cheaply deployable, easily fine-tunable, fast. > once weights are released they cannot be withdrawn Exactly. It has been already done. So? > We should not sell powerful chips or chipmaking equipment to China I agree. We should have competition and I'm rooting for China to design their own chips. They wouldn't if we let them buy our's. > We should crack down on industrial-scale distillation operations Well, at this time it's already irrelevant. We have GLM-2, DeepSeek, K3, which are already SOTA and can be used for distillation. > All sufficiently capable models [] should go through mandatory safety testing Problem is, it's unenforceable legally. Models are just math and all previous attempts to ban maths have failed and simply pushed US back. I will not be asking US government permission to calculate matrixes of my choice.
• jiaosdjf 3mo ago
  These globalists were happy outsourcing manufacturing and software development. They are now angry that they failed to secure their IP, which was already based on dubious extraction of the collective works of human IP. You can't have it both ways. Either it's fine to outsource to China, fine to use H1Bs to undercut US talent, or, foreign governments are strategically positioning and their espionage is a threat to America.
• hoihoi 3mo ago
  [dead]
• My_Name 3mo ago
  "Top AI adopters outperform across industries" And that is all that businesses will see. All the hand wringing over human rights etc will somehow be discovered to just not be important once they read that.
• Fokamul 3mo ago
  EU must cooperate with China in same way as with US. They're both same level of evil.
• pietz 3mo ago
  Global AI safety apparently means keeping the world safely dependent on the US.
• mgaunard 3mo ago
  tl;dr: we only want the USA to dominate because we think their military would never abuse AI (disproven a few months ago)
• vasco 3mo ago
  > Open-weights models that don’t have dangerous capabilities are a public good I don't think the qualifier "that don't have dangerous capabilities" is needed and that single phrase is a trojan horse / escape hatch that will allow them to say they are against any model they want. Smart but weasel behavior.
• kome 3mo ago
  this little blog post is absolutely shameful and tone-deaf. why is distillation bad if you have literally stolen - and let me underline this word: stolen - all the human knowledge to train your models? isn't that a contradiction? and also... i hope the last few years have made abundantly clear to absolutely everybody that the American government is as dangerous, if not more so, than other authoritarian governments all over the world. if the same logic is applied, i don't see why it should work for china but not for the US.
• hyperadvanced 3mo ago
  Love how these bros always cop to “China bad” when they run out of good arguments.
• DrScientist 3mo ago
  The letter misses out the key issue here. Should the worlds public knowledge be enclosed by a few powerful private companies or should it be available to all? The original vision for the web was a decentralized, open information sharing space designed to empower humanity. In my view the effort to privatise that via LLM distillation is the antithesis of that vision. https://medium.com/@timberners_lee/marking-the-webs-35th-birthday-an-open-letter-ebb410cc7d42 https://medium.com/@timberners_lee/marking-the-webs-35th-bir... In terms of the risks mentioned in the letter - I agree with the risk of AI enabling a powerful surveillance state - however I don't see that as a solely Chinese problem. The other risk mentioned - that knowledge can be dangerous - sure - but ultimately here, as there are so many low tech ways to cause mayhem, the ultimate protection is to have a society where people don't want to do it. And perhaps more importantly I'd note that the primary tool to justify a powerful surveillance state is the fear of terrorism ( and others ).
• simgt 3mo ago
  Until he advocates for an independent body akin to IAEA under the control of the UN, I'll call everything he says about AI safety and the greater good bullshit.
• tripzilch 3mo ago
  > authoritarian regimes have stolen and used AI "we're not saying open models are stealing, but we will quote the US vice president, saying this"
• shevy-java 3mo ago
  But who would trust Anthropic at this point in time? Too much money is in the ring now. They will just prefer to say what sounds best rather than the truth. Too many billionaires want to see more money now. The AI bubble will grow as a consequence of this.
• tripzilch 3mo ago
  So, after carrying out the single largest intellectual property heist in the history of mankind, scraping and continuing to scrape the entire Internet and whatever data they can get their grubby hands on, > At Anthropic we’re committed to cracking down on industrial-scale distillation uhuh I don't really believe their reasons
• Aissen 3mo ago
  We should fight against their call to ban distillation much more strongly: LLM SaaS should ease people getting their full query history and everyone should be allowed to pool them to distill or analyze or whatever. It's a basic requirement against future monopolies.
• m101 3mo ago
  His complaining about the cost of distilling is hypocritical when the cost of writing all that text on the back of which he trained is also far lower than the cost of producing that text in the first place. The moral issue stops at wherever he is forced to bear the cost apparently.
• deton3991 3mo ago
  [flagged]
• fsuts 3mo ago
  My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat If relations improve between the countries over ai, that comment could lead to billions in lost revenue for Anthropic
• makingstuffs 3mo ago
  The whole article boils down to two points: 1. “I’m fine with ‘em, as long as they’re useless” 2. “The only authoritarian regime which can be trusted with such power is the one which benefits my company”
• witx 3mo ago
  This person and company are disgusting. Engineers should grow a backbone and stop supporting anything related to Anthropic
• titaniumrain 3mo ago
  before anthropic's ipo, their models must be the best. :)))
• amelius 3mo ago
  > We should crack down on industrial-scale distillation operations. And what if I called extracting value from copyrighted works "distillation" ? It sounds hypocritical.
• dealrescue 3mo ago
  [dead]
• olalonde 3mo ago
  Stupid question but why should we care about Amodei's geopolitical insights more than anyone else's? I don't think knowing how LLMs work makes you particularly qualified to comment on this.
• bcjdjsndon 3mo ago
  Our position on open weights models = don't sell gpus to china?
• deleted 3mo ago
  [deleted]
• prmoustache 3mo ago
  "We should not sell powerful chips or chipmaking equipment to China," Why? Has Anthropic, or the country it is based in formally declared war against China?
• fenestella 3mo ago
  [dead]
• deleted 3mo ago
  [deleted]
• _davide_ 3mo ago
  After this level of bullshit I won't ever spend a second any other new/blog from anthropic
• ppap3 3mo ago
  Hypocrite to the max. All of them have contracts for """defense""" and suddenly it is worry some that others are allowed to do the same. They all support chasing people because of their origin without following the required legal steps. It is just a matter of time until they come for you and your opinion
• merelydev 3mo ago
  Why should consumers care what Coca-cola thinks about Pepsi?
• Lockal 3mo ago
  It is like Microsoft publishes a manifesto "Our position on open-source operating systems" in 2026: - Over the /last few days/ there has been a lot of discussion about open-source systems, especially those that are not made in US. (implying that these discussions appeared just now) - Reports suggest that /some US officials/ are considering banning the use of these open-source systems by US companies. (implying somebody other than their corrupted representatives support this ban) And then switching to a complete gaslighting regarding manufacturing capabilities of US vs China, about "open-source can not be trusted", about "GPT-2 is too dangerous to release"
• ExoticPearTree 3mo ago
  The Anthropic CEO has an entitlement problem. I read his opinion twice and somehow in his worldview, if any other country gets better at AI than the US, the apocalypse will come and the US will be no more. Or something along those lines. Secondly, he's like the Trump of tech: China, China, China. Thirdly, he needs to cut it out with the guardrails and other safety BS he's peddling. If we're going to get Skynet, we will get it no matter what we do.
• softwaredoug 3mo ago
  The Anthropic safety position may be counterproductive to US AI competitiveness. Between China and US theres a defacto unequal distillation environment. China has no qualms about distilling off other labs outputs. US labs actively avoid it for legal reasons. Whatever the actual legalities, US labs have a relative hand tied behind their back. If they didn’t, it’d be easier for Grok, Gemini, and Meta to catch up. If we care about US competitiveness, then one solution could be either OpenAI/Anthropic enter distillation agreements with other US labs. Or we decide to make distillation public domain / legal. Until then the only models Anthropic/US Govt could realistically regulate would be in the US market. And that’s as much a losing game as tariffs.
• irenaeus 3mo ago
  I thought his points were legitimate and well argued. I think that it's very easy to say "X should be unrestricted". I also think that eventually one of these models will be used to do something truly destructive and insane and that some level of "compelled responsibility" to the open-weight model ecosystem is inevitable.
  • rnewme 3mo ago
    So we should restrict chainsaws and excavators?
    • irenaeus 3mo ago
      We are not talking about chainsaws and excavators, we are talking about LLMs. LLMs have different risks in terms of scale and kind and must be considered differently.
• jimmydoe 3mo ago
  Both Dario and Altman are shit , but Dario is consistent. I don’t know who is worse.
• sarthak_ib 3mo ago
  [dead]
• mark_l_watson 3mo ago
  To me, the most weasel words are “”All sufficiently capable models, open and closed, should go through mandatory safety testing.”” I bet via regulatory capture that evaluating Chinese models will be a very slow process if Dario gets his way. I enjoy using Opus, but I am in the process of ripping it out of my toolkit and leaving it on the ground behind me, as I walk away.
• feiz45607 3mo ago
  [flagged]
• zeezer0 3mo ago
  He is concerned about the perpetration of incredibly deep repression of their own people. But completely blind to the fact the USA are the current champs at this.
• sbiru93 3mo ago
  pathetic
• tcldr 3mo ago
  Ban distillation of our outputs, but our distillation of the sum-total of civilisation's intellectual output – proprietary or otherwise – is fair use? Either everyone licenses, or nobody does. And if you can't enforce licensing bans for everyone, the de-facto loser is those who you'd probably want to support the most, start-ups and universities, while your adversaries gain the upper hand. The strongest argument for restricting distillation is arms control – but distillation is the way to defeat GPU embargoes. So, distillation goes on regardless. Only pre-training is seriously attenuated. If we're honest, the models are compressions of everything society has ever written. A few large corporations can't own that, no more than they can claim copyright for a zip file of the public library. The genie is out of the bottle, now. So open it up – inputs and outputs, forward-looking – for everyone.
  • jackcviers3 3mo ago
    On the other hand, the open weights models could crawl and annotate and rl the training data that Anthropic and OAI did in exactly the same way, and take the exact same legal hits. They use distillation because it's cheaper not to do so.
    • jatins 3mo ago
      > On the other hand, What is the other hand here? They could do slow/costly illegal think #1 instead of the fast/cheaper illegal thing #2 that they currently do?
      • monooso 3mo ago
        Genuine question: is distillation illegal, or just against Anthropic's terms of use?
        • deleted 3mo ago
          [deleted]
        • rad-b 3mo ago
          It seems to be only against ToS; the problem, however, is that “distillation” involves intent. Anthropic seems to be calling for ban on distillation because they cannot reliably identify who does it - if they could, they would’ve just banned all infringing users. However, if there is no systematic difference between regular use and “distillation use” then they need to establish intent, which is laughably hard if you cannot even distinguish use cases. Hence the call to make it illegal too (and just against ToS). Because if it is illegal, then they can rely on US to enforce the law. And the US, for a lack of technical solution, can use the good old “China bad” shortcut.
• dominotw 3mo ago
  > China has limited domestic production capacity, and therefore, due to the scaling laws, cannot build more powerful models than the US without US chips. So anyone with chips can produce a model ?
• nahuel0x 3mo ago
  So he is against China for its hypothetical usage of AI against civilians but is not against US and Israel that already use AI provided by him to operate a real genocide that kills thousands and thousands of Gaza children in the most horrible ways? Very ethical.
• spiritodim 3mo ago
  [flagged]
• timedude 3mo ago
  For all his mentions of China being an "authoritarian state" (as if there can be one that isn't), has Dario looked at what the usa has been doing around the world in recent years?
• man_luke 3mo ago
  The question is whether Claude suggested that he should write it.
• impalallama 3mo ago
  Considering the state of the current US Administration, its impossible for me to take cries of national security and worries about abuse from "authoritarian governments" seriously
• devaziz96 3mo ago
  Funny how this whole article is full of contradictions. they aren't against open-weights yet they are against everything to make them. Involving politics so much that I thought I was reading this on BBC instead of Anthropic. As a developer I fully support whoever want to make AI accessible and affordable and clean, which USA biggest providers aren't doing. On the other hand I see Chinese models being on bar with them with far less cost and more transparency but yet somehow they are still the bad guys out there :D Classic
• sagex 3mo ago
  Essentially, if SOTA models are freely available, the playing field then is on inference where people from all around the world can participate. This basically dilutes the value of model trainer like Anthropic. I can see the panic.
• rootlocus 3mo ago
  > My primary concern is the risk that authoritarian governments—not solely the Chinese Communist Party (CCP), although the CCP is clearly the most capable threat—build AI models that are more powerful than those built by the US, and use them to achieve permanent military superiority or perpetrate incredibly deep repression of their own people. There's not much stopping US from becoming an authoritarian regime, and the US is already using AI against its own citizens.
• aranaur 3mo ago
  Man, they're just worried about their bottom line
• ajnin 3mo ago
  I feel like this whole discussion boils down to one question (as commonly when considering political issues) : do we want a framework that favors progress globally for everyone, or do we want to ensure superiority of some above the others ? Sure that's simplistic, but human behavior is not that complex, there are a few basic needs and wants that drive everything. In that specific case, those currently in a dominating position (Anthropic, the US) want to put hurdles (regulations) to make it more difficult for others to catch up, and everyone else want to join forces to overtake them.
• mapledesk 3mo ago
  For smart Chinese hired by anthropic, are you still proud working there?
• deleted 3mo ago
  [deleted]
• mahmoudilyan 3mo ago
  I don't trust you Dario, that's the issue here
• gverrilla 3mo ago
  Fascist/liar.
• markmiro 3mo ago
  I read it and makes sense what he’s saying, but comments seem very against. Maybe the problem is it’s that Dario is saying it, so it’s easy to assume he has bias, which he does. I’m not inherently opposed to open models, even at the frontier. But it could make sense for an adversary to give away something for free for some time to create dependence, or to tip the balance of power
• imrozim 3mo ago
  [dead]
• QuantumNoodle 3mo ago
  Oh cry me a river. You went full tilt ahead scrambling before safeguards were set up and now want to set up safeguards protect your models from commoditization. Now that you've gotten your footing you preach about the dangers of powerful AI models.. Cats out of the bag. The fu*k you think was going to happen when you were building these tools? A Utopia?
• forluc 3mo ago
  [dead]
• shivanshuag 3mo ago
  It seems impossible to read this post without getting political. Dario seems to be saying that AI in the hands of USA is good but in the hands in China is bad, which is a very US centric view. There is a whole world outside the two countries which see neither as a good actor. As a reminder, USA was the first country which dropped an atomic bomb on civilians.
• cdrnsf 3mo ago
  There's a real irony in, rightly, highlighting the authoritarian nature of the CCP while failing to acknowledge the direction and inclinations of the present administration with which Anthropic et al are doing business.
• z0ltan 3mo ago
  [dead]
• ktosobcy 3mo ago
  Fuck hypocrites with "Usania good" attitude… ffs xD
• realo 3mo ago
  Dario speaks as if the USA was a benevolent friend. I am Canadian and certainly do not consider the USA to be our benevolent anymore. USA ... China ... same difference.
  • throw1234567891 3mo ago
    This, from where I sit there's no difference.
• jmull 3mo ago
  I'm a lot more worried about the US government than China's -- it has a lot more direct impact on my life and is largely controlled by billionaires who do not have good intentions toward the rest of us. And, to me, this letter comes off as quite insincere. Stopping distillation can only be explained as an anti-competitive measure. Their own explanation is nonsensical -- they say is needs to be stopped to help prevent authoritarian governments from overtaking the US at the frontier of AI. But by its nature distillation lags behind the frontier. Not to mention the US is one of the authoritarian governments we need to be concerned with, and the next thing they advocate for is full, worldwide regulatory control of AI, which is rather heavily authoritarian. These guys are making a $T gamble and need to screw over a lot of people very badly to make it pay off. You do not want to trust anything they say.
• aagha 3mo ago
  > We should crack down on industrial-scale distillation operations But we should not crack down on the illegal digestion of copyright text used to train our models.
• firemelt 3mo ago
  yourself distilling humanity too bitch
• FloorEgg 3mo ago
  I struggle to understand how they can frame for the greater good to be anti-distillation and pro-open-weight-models. I can understand how they would be against competitors distilling their models and for having access to competitor open weight models. At first glance it seems like a pragmatic answer: "banning open weight model use by US companies doesn't help", but to what question? What if those open weight models are poisoned and primed to create backdoors in US companies? If the government has intelligence this is a credible risk, what should they do about it? Overall it feels like the letter is conflating a bunch of motives, some of which may be opaque, and at face value doesn't seem logically consistent.
• olejorgenb 3mo ago
  https://www.antirez.com/news/172 https://www.antirez.com/news/172
• scottydelta 3mo ago
  Meanwhile: > DARPA, U.S. Air Force fly AI-controlled F-16 > https://www.darpa.mil/news/2026/darpa-us-air-force-fly-ai-controlled-f-16 https://www.darpa.mil/news/2026/darpa-us-air-force-fly-ai-co... May be someone else needs to be controlled.
• surcap526 3mo ago
  [dead]
• BVHauge 3mo ago
  [flagged]
• srcoder 3mo ago
  Just open all Models and competition is gone and we don't need to worry about all this secrecy anymore. Then implementation, tools and availability become selling points. The whole discussion on who has the largest ** will come to an end. All models are mostly trained on the same datasets anyways with just some other tools or system prompt in front of it