6 ms·

ANTI_DISTILLATION_CC This is Anthropic's anti-distillation defence baked into Claude Code. When enabled, it injects anti_distillation: ['fake_tools'] i

• nialse 7mo ago
  Paranoia. And also ironic considering their base LLM is a distillation of the web and books etc etc.
  • petcat 7mo ago
    They stole everything and now they want to close the gates behind them. "I got the loot, Steve!" I feel like the distillation stuff will end up in court if they try to sue an American company about it. We'll see what a judge says.
    • arcfour 7mo ago
      You're perfectly free to scrape the web yourself and train your own model. You're not free to let Anthropic do that work for you, because they don't want you to, because it cost them a lot of time and money and secret sauce presumably filtering it for quality and other stuff. Stole? Courts have ruled it's transformative, and it very obviously is. AI doomerism is exhausting, and I don't even use AI that much, it's just annoying to see people who want to find any reason they can to moan.
      • jtbayly 7mo ago
        Wut?They did exactly the same thing! Try this: If you want to train a model, you’re free to write your own books and websites to feed into it. You’re not free to let others do that work for you because they don’t want you to, because it cost them a lot of time and money and secret sauce presumably filtering it for quality and other stuff.
        • arcfour 7mo ago
          [flagged]
          • buzzerbetrayed 7mo ago
            [flagged]
            • jollymonATX 7mo ago
              Yeah these folks skin is often very thin. One poke too hard and it's "whatever" and them scuttling off. Really hope there is a day they introspect.
              • arcfour 7mo ago
                I introspect all the time. I just disagree with you so I have thin skin? Lol. I think it's transformative. I also think that it's a net positive for society. I lastly think that using freely available, public information is totally fair game. Piracy not so much, but it's water under the bridge. I hope you introspect some day, too, and realize it's acceptable for people to have different views than you. That's why I don't care; you aren't going to change my mind and I can't change yours either, so it's moot and I don't care to argue about it further.
                • jollymonATX 7mo ago
                  You had appeared to scuttle off but alas I was wrong (and sorry to imply you are a crab of some sort) however your comment followup on not changing minds might be a tad shell-ish. I'm open minded actually on the issue and these are major issues of our time. I'm personally impacted by this and it does make me wonder "will I write X thing again" and it is a very hard question to answer frankly. When you see your works presented in summary on search and a major decline in traffic you really do think about that. It impacts my ability to make money as I once did prior to 2024 (when it really hit) without doubt. Edit/spelling
      • petcat 7mo ago
        > Stole? Courts have ruled it's transformative, and it very obviously is. The courts have ruled that AI outputs are not copyrightable. The courts have also ruled that scraping by itself is not illegal, only maybe against a Terms of Service. Therefore, Anthropic, OpenAI, Google, etc. have no legal claim to any proprietary protections of their model outputs. So we have two things that are true: 1) Anthropic (certainly) violated numerous TOS by scraping all of the internet, not just public content. 2) Scraping Anthropic's model outputs is no different than what Anthropic already did. Only a TOS violation.
        • gruez 7mo ago
          >The courts have ruled that AI outputs are not copyrightable. "not copyrightable" doesn't imply they can't frustrate attempts to scrape data.
          • petcat 7mo ago
            Nobody is saying they can't try to stop you themselves. That's where the Terms of Service violation part comes in. They can cancel your account, block your IP, etc. They just can't legally stop you by, for instance, compelling a judge to order you to stop.
            • dpark 7mo ago
              > They just can't legally stop you by, for instance, compelling a judge to order you to stop. They probably can, actually. TOS are legally binding. More likely they would block you rather than pursuing legal avenues but they certainly could.
              • petcat 7mo ago
                The Supreme Court already ruled on this. Scraping public data, or data that you are authorized to access, is not a violation of the Computer Fraud and Abuse Act. Now, if you try to get around attempts to block your access, then yes you could be in legal trouble. But that's not what is happening here. These are people/companies that have Claude accounts in good standing and are authorized by Anthropic to access the data. Nobody is saying that Anthropic can't just block them though, and they are certainly trying.
      • airstrike 7mo ago
        Guess who else spent a lot of time and money and secret sauce? Do you hear the words coming out of your mouth?
      • unethical_ban 7mo ago
        Let's talk ethics, not law. Why is it okay for these companies to pirate books and scrape the entire web and offer synthesized summaries of all of it, lowering traffic and revenue for countless websites and professions of experts, but it is not okay for others to try to do the same to an AI model? Is the work of others less valid than the work of a model?
        • p1esk 7mo ago
          I don’t see why it’s not ok to do that to an AI model. Or are you asking why they don’t want you to do it?
        • sfn42 7mo ago
          I don't think anyone's saying it's not okay - I think the point is that Anthropic has every right to create safeguards against it if they want to - just like the people publishing other information are free to do the same. And everyone is free to consume all the free information.
        • gruez 7mo ago
          >Why is it okay for these companies to pirate books Courts have ruled it's not, and I don't think anyone is arguing it's okay. >but it is not okay for others to try to do the same to an AI model? The steelman version is that it's okay to do it once you acquired the data somehow, but that doesn't mean anthropic can't set up roadblocks to frustrate you.
      • two_tasty 7mo ago
        "...free to scrape the web yourself and train your own model." Yes, rich and poor are equally forbidden from sleeping under bridges.
        • kspacewalk2 7mo ago
          Meaning what? The poor gets to sleep in the guest room of the rich guy's house because muh inequality? Anthropic paid a lot of money for a moat and want to guard it. It is not wrong, in any sense of the word, for them to do so.
          • salawat 7mo ago
            Rich people aren't going to find themselves needing to sleep under a bridge, so the law really only exists as a constraint on the poor. Duh. The flex that "well a rich guy couldn't do it either" is A) at best a myopic misunderstanding perpetuated by out of touch people and B) hopelessly naive, because anny punishment for the rich guy actually sleeping under a bridge is so laughably small it may as well not even exist. Hence, the whole bit of "a legal system to keep these accountable, but not for me".
            • dpark 7mo ago
              You explained what “rich and poor are equally forbidden from sleeping under bridges” means, but not what this has to do with the statement that one is free to do their own scraping and training, which I’m pretty sure is what kspacewalk was asking.
            • kspacewalk2 7mo ago
              Okay, you explained what Anatole France meant, which is probably helpful for those few who didn't get it from the quote itself. Perhaps now you can explain what on earth this has to do with Anthropic not wanting to let other for-profit businesses mooch off its investment of time, brainpower and money?
      • nunez 7mo ago
        Lol; like heck we are. Try scraping the NYTimes at LLM scale. You can time how quickly you’ll get 420’ed or, at worst, hit with a C&D.
        • nunez 7mo ago
          (429'ed, I meant)
      • alpha_squared 7mo ago
        > You're perfectly free to scrape the web yourself and train your own model. Actually, not anymore as a result of OpenAI and Anthropic's scraping. For example, Reddit came down hard on access to their APIs as a response to ChatGPT's release and the news that LLMs were built atop of scraping the open web. Most of the web today is not as open as before as a result of scraping for LLM data. So, no, no one is perfectly free to scrape the web anymore because open access is dying.
      • andersonpico 7mo ago
        Your selective respect for work is a glaring double standard. The effort to produce the original content they scraped is order of magnitudes bigger than what it took to train the model, so if this wasn't enough to protect the authors from Anthropic it shouldn't be enough to protected Anthropic from people distillating their models. Your legal argument is all over the place as well. What is more relevant here: what the courts ruled or what you consider obvious? How is distillation less transformative than scraping? How does courts ruling that scraping to train models is legal relate to distillation? Nobody is scoring you on neutrality points for not using AI much and calling this doomerism is just a thought-terminating cliche that refuses to engage with the comment you're replying. In fact, your comment is not engaging with anything at all, you're vaguely gesturing towards potentitial arguments without making them. If you find discussing this exhausting then don't but also don't flood the comments with low effort whining.
      • loremium 7mo ago
        reminds me of `don't look up` a bit. there clearly is an imbalance in regards to licenses with model providers, not even talking about knowledge extraction (yes younger people don't learn properly now, older generations forget) shortly before the rug-pull happens in form of accessibility to not rich people
      • hax0ron3 7mo ago
        It is transformative, but if I make a bunch of requests to their API and use the responses to distill my own model, that is also transformative.
    • decremental 7mo ago
      [dead]
    • olalonde 7mo ago
      Also, begging to get "regulated": https://x.com/TheChiefNerd/status/2038565951268946021 https://x.com/TheChiefNerd/status/2038565951268946021
    • Andrex 7mo ago
      I just rewatched that scene last night on YouTube. Maybe this is the universe telling me to watch the whole movie again... It's cool to see Noah Wyle getting his due these days (The Pitt).
    • dpark 7mo ago
      [flagged]
      • petcat 7mo ago
        Not every lawsuit goes to court, or results in a decision. I'm sure you know that.
        • dpark 7mo ago
          You should ask Claude what a lawsuit is. Or perhaps you mean “trial” and not “court”?
          • petcat 7mo ago
            I feel like I'm talking to Claude right now. Am I?
            • dpark 7mo ago
              This is the new “I can’t defend my statement online” retort, huh? “Well I might be wrong but at least I’m not AI like YOU!”
              • deleted 7mo ago
                [deleted]
              • petcat 7mo ago
                Ever heard the phrase "settled out of court"? A lot of lawsuits are settled even before a court clerk processes the paperwork.
                • dpark 7mo ago
                  A lawsuit with no paperwork filed is not a lawsuit. That’s just an agreement. Again, you seem to be conflating lawsuit with trial.
                • 7mo ago
  • spiderfarmer 7mo ago
    That isn't irony, it's hypocrisy.
    • croes 7mo ago
      That’s capitalism
      • dmix 7mo ago
        As opposed to the rent-seeking copyright industry where 1% goes to the original creators if you're lucky.
        • jitl 7mo ago
          That’s capitalism too
          • dmix 7mo ago
            Technically state-capitalism since it's an industry created as a result of congress regulating commerce with aggressive IP laws (aka rent-seeking)
            • brookst 7mo ago
              Where can I see an example of any other kind of capitalism?
              • dmix 7mo ago
                Capitalism is always underpinned by a strong legal system which is why most criticism is about constraining growth in legislation, not killing off interference outright. Copyright law is a good example of a law that made sense in it's original form but turned into a monster with scope-creep. Although, if we're being realpolitik, every time government interference grows in scope and corrupts markets, capitalism still gets blamed and people call for more government to fix it (see: housing). So the capitalism vs state capitalism distinction isn't very meaningful in practice.
  • jaccola 7mo ago
    I would say not all that ironic. Book publishers, Reddit, Stackoverflow, etc., tried their best to attract customers while not letting others steal their work. Now Anthropic is doing the same. Unfortunately (for the publishers, at least) it didn't work to stop Anthropic and Anthropic's attempts to prevent others will not work either; there has been much distillation already. The problem of letting humans read your work but not bots is just impossible to solve perfectly. The more you restrict bots, the more you end up restricting humans, and those humans will go use a competitor when they become pissed off.
    • brookst 7mo ago
      It's really just tech culture like HN that obsesses over solving problems perfectly. From seat belts to DRM to deodorant, most of the world is satisfied with mitigating problems.
  • johnfn 7mo ago
    It is absolutely not paranoia. People are distilling Claude code all the time.
  • sheept 7mo ago
    It's not really paranoia if it's happening a lot. They wrote a blog post calling several major Chinese AI companies out for distillation.[0] Perhaps it is ironic, but it's within their rights to protect their business, like how they prohibit using Claude Code to make your own Claude Code.[1] [0]: https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks https://www.anthropic.com/news/detecting-and-preventing-dist... [1]: https://news.ycombinator.com/item?id=46578701 https://news.ycombinator.com/item?id=46578701
    • salawat 7mo ago
      Their business shouldn't exist. It was predisposed on non-permissive IP theft. They may have found a judge willing to cop to it not being so, but the rest of the public knows the real score. And most problematically for them, that means the subset of hackerdom that lives by tit-for-tat. One should beware of pissing off gray-hats. Iit's a surefire way to find yourself heading for bad times.
    • gmerc 7mo ago
      And conveniently left out they themselves distilled DeepSeek for chinese content into their model....
• crazylogger 7mo ago
  Why would this be in the client code though?
• mmaunder 7mo ago
  Haven’t looked at the code, but is the server providing the client with a system prompt that it can use, which would contain fake tool definitions when this is enabled? What enables it? And why is the client still functional when it’s giving the server back a system prompt with fake tool definitions? Is the LLM trained to ignore those definitions? Wonder if they’re also poisoning Sonnet or Opus directly generating simulated agentic conversations.
  • cedws 7mo ago
    Not sure, and not completely convinced of the explanation, but the way this sticks out so obviously makes it look like a honeypot to me.
    • mmaunder 7mo ago
      Great theory. I'll dig deeper.
      • mmaunder 7mo ago
        Claude Code has a server-side anti-distillation opt-in called fake_tools, but the local code does not show the actual mechanism. The client sometimes sends anti_distillation: ['fake_tools'] in the request body at services/api/claude.ts:301 The client still sends its normal real tools: allTools at services/api/claude.ts:1711 If the model emits a tool name the client does not actually have, the client turns that into No such tool available errors at services/tools/StreamingToolExecutor.ts:77 and services/tools/toolExecution.ts:369 If Anthropic were literally appending extra normal tool definitions to the live tool set, and Claude used them, that would be user-visible breakage. That leaves a few more plausible possibilities: Fake_tools is just the name of the server-side experiment, but the implementation is subtler than “append fake tools to the real tool list.” or The server may inject tool-looking text into hidden prompt context, with separate hidden instructions not to call it. or The server may use decoys only in an internal representation that is useful for poisoning traces/training data but not exposed as real executable tools.
• GorbachevyChase 7mo ago
  I like these guys less every day. The rate limits are so low they are close to not even useful as a provider.
  • cedws 7mo ago
    It made me raise my eyebrows when everyone was rushing to jump to Claude because OpenAI agreed to work with the DoW. Both companies are just as shitty as each other and will resort to underhanded tactics to stay on top. Go China to be honest. They're the most committed to open AI research and they have more interesting constraints to work under, like restricted access to NVIDIA hardware.
    • lobsterthief 6mo ago
      But then the Chinese government has the ultimate say and, propaganda aside, if they don’t like what your product is you might suddenly lose access to said LLM provider.
• 3form 7mo ago
  I was thinking just yesterday that the research that Anthropic was sharing regarding how it's easy to poison training was unlikely to be conducted out of goodness of the heart.
• jjcm 7mo ago
  It looks like it worked, fwiw. The qwen 27b model distilled on Opus 4.6 has some known issues with tool use specifically: https://x.com/KyleHessling1/status/2038695344339611783 https://x.com/KyleHessling1/status/2038695344339611783 Fascinating.
• torginus 6mo ago
  This made me think of something - at work, if we wfh, we have to use one of those MITM proxies that intercept HTTPS at the kernel level. Imo such a thing can easily read the traffic and thus is indistinguishable from a distillation attempt from CC's PoV. I've had CC freak out on my machine, and sometimes generate pretty bad results, the CoT is often also not available. I wonder it CC thinks I'm trying to distill the model. This is a common enough use case that I think the devs at Anthropic should consider.
• deleted 6mo ago
  [deleted]
• pqtr2 6mo ago
  Good explanation here https://alex000kim.com/posts/2026-03-31-claude-code-source-leak/ https://alex000kim.com/posts/2026-03-31-claude-code-source-l... its a server side flag set by the distillation detection, shouldn't affect most user's sessions. I don't think?
• huang-b62b5756 6mo ago
  It is a RCE, if what’s being injected is an executable and malicious tool