18 ms·
“We have information that Moonshot distilled Fable for the development of K3”
https://xcancel.com/mkratsios47/status/2079933645888880708 https://xcancel.com/mkratsios47/status/2079933645888880708
- mrandish 3mo agoHow was K3 trained on data distilled from Fable when Fable was only publicly available in the last two weeks before K3 was released? The timing just doesn't work.
- nchmy 3mo agoWe have information that Claude distilled billions of copyrighted, and otherwise-created-by-others, materials for the development of their entire business.
- noncoml 3mo agoYes, I know this is not Reddit but Clarkson’s “Oh no! Anyway…” is the perfect, and most fitting, reaction to this. Nothing else to say
- treetalker 3mo agorules for thee but not for me
- dang 3mo agoPlenty of HN readers feel this way and it's a good point, but it has also become an entirely cliché response which pops up like mushrooms anytime "distillation" appears. That means it's against the site guidelines, which ask: "Eschew flamebait. Avoid generic tangents. Omit internet tropes." - https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html I don't mean to pick on you personally! It's just that reflexive responses always tend to show up first in a thread, when what we really want are reflective responses [1]. Similarly, there's a strong tendency for threads to turn into generic discussions, whereas what we really want are specific ones [2]. [1] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=reflective%20reflex%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor... [2] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=generic%20discussion%20by%3Adang%20-flamewars%20-flamewar&sort=byDate&type=comment https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
- Bratmon 3mo agoLaughing at the idea of distillation being bad is exactly as cliche/flamebaity as complaining that your model got distilled. No more, no less.
- cmdocidjcije 3mo agoAt this point distillation is part of the ecosystem and everyone should embrace it. If distillation is a threat to one’s business model, then the business strategy needs to shift.
- cassianoleal 3mo agoI have to agree. I've flagged the post.
- phikappa 3mo agosure but anthropic is not literally in the comments complaining, so it's not quite apples to apples, right?
- ceejayoz 3mo ago> it has also become an entirely cliché response To be fair, that's also the case for the link itself we're discussing.
- supriyo-biswas 3mo agoI think then we should ban these sorts of posts about the allegation of distillation, since being able to post the story but then warning accounts with comments about the hypocrisy, is not the correct way to go about it.
- latexr 3mo agoI agree in the abstract, but perhaps the way to avoid generic responses is to disallow (or segment) generic submissions. This website is no longer HN, it should be renamed AIN. There is only so much to say about the subject, and if cliché submissions keep getting accepted and upvoted and shoved to every visitor without a way to avoid them (barring leaving the website entirely), then people will eventually gravitate to the same responses. If your neighbours play loud music every night, they don’t get to complain that everyone is always mentioning the loud music to them. You are a fantastic moderator, but there’s only so much even you can do. If nothing changes about the website, the problem will only get worse. I warned years ago that this would happen, the signs were on the wall immediately.
- unethical_ban 3mo agoLet me put it in an HN-acceptable format: Given the disregard for intellectual property rights the AI labs had in creating the technology, many people feel no sympathy for second-order AI labs using similar techniques to build technology off the US frontier labs. I think fighting distillation will always be cat-and-mouse, and that it's more of a concern for the stockholders and perhaps an iota of national security. It can't be stopped entirely; the "problem" will always be there. I'm much more concerned about asymmetry of power between citizens and their governments with omnipresent surveillance and analysis being done on everyone living their lives. Societies throughout history have taken as a given their power to overthrow malicious governments when things hit a breaking point, and I am scared that this technology will lock societies into a state of total subordination for eternity.
- onraglanroad 3mo agoThat's a better comment but > Societies throughout history have taken as a given their power to overthrow malicious governments when things hit a breaking point, simply isn't true. People throughout history have simply accepted that society is the way it is and sometimes used whatever means they could to get to the top. Revolutions have been very rare and usually ended up with the revolutionaries simply taking the place of the previous rulers. "Meet the new boss..."
- unethical_ban 3mo agoThen let's call it a reset, one way or the other. I'm afraid society won't be able to hit the reset button on a government in the future.
- tibbydudeza 3mo agoProof - they also claimed that China has an ASML UEV machine - crickets when ASML said it was impossible due to all the safeguards and assistance needed to operate one. The current US administration is known to be collection of BS artists and liars.
- beaker52 3mo ago[flagged]
- dang 3mo ago> /giphy nobody cares SpongeBob meme Can you please not do this here? There's nothing wrong with it, we're just trying for something else on this site. "Don't be snarky. [...] Omit internet tropes. [...etc...] https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- oliculipolicula 3mo agoGenuinely want to know if you're surprised that there have been almost no commenters concurring with (what seems to be) a well-substantiated (& on the face of it defensibly center-right) tweet from the WH Will contrarian dynamic take off here? Will it be flagged? Will you unflag it? Edit: catigula, mattrighetti With more substantive technical comments towards the middle
- dang 3mo agoYes, I think it just takes time for more substantive comments to show up. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=reflective%20reflex%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
- oliculipolicula 3mo agoAh CD didn't seem to have taken off. Not surprised, but had thought you'd be. [My more convivial analysis summarised as] Right-wing post -> left majority take -> lack of good places for right reflective minority takes to shift the convo
- alightsoul 3mo agohow is it possible to distill fable only a month after its release? maybe they are confusing opus with fable.
- cmdocidjcije 3mo agoCreate a couple thousand Claude max accounts and split the work amongst them perhaps.
- skeledrew 3mo agoThat's crazy income for Anthropic to invest into Legendos.
- throw10920 3mo ago...which they already had: https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens-in https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens... The line "how could they do it in such a short time" is absolutely idiotic. The infrastructure was already there, it's massively parallel, and it's not like the only thing being distilled on is Fable.
- wongarsu 3mo agoIf by "distill" they mean "used it for fine-tuning" then they might have used it in the final stages of fine-tuning of Kimi K3. I image they might have already been using Opus, and when Fable became available it was easy to switch over to it It would have been a tiny part of the overall training, given the timeline
- supriyo-biswas 3mo agoHonestly, it wouldn't surprise me if they just found evidence of distillation once in 2025 against some Chinese AI lab, and they've been lying about the rest to create a narrative.
- warkdarrior 3mo agoI also heard that K3 stole the 2020 election, among other things.
- redsocksfan45 3mo ago[dead]
- superloika 3mo agoI think they deserve, by Justice, to have their models pillaged and raped, just like they did to the internet. They didn't ask for permission when they took the entire of the internet, after all, and given their behaviour is nefarious, it's of Justice that they receive nefarious treatment by others, including chinese AI labs. The Chinese are not gonna deterred, but the posturing by the Americans is so blatantly hypocritical that everybody is cheering for their demise. See, for example, one of Francis Fukuyama's latests videos on youtube.
- cwmoore 3mo agoI still believe taxing the bots, and implementing actual UBI, would address both problems.
- runarberg 3mo agoAnd I believe a socialist revolution in international solidarity of the working classes against our exploiters the capitalist owning class would address both problems as well (and more), but in the meantime I’ll be happy whenever I spot poetic justice in the wild.
- Chance-Device 3mo agoI think we have some historical precedents for this that went less well than you might hope.
- runarberg 3mo agoI am not aware of any revolutions where an international solidarity of the working classes overthrew their capitalist exploiters. I am aware of a couple of national solidarity movements which overthrew a monarchy (France; Russia) and a dictatorship (Cuba), but no international ones.
- Chance-Device 3mo ago
- throwa356262 3mo agoKimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies
- cute_boi 3mo agoEven if they distilled this crappy politician should have no issue. Anthropic pirated whole ebook collection and millions of github repo with gpl license. We should do more distillation and figure out how to create faster leaner and better models.
- tesch1 3mo agoBut training was ruled fair use, just the way they got the copies was illegal. Like distillation?
- sosodev 3mo agoDistillation is a very vague term. It can mean anything from training exclusively on a model's output to using it for a very small portion of the training. In this case it is almost certainly towards the very small portion side of the spectrum.
- hobonation 3mo agoI sort of did it. I got Fable to set up an AI system with better and better prompts within my app. At the end of it, Fable made me an AI system that works well enough that my users don't need Fable. Obviously, it's not K3 level. But Fable did just put itself out of a job in this case.
- Gregaros 3mo agoYou did not distill Fable. Relevantly, what you did provides no evidence contrary to the parent’s assertion that Moonshot did not have time to distill Fable.
- traceroute66 3mo ago"we have information" says a US Government official who almost certainly has had Anthropic and/or OpenAI on the phone spinning him stories. See also, don't trust anyone in Trump's government who says "we have information". "they distilled us" is fast becoming standard US FUD. The same as people telling me with a serious face that the Chinese models are distilled just because it says "I am Claude". I am not the only one, look at this post on interconnects about Kimi K3 for example:[1] It should be clear looking at this model that if adversarial distillation from the closed frontier models in the U.S. contributed, it is at most to a relatively small degree. AI observers who followed the distillation panic and came away with the wrong conclusion that Chinese AI labs are only producing good models due to IP theft are in for an awakening – that Chinese companies are extremely good at building models in the same way the leading American companies are. [1] https://www.interconnects.ai/p/kimi-k3-the-open-weights-escalation https://www.interconnects.ai/p/kimi-k3-the-open-weights-esca...
- kouteiheika 3mo agoAssuming they did then they surely paid for them, which makes it "not stealing". Am I also "stealing proprietary U.S. technology" by harvesting my Claude chats from my `.claude` directory and training a bunch of models on them? That said, I doubt the "they distilled Fable" is the reason why K3 is as good as it is, considering the timelines involved, and that Anthropic hides thinking traces, and their overly aggressive "safety" filters. This constant FUD spread by Anthropic is so tiring.
- sosodev 3mo agoModel distillation can't be stealing at all if you rationally apply copyright law to it. Anthropic is not deprived of Fable so there is no theft. At best it would be infringement, but even that might not hold up in the courts given the current position that model outputs can't be subject to copyright.
- deleted 3mo ago[deleted]
- yencabulator 3mo agoIt's at most a Terms of Service violation escalated into political theater because checks notes China bad.
- skeledrew 3mo ago> harvesting my Claude chats from my `.claude` directory Just reminded me to set a backup on that directory. Just in case someone sees it fit to override my setting to preserve my chats for 10k years.
- Catloafdev 3mo agoI wonder how they detect this kind of thing. Seems like this is going to be a perpetual issue until it stops being worth doing. Side note, didn't they stop releasing real thinking tokens for Fable? Or is it still part of some subs or API usage?
- mattrighetti 3mo agoIs distillation something we have to live with or are there ways to prevent it?
- sosodev 3mo agoRealistically you can't prevent distillation. OpenAI / Anthropic are slowly moving towards hiding the steps in-between input and output (hidden thinking), but that only helps so much. Imagine you put a file into Claude and say "do X to this" and it returns it to you without showing any of its internal reasoning. That's harder to distill, but the simple mapping of input to output still creates very valuable training data. It is reflective of all the training the model did to learn how to do that transformation.
- make3 3mo agoYou can also get it to think in the output tokens pretty easily, eg "Here's a math problem, I want your reasoning first, then the answer" which is what I assume they're doing.
- Cytobit 3mo agoYou make it sound like a bad thing.
- epolanski 3mo agoIf it was genuinely useful, we would've long reached the point where you train a model on a previous one's output in an ever improving loop. But this doesn't actually work.
- HarHarVeryFunny 3mo agoYou can prevent it by outputting a reasoning "summary" instead of the actual reasoning trace. Which Anthropic already do.
- sowbug 3mo agoIf you build a device that can help build devices, you shouldn't be too surprised when people use it to build devices.
- throwa356262 3mo agoIn the meantime, reddit is making fun of Opus for "distilling" Qwen: https://www.reddit.com/r/ClaudeCode/comments/1tqaist/opus_48_distilled_qwen/ https://www.reddit.com/r/ClaudeCode/comments/1tqaist/opus_48... (don't take this too seriously)
- deleted 3mo ago[deleted]
- feverzsj 3mo agoIf web scraping is legal, so is distilling.
- dgellow 3mo agoI would support distilling even if scraping wasn’t legal, I don’t think there is much of a relationship between the two
- bradfa 3mo agoI can understand that the AI labs might care about other labs distilling their models as it can eat into their competitive advantage, but do consumers care at all? Aren't consumers benefiting from this practice by getting better cheaper models as a result?
- warkdarrior 3mo agoI demand my right to pay 3x more for AI access. cf https://www.reddit.com/r/codex/comments/1uyj6pq/kimi_k3_is_13_the_price_of_fable_5_and_beating_it/ https://www.reddit.com/r/codex/comments/1uyj6pq/kimi_k3_is_1...
- gruez 3mo agoBut nobody really pays the 3x (ie. api) rates, except for enterprises. Everyone else are using the consumption plans, which are heavily discounted[1], possibly cheaper than even the chinese models, which don't do consumption plan discounts. Even in your linked reddit thread, the OP agreed with this sentiment. [1] https://x.com/SemiAnalysis_/status/2064815044085318040 https://x.com/SemiAnalysis_/status/2064815044085318040
- applfanboysbgon 3mo agoFor now. We've seen this pattern play out literally a hundred times in tech and you're incredibly naive if you think this will last forever. And what is your point? It should be okay for US consumers if the US government illegalizes accessing open-weight models within their borders because they're currently getting a subsidized token rate?
- bdcravens 3mo agoThere are many consuming APIs for Hermes.
- charcircuit 3mo agoI have paid API rates. I needed AI to cleanup file space and I didn't want to gamble with the alignment of Chinese models.
- tamimio 3mo ago“If you can’t compete with them, get them banned” - US AI companies
- make3 3mo agoThere's no real way to compete with someone who gets the output of your own work for almost free in comparison. I sympathize with the argument saying that they ripped the whole Internet and books first though
- jasonmp85 3mo ago[dead]
- m_ke 3mo agoAnthropic should think hard about all their fear mongering. It will only end up backfiring on them and everyone else involved. They definitely used closed private saas products to train their own models, to prove that just drop random small screenshots of any popular product behind a login screen and see how well it's able to identify all of them. ex: https://x.com/michalwols/status/2079968211865330165 https://x.com/michalwols/status/2079968211865330165 or other similar "AI" startups https://x.com/envconfig/status/2079613455296827402 https://x.com/envconfig/status/2079613455296827402
- mbix77 3mo agoDidn't they just pay a fine for stealing all those books?
- mrandish 3mo ago$1.5 billion fine for downloading 7 million books from LibGen and other pirate torrents. That's also the case where the judge ruled that training AI models on books could qualify as fair use, but storing millions of pirated works in a central internal library without licensing constituted copyright infringement. It will be interesting to see if courts consider training on data distilled from a model fair use. Assuming the allegation is true. Someone distilling data from a cloud-hosted model: - Paid the model creator to use a publicly available product. - Never copied or even had access to the model source code or weights. - Created a derivative work based on the model's responses to their particular input. - Trained their own model on the distilled output That distilled output is arguably a collaborative creation because a distiller's prompts are their own unique intellectual property. So they never pirated anything. I'm struggling to see how distillation is copyright infringement. At most it seems to be a paying customer violating one of the license terms, perhaps akin to a "no commercial use of derivative works" clause. But in the case of giving away an open weight model, is it even 'commercial use'? I guess if the distiller asserts copyright on the weights but gives them away, it's technically 'commercial' but even if they can win that argument, they're left with zero direct damages and suing for some value delta based on the alleged revenue they were deprived of. Is that delta the difference between the distilled model existing and the next best non-distilled open weight model existing? And then they have to collect damages from a portion of the revenue of third parties who commercially served that free model?
- wincy 3mo agoWell I mean it still worked out for them because they wouldn’t have had the 1.5 billion to license before doing the training and the company exploding into a trillion dollar company?
- xnoto 3mo ago"we ripped off the entire ecosystem of copyrighted data but I draw the line when we get ripped off"
- Geee 3mo agoYou wouldn't distill a car.
- solumunus 3mo agoThat tickled me!
- wmf 3mo agoI think Xiaomi already distilled the Porsche Taycan.
- caycep 3mo agodistillation of wheat, barley and malt is delicious, though!
- HarHarVeryFunny 3mo agoWhy not? Ford distilled Chinese EVs. https://www.businessinsider.com/ford-ceo-taking-apart-tesla-chinese-evs-jim-farley-2025-11?utm_source=reddit.com https://www.businessinsider.com/ford-ceo-taking-apart-tesla-...
- baq 3mo agoand the Chinese distilled western cars for years and years before that and weren't shy about it.
- 3mo ago
- solumunus 3mo agoGet your violins out folks.
- HeavyStorm 3mo agoPoor AI labs... All they hard earned training, done via scraping a lot of people works for free, now being scraped through payed subscriptions...
- madduci 3mo agoSo what is the issue here? Distilling is still fair, on the same level like Anthropic scraped copyright protected material for their training. So here robbers are blaming robbers? These claims are just pointless, everytime
- xnoto 3mo ago++
- make3 3mo agoIt's about the claim of whether these companies could develop a similarly powerful model without larger companies building their own first, which is an important point, and it's likely not the case. It's also about the larger companies explaining why they can't be as efficient, of course they can't, they're not just ripping the outputs of another model that someone else invested billions to train.
- PaulHoule 3mo agoSimply knowing it is possible to do something makes it easier to do.
- cindyllm 3mo ago[dead]
- doctoboggan 3mo agoYeah agreed, from one standpoint I couldn't care less that they did a "distillation attack", but I am interested in knowing if China is able to develop open weight frontier models without the prior existence of a huge model to distill from.
- IncreasePosts 3mo agoWhy would that matter? OpenAI or whatever frontier lab couldn't have built their frontier models without the entirety of humanity unknowingly developing their training set for 5000 years. It would be one thing if Moonshot was breaking into OpenAI servers and stealing trade secrets, but the only thing they are doing is looking at the output of the program, which is exactly the service that OpenAI offers. So, at best, this is a ToS violation. Sucks for the frontier labs I suppose, but live by the sword - die by the sword.
- caycep 3mo agohonestly if they did what he said they did, it seems like it would be cheaper just to train your own model from the get go
- browningstreet 3mo agoI haven't seen that point yet, and I was looking for it. Presumably Moonshot paid for that Fable access and Anthropic got paid. How much of the frontier model revenue stream is supported by paid distillation traffic? Obv paid kimi services are eating that on the other side, but money is changing hands at every stage.
- himata4113 3mo agoDoes this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre-cursor acquisition. And lastly, kimi architecture is vastly different than that of fable as it uses mechanisms developed by... kimi themselves. US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable. Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. edit: (moved this to bottom) The only argument they have here is that they use GB300 GPU's which for some reason should not be available to chinese citizens.
- JKCalhoun 3mo agoLegal, illegal… The word I would use is inevitable. It reminds me of the (PC) clones wars…
- random_coder_nz 3mo agoIt doesn't matter. It is most likely a pretext for upcoming actions mostly likely executed via yet another retarded executive order. The guy that posted this looks like he's drowned himself in the MAGA Koolaid.
- antisthenes 3mo agoIt also doesn't matter for a simpler, and much grander reason. All LLMs are trained on the corpus of humanity's knowledge, the legacy of everyone who's ever lived and our civilization as a whole. Anything that prevents or circumvents the accumulation or gatekeeping of this knowledge and puts it in the hands of more people (that are not AI company shareholders) is a good thing. Whether that is done by open sourcing the model weights, the training set, or by making the output better and cheaper, it is all fair game and is, as another poster mentioned, inevitable in the long run.
- 3mo ago
- jmward01 3mo agoIf 'distillation' means training on outputs then what is the legal concept of ownership of outputs? And, more broadly, is this something that could be skirted by doing it in different countries that have different legal structures? Basically, are they saying they own those outputs, not the companies that paid for the tokens, and only they can train on them? I suspect a lot of companies are saving their token histories and using them to fine tune internal models.
- nradov 3mo agoThe legal concept is that LLM vendors can put pretty much whatever they want in their terms of service, and cut off or sue clients who violate those terms. They have the right to refuse service to anyone for any reason (or no reason at all).
- deleted 3mo ago[deleted]
- jauntywundrkind 3mo agoTwo recent ones that really really hit me, > we're entering the most geopolitically volatile moment since the trinity test lit up the alamogordo desert and the only US policy prescription is a big button labeled sinophobia https://bsky.app/profile/thebadcode.com/post/3mr3skoyass2k https://bsky.app/profile/thebadcode.com/post/3mr3skoyass2k , and, > every vendor cranking the big dial labeled "sinophobia" and looking back at the us government for approval The government itself doing the propaganda here, skipping the vendors. Sinophobia intensifies. War drums of "be afraid be afraid be afraid" beat louder. It's so bad, it's so stupid. Kimi lands one showing pretty clearly this was absolutely the determining concern happening at vast scale, that they can just a lot of this themselves, and this noise pollution from the most hopelessly lost aggro administration ever still gets blared out the trumpets of war & discord. What a joke. Give me a break, give it a rest. War here is less winnable than the Iran war they started. They're going to make America itself so much worse, these people so hungry to put down free and good models. This pathetic attempt is not going to work, you are just going to once again hold the US citizens hostage & make their lives worse, for sick political games.
- orangecat 3mo agoNot surprising that Bluesky is perpetually in peak woke mode, but the racism claims are absurd. If Russia were doing the same thing, would we have no problem with that because they're white?
- jauntywundrkind 3mo agoThis feels radically off target to me. Sinophobia as such (not always, and there's obviously a relation) isn't about the Chinese people, about their racial identity. It's about a menacing aggressive world power (actually two such MAWPs in this particular example), about a tired anti-Communist McCarthyism (which has always been an excuse to clamp down on the left/progressives, a menace to free speech). If Russia were still the USSR and the cold war hadn't ended and they were our AI competition, & where giving away the latent matricies of reality that the US profiteers extracted by stealing all the worlds knowledge illegally, and which they want to use to raise the ladder & leave a permanent plebeian underclass, we the US imperial fatcats would be doing the same sabre rattling and fearmongering and tension raising for sure. Sinophobia here is just a particular fear of "other" for the only other that's relevant. And that sucks, no matter who it is we are trying to other here, no matter what phobia the propagandists of the GOP and Technofascism are spinning, ginning up. To fixate is to be unable to see the point.
- avazhi 3mo agoAnd? Nobody cares. This is neither a controversy nor news, and that would be the case even if Anthropic hadn’t just settled a 1.5 billion dollar lawsuit where they trained Claude on thousands of books without permission lol. To be clear I’m not taking a jab at OP - I’m saying the labs crying about distillation have neither a legal nor a moral leg to stand on. There’s nothing wrong with distillation.
- sajithdilshan 3mo agoHow the tables have turned. It's okay for Anthropic to train their models on copyrighted data, but it's wrong to steal the stolen data from Anthropic models.
- NichoPaolucci 3mo agoI wonder if this points at a “shared” future (or at least things will eventually converge there whether companies like it or not). Ultimately, if you’re going to release these models that are fundamentally built on shared data - it’s pretty wishful to assume you’ll be able to harbor that model and the data, forever, and profit from it. It also leads me to think about things like the original release of Fable 5, people were complaining that it was safeguarded too much - if you lock the models down too much they cease to be useful. So it’s going to be increasingly difficult to protect a model from competition while ALSO keeping it useful.
- nradov 3mo agoWe might see a future where the US frontier LLM vendors place really strict licenses on them. No consumer access. Only sell to enterprise customers in a limited set of countries, with heavy monitoring and auditing down to the individual employee user account level. (I'm not saying that this is a good thing, just that some LLM vendors might try that approach to maintain their "moat".)
- guess_who_is 3mo agoIf you ask fable, it will identify as deepseek
- Chance-Device 3mo agoHmm. I wonder when this was detected. And was the CoT trace cut from Fable from the start on June 9th or just after the export ban and relaunch? Is this what the export ban was actually about? I honestly don’t know, just wondering aloud.
- mrbonner 3mo agoHah tales as old as time. what’s next? Distillation of Disney theme park?
- skeledrew 3mo agoSuper interesting. So Fable was really made available... a couple weeks ago? And K3 a few days ago? That's a really impressive feat to distill enough data AND train AND review to get a release that works really well in that time period. Mad props to the Moonshot team :flame:.
- tristanj 3mo agoFable was launched on June 9 (for 72 hours), then K3 launched on July 16. Timeline wise, Moonshot had over a month to post-train K3 on Fable distilled data, which is more than enough time.
- skeledrew 3mo agoSo in 3 days they were able to distill enough data to make a significant difference in the K3's performance? Without triggering any limiters when there's already suspicion of distillation? That's still a heck of a feat. Like the bank robbers who were able to keep coming back to take more even though the bank was aware they had gotten robbed recently and had the resources to put sophisticated security systems in place.
- tanh 3mo agoFor code can't they distill from public GitHub commits? If they could figure out who used Mythos/Fable assitance in the commits.
- SwellJoe 3mo agoI think they're distilling "reasoning", not merely code. There's plenty of human generated code. What they're trying to extract is the process by which really large models "think" their way through complicated problems. That's what all the "traces" datasets on HuggingFace are about.
- dmitrygr 3mo agoOMG someone used our data to make an MK model! Just like we did to every author in the world!
- neals 3mo agoHow does one distill? Just send a million request asking for information? Start with the letter A?
- verdverm 3mo agoProbably the agent workflow traces, including thinking sections, are of main interest. Used in late training for decision making and problem solving strategies.
- juancn 3mo agoSo? We have information that Fable was distilled from humans. If it works it works. Isn't that the argument? AI outputs are not copyrightable, so distillation is fair use. It may be a TOS violation, but that's a private matter. Cancel the accounts used for distillation and be done.
- mbmbn 3mo ago[flagged]
- codedokode 3mo agoSmartphones are mostly Chinese now (except for iPhones and Samsung).
- strictnein 3mo agoApple and Samsung are ~40-50% of the global market, depending on the source. Saying that smartphones are mostly from Chinese companies isn't accurate. And Samsung and Apple are gaining market share, while the major Chinese brands are losing market share. https://www.idc.com/promo/smartphone-market-share/ https://www.idc.com/promo/smartphone-market-share/ https://gs.statcounter.com/vendor-market-share/mobile/worldwide https://gs.statcounter.com/vendor-market-share/mobile/worldw...
- hdaz0017 3mo agoiPhones are predominantly made in China Foxconn facilities in cities like Zhengzhou (known as "iPhone City") and Shenzhen.
- strictnein 3mo agoYes, but I was responding to this comment: "Smartphones are mostly Chinese now (except for iPhones and Samsung)." Which implies that we are talking about the companies, not where they are manufactured.
- jerrythegerbil 3mo ago“However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.” What’s actually happening behind the scenes is that certain inference providers will classify a prompt and it’s re-routed transparently to Anthropic and that’s used for distillation training, only distilling the complicated traces they need, originating from real user prompts and traces. These inference providers are explicitly blocked in the claude cli if you reverse engineer it. The real picture is that these Chinese labs have figured out how to get exactly what they need, at a high quality, directly from distinct and unique real user prompts. It’s only “covert” because Anthropic doesn’t like it, while simultaneously being perfectly fine to do.
- throw10920 3mo ago> What’s actually happening behind the scenes is that certain inference providers will classify a prompt and it’s re-routed transparently to Anthropic and that’s used for distillation training Uh, no. There are Chinese networks of tens thousands of fake identities specifically to get access to Anthropic models directly. https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens-in https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens... Don't make up stuff and/or lie to suit a political agenda. It's extremely dishonest.
- grim_io 3mo agoSo, if it's that easy and fast to "copy" Fable, is it really worth that much in the first place? Sounds like the opposite of the conversation Anthropic would want to have.
- pandinus 3mo agoAs with many others among these threads I don't see how the timing works out for K3 to have trained on distilled Fable usage. There should be at least a tacit academic acknowledgment of Kimi's own design efforts. Distillation itself, however, is still clearly valuable - else competitors wouldn't pay so much to their rival on distillation campaigns or try to circumvent anti-distillation defenses. As for the morality of it, if you paid for the tokens they're yours. It is already understood that you own the output. Seems to me like a variation of ordinary business arbitrage. Providers might object to certain use-cases or intention and try to craft terms around that, but that's hard to enforce at scale.
- SwellJoe 3mo ago"It is already understood that you own the output." I don't think it's settled that anybody owns the output. There seems to be some question whether LLM output can be copyrighted (and there should be). I'd rather it weren't possible, actually. I think it's better for humanity if we acknowledge that what was legitimately ingested into these models is our collective commons (and what was illegitimately ingested into these models also shouldn't exclusively profit the people who illegitimately did so). I don't know how that squares with the AI industry recovering its trillion dollars in investment, but I reckon they should have thought of that before.
- syrrim 3mo agoThere's no question about it. Llm output is not copyrightable. Which is moot anyways, because training on copyrighted data is completely legal.
- catigula 3mo agoI’m certain they did. The problem is… what are you going to do about it? This is obviously an idiotic and dangerous Cold War and has no happy ending.
- MiguelVieira 3mo agoHere's a site that asks the same questions to 22 models and compares how similar their responses are. https://typebulb.com/u/lab/you-re-relatively-right/full https://typebulb.com/u/lab/you-re-relatively-right/full According to these results GLM 5.2 is very similar to Google Gemini and Kimi K3 is very similar to Fable 5. The American frontier labs are not similar to each other.
- xyzsparetimexyz 3mo agoInteresting. This dooes lend credence to the distillation idea. Good for them!
- trollbridge 3mo agoGLM 5.2 is light years ahead of anything called “Gemini”.
- qeternity 3mo agoI think it’s fairly obvious the Chinese labs are doing mass distillation. I also think the Fable accusation is wrong and it was most likely Opus 4.8 which itself is likely a distillation of Fable.
- sailingparrot 3mo agoYou have it backwards IMHO, obviously can't prove it, but I would bet that Opus was used to bootstrap Fable. It becomes very confusing since we have started calling everything distillation, but most likely what both Anthropic did for Fable and Moonshot did for K3 was using Opus traces in the reasoning SFT stage during mid training.
- thundoe 3mo agoThe Irony. These models have been created distilling Internet without ever asking for permission or paying anyone. Internet was the first model.
- nozzlegear 3mo agoCry about it IMO. Anthropic reaps what they sow.
- stranded22 3mo agoSeems like an advert for K3 to me. Fable level performance, for much lower price. But really, this is the USA getting ready to bring AI companies completely under the control of the Trump administration for ‘national security’
- yeodev 3mo ago[flagged]
- codedokode 3mo agoDo you by chance also have information about Anthropic's training data sources?
- lowbloodsugar 3mo agoChina has done this with absolutely everything, starting with “customs inspections” of ships engineering sections by “inspectors” drawing diagrams of what they see. Bit late to be worrying about it now. This only matters now because China is now near parity in tech and vastly superior in production ability. Meanwhile we run out of bullets in a five month war with Iran.
- Jaauthor 3mo ago"Well, Steve, I think there's more than one way of looking at it. I think it's more like we both had this rich neighbor named Xerox and I broke into his house to steal the TV set and found out that you had already stolen it."
- cmiles8 3mo agoBut wasn’t fable distilled from knowledge taken from others? I get why Anthropic is angry here, but it would appear they’re not really in a position to complain about this.
- storus 3mo agoI doubt they did any distillation as Hinton defined it (requiring logit access). They most likely ran a bunch of prompts/conversations and captured the results. Those conversations already missed thinking tokens, replaced by some confusing quasi-summaries. Then they took those and ran basic SFT or maybe DPO if they had competing responses. As there is no copyright on the output of AI, I am not sure where is the "covert industrial distillation" part of the problem.
- mrhottakes 3mo agoGood. If Fable is really so smart, it wouldn't let itself be distilled.
- martinjc 3mo agoSo what? I want the best model at the cheapest price. You guys illegally trained on books, movies, audiobook etc.. Why should we care?
- deaton 3mo agoWho cares. Anthropic distilled the entire internet, and then a good bit more beyond that.
- alastairr 3mo agoPresumably the frontier labs themselves can do their own distillation far better than the chinese labs can. Why can't they just beat them at their own game and release / host low cost intelligence and own the whole game. There will always be a market for the more expensive frontier intelligence.
- stephbook 3mo agoI don't know what purpose these "they copied us" crying is ever going to achieve. Europeans stole Chinese silk worms. US stole European books, looms and rocket scientists. Who cares? Be grateful you've got people inventing stuff worth copying.
- drop_star 3mo agoAmerica, the perpetual victim
- mrhottakes 3mo agoWe're winning so much, we're getting tired of winning
- surgical_fire 3mo agoFirst: Even if true, I don't care. Second: Post is rich with allegations but light with evidence. Can very well be bullshit.
- 4chandaily 3mo agoSeems to me like Moonshot is a paying customer, and if their business isn't worth the money Anthropic is charging, perhaps they should raise the price per token charged for it. Otherwise, I don't see why this is a story. "AI Company pays another AI Company for training data" just isn't that interesting.
- riknos314 3mo agoIf it's true that in under 15 days of access significant improvements were realized in K3, then the moat of closed-weight models is far smaller than previously thought. Doesn't bode well for the valuations of these labs.
- teravor 3mo agothe distillation everyone talks about in respect to LLM's isn't nearly as easy as most think. none of the frontier labs provide probability distributions over the tokens which is the actual method of distillation you use to train a smaller model based on a larger one. they don't even provide all the tokens. therefore this so-called distillation the frontier labs whine about is just a set of clever methods to work the existing LLM into the training process for a new model. methods like having the existing model grade the output of the new model and work those grades into the RL method. give the new models structured tasks and use the existing model as a source of truth for those tasks and a myriad of other hacks. efficiency scales with the gap between the models and generally allows an efficient bootstrap process. the implication that distillation wouldn't allow further advancement is false however, you can then start doing the same thing the frontier labs have been doing: dumping cash on humans to provide the signals or burning tokens on exploratory paths and grading the results. what openai and anthropic don't like is that fact that all the cash they burned can be used to benefit everyone and not just them. and that no matter how much more cash they burn to build up the gap it will closed at a small fraction of the price.
- deleted 3mo ago[deleted]
- 4ndrewl 3mo agoSo? Their business model requires building on the labor of others for free. Isn't that how it works?
- wnmurphy 3mo agoIt's funny to me that these models were created by effectively "distilling" all available content including the proprietary works of many other people, but now it's a problem that someone is doing the same to them. You're using available information (copyrighted works, or the output of another model) to train a model to encode the information in a new form. Why is the former not theft, but the latter is theft?
- stldev 3mo agoSo company who stole stuff to make their stuff is mad because another company is stealing their stuff. And now a regime best known for lying to their own people is the one trying to convince me? Go, China!
- sleepyguy 3mo agoIs this a surprise, I think history has proven that the Chinese technology theft is part of their strategy. They let the American tax payer or "The West" shoulder the cost and then steal it. Waiting for the whataboutism....junk away...
- buellerbueller 3mo agoIt wasnt until 1891 that America extended copyright protection to foreign authors. https://en.wikipedia.org/wiki/International_Copyright_Act_of_1891 https://en.wikipedia.org/wiki/International_Copyright_Act_of... IP "theft" has been a longstanding part of any developing nation's economy.
- Perenti 3mo agoYes, China stole the recipe for pasta and noodles from America. China stole gunpowder from America. China stole printing from America. China stole the idea of paper money from America. China stole the idea of entry exams from America. China stole the invention of the rocket from America. Seriously, do you really believe this American Bubble bullshit that everything of value was invented in the USA?
- sent-hil 3mo agoReminds of the quote by Bill Gates. > "Well, Steve [Jobs]… I think it’s more like we both had this rich neighbour named Xerox and I broke into his house to steal the TV set and found out that you had already stolen it." Source: https://www.goodreads.com/quotes/824084-well-steve-jobs-i-think-it-s-more-like-we-both https://www.goodreads.com/quotes/824084-well-steve-jobs-i-th...
- NetOpWibby 3mo agoThat's an amazing quote LMAO Wow.
- chasd00 3mo agoif there are no consequences then who cares? You're not going to take Chinese companies to court and stealing IP is nothing new either. It's going to take some sort of policy change at the federal government level to do anything but they haven't done much up to this point. Maybe AI is important enough to actually get some kind of policy change, sucks for everyone else who have had their IP stolen with no consequences whatsoever.
- gensym 3mo agoI fear they are laying the groundwork to ban US citizens from using Chinese models, so that they can make sure that Brockman gets his money's worth.
- chasd00 3mo agowhy would you fear that? At least it's something to fight unfair business practices. As for American AI companies violating copyright there's a venue for that, the courts. File a case if you feel your copyright has been violated and get your day in court.
- deleted 3mo ago[deleted]
- muldvarp 3mo agoOkay? We have information that Anthropic sucked up all of the internet for the development of Fable.
- bakugo 3mo agoFun fact about K3's distillation: As of a couple months ago, when using Claude to write adult content through the API, sometimes it will silently inject a system prompt giving the model a bunch of guidelines on exactly what kind of adult content it's allowed to write, steering it away from anything "questionable" ("Claude will not write etc etc"). Moonshot distilled Claude so hard recently, they actually ended up distilling this prompt injection, too. Using K3 to write adult content results in it randomly hallucinating the injected Claude prompt during thinking, and it will quote parts of that prompt, complete with the name "Claude". Not that I think distillation is a bad thing, just thought this was funny.
- scronkfinkle 3mo agoso they distilled one of the best models in the world AND released it for free to everyone. Where can I send them flowers as a thank you?
- stego-tech 3mo ago[flagged]
- dang 3mo agoCan you please not fulminate or post flamebait on HN? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html. You may not owe AmericanAIBros better but you owe this community better if you're participating in it.
- stego-tech 3mo agoI’m familiar with the guidelines and I stand by what I said, including how I said it. This has been a regular talking point from AI companies since DeepSeek first hit the scene, and I feel a glib response to what is very clearly an insincere and nakedly hypocritical talking point is warranted after years of this slop. Hypocrisy doesn’t warrant professionalism, it warrants corrective action; in text, the best I can offer is a tone and tenor that matches the original argument. Considering these dolts have now made the claim that open weights somehow equates to AI communism, this sort of response is even more necessary than before to reflect the complete absence of decorum from the people making these grievances in the first place.
- deleted 3mo ago[deleted]
- brap 3mo agoAnyone surprised by this is incredibly naive. By all means use whatever works for you, I’m not even going to try to make an argument on ethics (and honestly I’m not even sure where I stand, given the behavior of American AI companies). But I just cringe every time I see people acting like any of this is done in good faith. Open source coming out of China is a state-sponsored criminal enterprise, built only for the benefit of the Chinese regime, one of the worst to exist in human history.
- NetOpWibby 3mo agoGOOD I love using Claude but Fable's unusable wrt useful work like cryptography, biology, &c. Kneecapping my productivity when I pay $100/month is annoying af.
- nmeofthestate 3mo agoWeird 'discussion'. Almost entirely single messages with no threads, all with the same anti-Anthropic/AI position.
- stratos123 3mo agoYeah, some AI-related posts on HN are strange. It might be real people coming to gloat whenever they see a title they like, but I sure hope somebody is checking whether they are real people.
- kamranjon 3mo agoSo here is an important question I think. If LLM outputs aren't copywriteable and you create your own synthetic training set using Fable and share it publicly on huggingface, and someone else uses that training set to fine-tune a model, would this be considered illegal? I ask because this happens all the time, synthetic datasets have basically become a key aspect of training a model at this point. I even generated a synthetic set from DeepSeek v4 to aid in fine-tuning a classifier just a few weeks ago. So I just wonder on what grounds any of this makes sense, I wouldn't be surprised if some of these American labs were using open models on their own self hosted infrastructure to generate training data, but by nature of them being open nobody has to know. I'll make a prediction: I don't think we will ever see any of the evidence of this "distillation" before they end up implementing some type of ban.
- econ 3mo agoI have an idea! If they are so hungry for citable content they should start a cheap or free blogging platform with images and video and a blogroll and verified credentials and resumes, with your own html css etc and domain name and a git server and a mail client and their own advertisement platform and aggregator and a chat platform, scientific journals too obviously, tools for writing and publishing books and documents. API available everywhere to avoid training on its own output. Because there is no way in hell I'm going to make an effort creating quality content for existing platforms. The website should be entirely my own without moderation subject only to my local legal system. Can just insert this comment as a prompt and vibe code everything in a few days⸮
- deleted 3mo ago[deleted]
- strictnein 3mo ago[flagged]
- sailingparrot 3mo agoYes they distill, but if you think you can trivially get a frontier-level model by "just" distilling from Claude's public API. you fundamentally do not understand the amount of work that goes into a modern post-training stack. Without even talking about the fact that any distillation that was done was on Opus, as the timeline of Mythos/Fable vs Kimi 3 release dates just do not match in any plausible way. If you want to read an educated take from someone that has actually spent the last few years working on post training I recommend Nathan Lambert's: https://x.com/natolambert/status/2079616308203942332 https://x.com/natolambert/status/2079616308203942332
- strictnein 3mo agoVery interesting, thanks!
- trollbridge 3mo ago“Distillation” is just a term of art these days. Usage of competitors’ models these days is mostly around RLHF (minus the H, I guess).
- calendar938 3mo agoLet's be real. In reality, it's the Chinese kid getting the 1600.
- strictnein 3mo agolol, true true. Should have used a different example.
- anfogoat 3mo ago> Do we need 40 people saying the same thing about how they don't feel bad and it serves them right and all that surface level stuff on every single one of these? Apparently we do, given that we've got govt officials wasting time, money, and effort on whining about this now.
- NDlurker 3mo agoGood. Keep it up
- bparsons 3mo agoIP protections for me, not for thee.
- tacone 3mo agoSo it is as "dangerous" as Fable?
- rambojohnson 3mo agowho cares. all these frontier models are trained on theft.
- thih9 3mo agoOne of the replies: > @MehdiKarech > I don't remember letting Anthropic or Open Ai scrapping my GitHub, my research gate and all my online writings L O L https://xcancel.com/MehdiKarech/status/2080000779859939678#m https://xcancel.com/MehdiKarech/status/2080000779859939678#m
- jchw 3mo agoIs this person trustworthy? I struggle to believe that in the relatively short time Fable was available it has already been distilled so effectively. If this really is actually true, very impressive work.
- Gud 3mo agoOK, DIRECTOR Michael Kratsios, but why should we give a shit? American AI corporations are pushing up the prices for computing, making it unaffordable for the common man. Additionally, they have built their entire business on stealing(yes, stealing) work from us. So fuck em
- blaufast 3mo agoThe frontier labs' work is more akin to discovery than artistic expression. An art piece is valued for its uniqueness and individuality, but AI is valued for verifiable correctness. Discovery cannot be unseen and is easily replicable. I think the AI labs are in a tough situation because their work is more similar to fundamental scientific discovery than say, a unique painting or song. Mendel doesn't get a cut every time somebody uses the principles of heritability he discovered, and Einstein's family aren't getting royalties if you compute relative speeds. I think the frontier labs should expect to be treated more like scientists than artists in this regard.
- linkregister 3mo agoCommenters are overlooking the significance of this information and posting emotional reactions based on perceptions of fairness or feelings of schadenfreude. The economic viability of Anthropic and OpenAI rely on their being able to charge more for model access than their R&D and inference costs. If the market price for SOTA model access drops below that level, then these businesses will have to decide whether to continue to lose money or to reduce spending on R&D. Moonshot's papers [1] claim that their training load was primarily from synthetic data and model self-teaching rather than RLHF and therefore keep their costs low. If Moonshot genuinely does not rely on human-led training, they will surpass US closed-source model providers. The United States government considers US supremacy in "AI" as a national security consideration. This announcement is noteworthy because it implies that Moonshot's success is in fact due to distillation. It's in the interest of US frontier labs to place barriers to this if they find themselves in the position of subsidizing rival labs' research. 1. Kimi K2, https://arxiv.org/html/2507.20534v1 https://arxiv.org/html/2507.20534v1
- asadotzler 3mo agos/announcement/claim You don't get to call Moonshot's a "claim" and this political hack's an "announcement." They're the same thing. Treat them the same. Diction designed to favor one of two equal positions is some weak sauce.
- linkregister 3mo agoYou're calling someone a political hack, but imposing neutrality on my statement. I don't even necessarily disagree with your assessment of this spokesperson. But you must admit how inconsistent you're being.
- amazingamazing 3mo agoIt doesn’t matter. Distillation is impossible to stop. They could release an extension that intercepts requests and in return gives you a discount like Honey and get the same data.
- 3mo ago
- softwaredoug 3mo agoOpenAI and Anthropic should enter into distillation agreements with other US labs. Turn a threat into a profit center. Other US labs cannot directly distill from OpenAI/Anthropic as it’s a violation of the terms of service. It holds other US labs back. Leading them to build second tier models And in the end OpenAI/Anthropic may be unable to prevent distillation. Why fight it when there’s clear money to make here?
- gozucito 3mo agoOpenAI/Anthropic are already charging for access to their closed models. They're getting paid. They are also not interested in agreements. They want to keep as big a moat as possible because they love money. And you need two to tango.
- softwaredoug 3mo agoYeah they may not be interested. But pretending you have that moat a dumb strategy that's not working.
- 062570864389 3mo ago[dead]
- alexruf 3mo agoWho cares? Is it theft if a thief gets robbed of their stolen goods? Gives me more of a modern Robin Hood vibe tbh. No, seriously: first of all, that's not the AI labs' data, it's ours. And if the AI labs think they can rake in tons of money using our data, then I'm actually glad if someone comes along and at least offers us a good product at reasonable prices.
- BigTTYGothGF 3mo agoGood for Moonshot.
- asadotzler 3mo agoA company that distills LLMs should be called "Moonshine" not "Moonshot" Ba-dum-tss
- bradrn 3mo agoHah, I had the same thought on seeing the title!
- wseqyrku 3mo agoEvery model is distilled internet. This is just the natural progression of that.
- ekelsen 3mo agoReminds me of this classic line from the 1973 movie The Sting: "What was I supposed to do? Call him for cheating better than me in front of the others?!" Said in response to being out-cheated at a high-stakes poker game. Except in this case, it sounds like that's exactly the path they have chosen. https://getyarn.io/yarn-clip/7612c4ce-1077-479f-a7bf-617dbc6fc6d1 https://getyarn.io/yarn-clip/7612c4ce-1077-479f-a7bf-617dbc6...
- sensanaty 3mo agoFor a buncha supposed capitalists, they sure do hate fair competition eh?
- eyevz 3mo agoK3 frequently refers to itself as Claude in reasoning when instructed to play a role.
- deleted 3mo ago[deleted]
- zuzululu 3mo agoIf distilling is fair then so is banning it. It's funny how people cry about Anthropic using books and materials to train itself but then when Anthropic does something about it they think its unfair. Pick a lane.
- deleted 3mo ago[deleted]
- sscaryterry 3mo agoHow much credible, provable evidence? None really.
- seydor 3mo agothe bar for credibility in the US administration is in the subatomic scale. proof is generally not even needed
- nahuel0x 3mo agoInformation wants to be free.
- felooboolooomba 3mo agoThe pot calling the kettle black.
- cyanydeez 3mo agooh know, better make them pay up your legal fines for stealing all that from the public good.
- guybedo 3mo agoi have information Anthropic distilled thousands of books, articles, etc ... with their author consent.
- goldenarm 3mo agoI have information that Anthropic distilled the internet for Fable
- GiorgioG 3mo agoI don't even understand this fucking AI "arms race" at all. Who can burn money the fastest?
- ProofHouse 3mo agodidn't Anthropic distil a few tens of millions of books?
- seydor 3mo agoGood pretext for bannign chinese APIs in the USA and their vassal states. If distillation is so good, why aren't US companies distilling each other?
- goldylochness 3mo ago[flagged]
- InsideOutSanta 3mo agoWe already know K3 is really good; you don't need to glaze it even more by telling us it's like Fable.
- blks 3mo agoConsidering that LLM are created using stolen content or against licensing, it’s only fair to distill and open source them.
- matheusmoreira 3mo agoSo what? Am I supposed to be upset by this? Distill away. Actually, can I help out somehow? As long as they keep publishing open weights, I'll give them my full support.
- cregy 3mo agoI think the major take away is, Chinese labs are very good at stealing others AI work and offering it did a fraction of the cost. This will be the reason the AI stock market bubble bursts. Unless they add in protection similar to patents, which stops these copied models being used by business.
- JumpCrisscross 3mo ago"Samuel Slater (June 9, 1768 – April 21, 1835) was an early English-American industrialist known as the 'Father of the American Industrial Revolution', a phrase coined by Andrew Jackson, and the 'Father of the American Factory System'. In the United Kingdom, he was called 'Slater the Traitor' and 'Sam the Slate' because he brought British textile technology to the United States, modifying it for American use. He memorized the textile factory machinery designs as an apprentice to a pioneer in the British industry before migrating to the U.S. at the age of 21." https://en.wikipedia.org/wiki/Samuel_Slater https://en.wikipedia.org/wiki/Samuel_Slater
- some_random 3mo ago[flagged]
- JumpCrisscross 3mo ago> slavery, extractive colonialism, punitive expeditions, etc? You're seriously comparing intellectual property transgressions to slavery and colonialism?
- kjs3 3mo agoPeople who built a business model around stealing other peoples stuff are vewy, vewy upset that someone has built a business model around stealing their stuff. Oh no! Anyway.....
- exabrial 3mo agoOh the irony... LLMs go and read a billion pirated book, but now are crying when their models get distilled from a million queries.
- laweijfmvo 3mo agoif its that simple, why doesn’t Anthropic just distill its own models and release Fable 5.1, 5.2, ...?
- stratos123 3mo agoDistillation lets you train a comparable model for a fraction of the cost (it's effectively a way to get a lot of very-high-quality training data). If you're already at the frontier, pushing it requires the ordinary, expensive kind of training.
- overgard 3mo agoIt's both acceptable and inevitable. Play stupid games, win stupid prizes. This is what they deserve for the awful way they treat creators and creative people's intellectual property and livelihood.
- chriswunan 3mo agoHuman only have this much knowledge. They will become similar anyway.
- mikelitoris 3mo ago“Hey stop stealing my data. I stole it fair and square.”
- jstummbillig 3mo agoI think at some point the issues around copyrighted work and model distillation have to be disconnected to advance either idea. 1) Compensation of right holders is one issue. 2) Distilling models is an entirely separate issue, because model building is value add, and that is important because if we arrive at a place where you can produce a model, that gets to ~100% of what people perceive of the models value (on top of also not compensating right holders, yourself) you are discouraging development of better models and, again, in no way helping with issue 1) Unless anyone actually distills a model and then also does something for rights holders, any schadenfreude simply detracts from this issue, in addition to the other issue (well, that might not be an issue if we would rather slow down model development right now, but again, forever worse models still don't help solve issue 1)
- CamperBob2 3mo ago"No fair stealing what we stole fair and square"
- oybng 3mo agoWho cares. It's all theft
- zetazzed 3mo agoWho will invest in generating data for the frontier of AI if their output will immediately be used to train a competing model? Forget China vs. US, this applies within-country too. After exhausting all the publicly-accessible data on the internet, the frontier labs started spending hundreds of millions of dollars to generate data across a variety of fields. Just look at Mercor doing $1.2bln/year with 90% coming from the top labs (https://www.theinformation.com/articles/mercors-fast-growth-relies-biggest-ai-companies-documents-show https://www.theinformation.com/articles/mercors-fast-growth-...). That pushes ahead what AI can do in medicine, science, coding, and math. But if other companies are going to free ride on this investment, it doesn't make sense to continue. So AI will largely hit a wall, frozen at the current level and work will all shift to cheaper inference.
- sailingparrot 3mo ago> So AI will largely hit a wall, frozen at the current level and work will all shift to cheaper inference. Looks like an ideal outcome to me until (if) we are able to solve the alignment issue.
- xinayder 3mo agoI wonder if this was caught with the malware code Anthropic included that detects if you're in China...
- OhNoNotAgain_99 3mo ago[dead]
- Alifatisk 3mo agoHonestly, I don’t have any sympathy at all. Anthropic can complain all they want, but they seemed fine with pirating books. What Moonshot AI has done is to offer almost Fable 5 comparable performance at lower prices than Anthropic insane margins. This is what I call competition, which the Director seem to embrace. This is what the Chinese always been good at. Take expensive innovation and streamline it to lower prices. But we are at a point where labs like Moonshot actually contributes a lot to the research field as well. They are pushing the innovation forward and squeezing the prices. Very well done. Whats even weirder is the bizarre mechanisms Anthropic implemented to prevent distills which they had to sacrifice their customers for. They hid the internal CoT reasoning and returns summarizations instead. This made it difficult for users to trace things. They made Fable 5 silently switched over to Opus 4.8 if it detected blacklisted prompts (almost anything triggered this) to sabotage distills. And now, they are still complaining about distills? So their customers have gotten sacrificed over nothing. Whats even weirder is the timeframe here, no way the Moonshot team managed to plan conduct a large scale distill, then pre-train, RL, fine-tune, benchmark, marketing and release to their platform since Fable 5 got whitelisted. > they developed a sophisticated internal platform to conduct large scale distillation I am very curious about this and would love to learn more on how they did this. Wish we had more details. I know the team behind DeepSeek have also done clever things to distill too. I am aware of these ”transfer stations” that acts as a proxy, but I don’t think they are helpful in this case.
- pizzly 3mo agoSo this means sanctions. U.S. Treasury Secretary just threatened sanctions if they discovered Chinese models are distilled from American models. https://www.cnbc.com/2026/07/21/bessent-china-ai-sanctions.html https://www.cnbc.com/2026/07/21/bessent-china-ai-sanctions.h...
- zmmmmm 3mo ago> We have information ... Hard to think of a weaker way to express this. Strongly suggests veracity of said information is poor.
- phs318u 3mo agoDo I, as a paying user, own the output the LLM has produced or am I merely licensing the output? Most paying users assume ownership, in which case I’ll do with that output what I want. If the LLM outputs are not owned by the user, but are actually licensed, please clarify the terms of commercial use.
- program_whiz 3mo agoTo borrow the argument from the apologists: But everyone learns by example! How is this any different from a person just reading the outputs of Fable, learning, then producing output. Surely reading outputs, gaining knowledge, then producing work isn't illegal, or all art/writing would be illegal. Funny how that argument seems so vacuous in this situation, yet others find it compelling when justifying the mass theft of art and writing for model creation. In this case the model is "just learning priors" before it "creates its output which is novel", nothing problematic.
- dudeinhawaii 3mo agoI think most of the comments in this thread are missing the point. It's not about whether it's legal/ethical/etc. It's about the narrative that "Chinese models are at Fable level". The truth (if correct) is the China continues to copy, and the proprietary US Models continue to lead the state of the art. There is no K4 without Fable 6, GPT-6. That, matters.
- sailingparrot 3mo ago> There is no K4 without Fable 6, GPT-6. That, matters. That's simply not true though. Chinese labs very clearly have the entire stack developed and working. Using traces from claude allows them to shorten their training time by some amount, that's it. Remove Fable 6 and you still have K4 eventually, just 2 months later at best.
- dudeinhawaii 3mo ago[flagged]
- syngrog66 3mo agoworld's smallest violin plays...
- TZubiri 3mo agoThe discussion of "haha LLM companies stole data and now they have their data stolen so it's the same thing and it's fair." was reductionist when it started, and it's been like 3 months, and every internet user throws it like it's the hottest take ever, have another take please. Also have nuance, don't jump to hit your HOT_TAKE key in your keyboard, actually read what the chinese are doing, and then you can pass on your judgment on whether it's ok or not. It's not the same thing if they scrape an openly published dataset and it's an IP dispute. Or if they are using,network and financial pooling mechanisms that are shared with CSAM providers and cybercriminals, mutually providing each other alibies, and using black markets of passport-backed identities to setup thousands of accounts and circumvent bans and detection. While we are at it, if there's a case that was settled, it's a closed case, it can never invalidate any other disputes. That case is closed, and it was settled by the parties that claimed to be damaged, that's done. If you didn't think so, you wouldn't have taken the settlement, and if you didn't have a say in the settlement, it's because you weren't damaged so who cares, go make a claim where you are the defendant if you believe otherwise. But thankfully in no legal system does the existence of a claim against you prevent you from making claims of your own. Nuance is a good thing.
- Stevvo 3mo agoTaking statements from this administration at face value is foolish. They have put out so many falsehoods that its safer to assume all of it is false.
- Grimblewald 3mo agoAnd I have evidence that anthropic distills from openai, moonshot, alibaba etc. So unless anthropic holds itself to the standard they're implying should be followed, why should I care others did to them as they do unto others? Seems like a nothing burger. Has the same vibe as a bully crying foul because they got hit back. Also, if distillation is what got K3 to where it is, why is it better in many areas? Also, big fucking kudos to k3 team for putting this together do fast givne how new fable access is, if it is true and had a meaningful impact. The real story there for me becomes one of extreme competence.
- paradox242 3mo agoThey stole all the data for the model in the first place so fair is fair.
- Biologist123 3mo agoAre they serious when so much of training was on illegally acquired data? Live by the sword, die by the sword.
- 627467 3mo agohttps://www.americanheritage.com/copywrong-short-history-literary-piracy https://www.americanheritage.com/copywrong-short-history-lit... Now, of course it's in the "creators" interest to prevent their biz models to breakdown due to piracy but it will be interesting to see how it will turn out. Its similar to any other form of piracy in internet age: you can't pretend to have global distribution and absolute global control at the same time.
- xingped 3mo agoGod I'm sick and tired of AI companies being so damn whiney all the time. Shut up already. No one cares.
- cvanelteren 3mo agoI mean it identifies as Claude in 1 in 5 cases when asked so pretty obvious to me.
- 8note 3mo agoif these can be distilled so easily, does the model actually need to be so big and trained with so much electricity?
- K0balt 3mo ago[flagged]
- rustcleaner 3mo agoNice! I hope to see continued liberation of these locked up SOTA models. Cloud is a virtual prison, since other people's policies on what they think a user should and should not do cannot be [easily] bypassed, if enforced remotely on a cloud. All digital natives should be skeptical of cloud-hosted services or software. Think like an intelligence agency or sovereign: how are you going to get screwed by the cloud? Your data is fully accessible by the provider, and they can surveil your activities. You probably can't pirate it, so you are a slave in their rentier model. You could be prevented from doing something you want to do, because the provider disagrees philosophically or economically with your desire. You could be stripped of your information/data by a ban due to their policy enforcement system triggering. One should live by the maxim: you don't have the thing if you don't possess the file or its processing. That goes for streaming, software, machine learning models, file storage, etc. But I digress; I am happy to see these paternalistic rentiers getting bit by these liberation/copying efforts, and human interests are served every time the digital and infrastructure locks are broken. I will always stand by the distillers!
- mycall 3mo agoDo you think China is doing this for the reasons you mentioned: liberation..efforts, human interests and freedom of policy?
- bigyabai 3mo agoYes, and they gave me the weights to prove it. Do you think Anthropic is in it for the love of the game? It looks like they're scared, to me.
- preisschild 3mo agoIts obvious they do it for soft power, but at least they actually deliver something useful to the world and not something locked down that only a single company owns.
- qmmmur 3mo agoDoesn’t feel nice, does it, Anthropic?
- voxelc4L 3mo agoWatching the tenants of intellectual gatekeeping bang up against the inevitable paradoxes of late stage capitalism - a spectacle worthy of jiffy pop.
- nessex 3mo agoPaying the list price Anthropic decided upon for outputs from an LLM, then using those outputs for your work? Or sourcing those outputs from others that paid for the service, and using those? If that's "distillation", it seems fine to me. And a lot like what most are doing with these same providers. Anthropic: training AI is "transformative", it's not copyright infringement if we don't re-transmit the copyrighted books we trained on Anthropic: training AI models on our outputs is stealing our secret sauce, outputs that could only be produced by us Anthropic: AI model outputs are unreliable and do not reflect Anthropic's views, we are not liable if they harm you (not exact quotes, they're "distilled") So Anthropic "distills" knowledge of others with reckless abandon, packages it up, sells it to you, claims it's your fault if anything bad happens but then also lobbies to treat you as a criminal if the outputs you paid for end up being transformed into any sort of competition for them. By you, or others that use the outputs you paid for.
- dizhn 3mo ago[flagged]
- woggy 3mo agoGreat engineering feat considering the small time window that Fable was available for.
- speedylight 3mo agoIt’s ok when they steal the entire internet and everything not the internet they can get their hands on but distillation is bad.
- xiaodai 3mo agoimagine if a company's MO is to PAY Claude to use their model and distill the results. That's fair use of their model!
- cloudie78 3mo agoOkay, and? We have information that all the big labs used copyrighted works for training. The big boy labs wanna cry now about distillation? Training an LLM is distillation too. Or are they crying because they don’t actually have a moat and they want Uncle Sam to step in somehow, lest the entire bubble pops and economy unwinds? Here’s a lesson from the automotive industry, people want econoboxes not formula 1 cars.
- benterix 3mo agoLOL. A thief accusing someone else of theft. Also, I love their choice of words. Like "distillation against", "stealing proprietary technology" - it's all aimed at certain people.
- pbgcp2026 3mo ago... but these MFs cripple our work by introducing stupid "guardrails".
- deleted 3mo ago[deleted]
- yanhangyhy 3mo agoeven they are using Distillation, its also a pretty scary ablity.. with such short time and result.
- hashstring 3mo agoWe also have evidence that Anthropic distilled all human info they could get their hands on for the development of all their models. Distillation should be fair game given the (current) game of LLM training. Yes, as a model creator you probably want to protect against it, but it does make you a hypocrite. > The developer OpenAI has said it would be impossible to create tools like its groundbreaking chatbot ChatGPT without access to copyrighted material, as pressure grows on artificial intelligence firms over the content used to train their products.
- kevincox 3mo agoIf anything this distillation is more ethical than the original as it is on machine-generated content rather than copyrighted human-labour products.
- MillionY 3mo agoAs far as I know, this is the easiest way to make that accusation true. https://pbs.twimg.com/media/HN56nDXaYAAUS4V?format=jpg&name=small https://pbs.twimg.com/media/HN56nDXaYAAUS4V?format=jpg&name=...
- poulpy123 3mo agoOh no they did with us what we did with the rest of the world, it's bad
- nektro 3mo ago[dead]
- rcbdev 3mo agoTruly, there is no honor among thieves.
- vrighter 3mo agoso?