11 ms·

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely tr

Does this matter? Distillation is not illegal by every definition of the word. There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. Another example is that it appears that the upper limit of what you can do is ultimately dependent on people working on the model, otherwise grok would be a LOT more competitive pre-cursor acquisition. And lastly, kimi architecture is vastly different than that of fable as it uses mechanisms developed by... kimi themselves. US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable. Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. edit: (moved this to bottom) The only argument they have here is that they use GB300 GPU's which for some reason should not be available to chinese citizens.

JKCalhoun 3mo ago
Legal, illegal… The word I would use is inevitable. It reminds me of the (PC) clones wars…
random_coder_nz 3mo ago
It doesn't matter. It is most likely a pretext for upcoming actions mostly likely executed via yet another retarded executive order. The guy that posted this looks like he's drowned himself in the MAGA Koolaid.
antisthenes 3mo ago
It also doesn't matter for a simpler, and much grander reason. All LLMs are trained on the corpus of humanity's knowledge, the legacy of everyone who's ever lived and our civilization as a whole. Anything that prevents or circumvents the accumulation or gatekeeping of this knowledge and puts it in the hands of more people (that are not AI company shareholders) is a good thing. Whether that is done by open sourcing the model weights, the training set, or by making the output better and cheaper, it is all fair game and is, as another poster mentioned, inevitable in the long run.
jaggederest 3mo ago
Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.
- dylan604 3mo ago
  This is why I don't give a shit that this is happening. It's actually kind of funny to me.
  - azinman2 3mo ago
    Unless you’re from mainland China, you should.
    - bilbo0s 3mo ago
      Why? Serious question. I'm from the US, and I think it's hilarious.
      - breppp 3mo ago
        One example is that a totalitarian government known for erasing historical record of its crimes will control an arbitrator of truth.
        dylan604 3mo ago
        so what? this has nothing to do with them taking from those that took before them. china censoring information they do not like is nothing new and seems irrelevant to this conversation
        breppp 3mo ago
        Fair enough you don't care, some people might want to know about the Uyghur happy camps, mass organ harvesting and such. In a world where information sources are only going to dwindle, it is not in anyone's interest to empower actors that will use these to manipulate perceptions
smeeth 3mo ago
Uh, what? > Distillation is not illegal by every definition of the word Note that Anthropic (and USG) alleges [0] not only that Kimi was distilled, but that they actively circumvented measures intended to stop distillation. There are multiple ways that's illegal, including: - Civil breach of contract. Anthropic's TOS explicitly say you can't do what Kimi is alleged to have done. - Economic espionage: 18 U.S.C. §1831 criminalizes obtaining a trade secret through theft, fraud, or deception while intending that it will benefit a foreign entity. - Trade-secret misappropriation: if Anthropic could argue industrial-scale querying reconstructed proprietary aspects of Fable (like by showing it produces similar outputs, as others have done) then it's illegal under 18 U.S.C. §1832. - California computer-access statute §502 bars knowingly accessing a computer system and, without permission, taking, copying, or using its data. - Computer Fraud and Abuse Act protects against the case where restrictions against an activity are circumvented (like Kimi is alleged to have done). > There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. A lack of prosecution does not make something legal. There is also the scale/commercialization thing, which isn't an issue with random tiny HF datasets/models. Remember: Kimi also sells K3 inference. > kimi architecture is vastly different than that of fable How do you know that? Do you work for Anthropic? Also, this has nothing to do with architecture, we are talking about data. > US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable. Cool. The difference is that one of those things is legal (because they chose to open-source) and one of those things is illegal theft of trade secrets (because it was stolen). > Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. 1) this has nothing to do with other labs, just Moonshot (and Z.ai, MiniMax, DS) 2) slandering or not it happens to be completely true, so, there's that [0] https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks https://www.anthropic.com/news/detecting-and-preventing-dist...
- skippyfish 3mo ago
  > Civil breach of contract. Anthropic's TOS explicitly say you can't do what Kimi is alleged to have done. Ah yes, I remember when Anthropic crawlers abided by the TOS of the websites they slurped up. All your other points are downstream from this, which makes them pretty tenuous. Labs don't think that ToS or other explicit wishes of content providers apply to them, but they expect everyone else to abide by theirs.
petilon 3mo ago
[dead]
linkregister 3mo ago
It matters because the closed-source frontier labs spend lots of money on human data (RLHF / RLAIF with human oversight). Moonshot is accused of circumventing these costs. Frontier labs add research costs into their inference pricing. If the market doesn't permit them to sustain sufficient pricing to have a positive cash flow, then their business prospects become weaker and they risk insolvency. Furthermore, other leveraged companies are at risk. The reason why the United States government is weighing in is because it's in the national interest of the US to have supremacy in "AI". Legality or lack thereof is one of many data points about whether a thing is noteworthy. Moonshot performing distillation is rational from their point of view. Reducing costs is in the interest of businesses. It's also rational for frontier labs and the US government to add obstacles to this process. As consumers this is probably a positive development.
- noja 3mo ago
  Isn’t that the same argument they are making for replacing human labour? Circumventing costs.
  - SubiculumCode 3mo ago
    There are many frames that one can place upon this issue. They do not contradict the other. There are moral framings (stole the internet so go eff yourselves, is one), but so is national security, and so is the doomer recursive self improvement risk, and then there is the framing purely on what this implies for future AI training. I mainly focus on the last. It will be hard for a frontier lab to justify spending the compute and data curation needed to advance AI further if that expenditure can be assimilated into your competitor's products within months/weeks. So reality will present labs with three choices: A. Cease spending massive amounts of money and compute improving those models. B. make those improved models more difficult to distill from, either through some regulatory regime, or some technical solution, which seems unlikely to me. C. making the best models available only to select partners and government. In all these potential outcomes, China, which lacks compute that U.S. labs enjoy, will likely stop seeing massive improvements in their AI models. Improvements to be sure, but right now they are enjoying gains from distillation AND their own model innovations, and these potential outcomes would largely stop one of those sources.
mattertoast 3mo ago
It does matter in that these LLM companies need to be run into the ground, and every embarrassing clod working for them run out of town. It's showing that 'distillation' is a viable way to reclaim all of what they stole and hoard, and with enough luck their debts will come due in time for them to feel it.
kevinqi 3mo ago
I agree distillation isn't illegal; I also think Moonshot/Kimi is very impressive. But the more interesting question is whether labs like Moonshot can be a real competitor to OpenAI/Anthropic. If you can only play catchup (however quickly you do that), then you're never going to be at the frontier - I think that's why distillation matters.
- mring33621 3mo ago
  People that think the Chinese are only able to copy western tech are in for a wakeup call. Actually, that has already happened in many domains, it's just that most western people (USA especially) won't admit it.
  - kevinqi 3mo ago
    my assertion isn't that china isn't able to surpass western AI. I think it may well happen. I've been to china many times and am well aware of how ahead they are in many technological/societal areas. at the same time, I don't buy the idea that distillation is unimportant in assessing what Chinese labs are capable of. If it wasn't, why did Kimi's release timing coincide so well with Fable's launch? and if Anthropic hadn't released Fable, would we have Kimi today? If the answer is no, then I think that's still a very important point to consider.
    - Daishiman 3mo ago
      > and if Anthropic hadn't released Fable, would we have Kimi today? If the answer is no, then I think that's still a very important point to consider. That works both ways, competition and performance spur new developments. You don't think the American labs are looking at Chinese research on how to reduce compute per token?
    - ericmay 3mo ago
      Spot-on and you're asking the right questions. And the other problem in these comments is that folks seem to think if China pulls ahead we can't just distill their models, provided that distillation is a key part of "this". If it's such a great strategy we'll just use it too if we want to. Boom roasted. For some reason folks seem to think that China can take action and then other countries can't also take action or respond to that action and it comes up again and again. China has hypersonic missiles! Pack it up boys time to go home. Nothing we can do. Dang shucks. China distilled American AI models, welp time to just close it all down and let's just write off those trillions of dollars and all the literal geniuses financing and building these things. Oh well China can just copy American models while we spend all the money! Ok we just stop developing models and we'll just copy their models. China will flood the market with their cheap products! Nope can't do anything like, oh, idk, not buy any of those products or just raise the prices on them in local markets. It's never-ending. I don't understand the lack of capacity to reason about other actors that takes commonly takes place. And that's just China, never mind other general issues.
mNovak 3mo ago
> The only argument they have here is that they use GB300 GPU's which for some reason should not be available to chinese citizens Note that Chinese companies are free to rent from GB300 clouds internationally. There are large datacenter hubs in Singapore and Malaysia serving chinese and other customers. Though there is also reported [1] significant smuggling of Nvidia chips into China as well. [1] https://epoch.ai/publications/chip-smuggling https://epoch.ai/publications/chip-smuggling
ryandvm 3mo ago
Boy I tell you, I am having an awful hard time summoning pity for the organizations that have themselves distilled all of humanity's knowledge into mysterious labor-market-masticating black boxes.
- matheusmoreira 3mo ago
  https://news.ycombinator.com/item?id=47567575 https://news.ycombinator.com/item?id=47567575 > protect our first-party products from abuse like bots, scraping Won't you think of the trillion dollar corporations?!
atleastoptimal 3mo ago
It matters because everyone imagines the inevitable "closing of the gap" between closed and open source, but the rate at which open source catches up with closed source seems to depend on being able to train on and distill the outputs of open source models. As long as performance of open source models is at least partially dependent on frontier-model outputs, then that gap will remain in place by definition. >Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. If the distillation is irrelevant to why it is competitive, why do they do it then? Obviously is helps improve their benchmarks/performance to some degree, otherwise they wouldn't need to do it.
- himata4113 3mo ago
  Never claimed that it is irrelevant. And kimi k3 is on the same level and sometimes outperforms fable 5 - that cannot be explained by distillation. The reason why gap is not closed is simply the fact that fable was trained months ago so in theory the frontier labs are still 1 (small) step ahead. Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competitively.
  - atleastoptimal 3mo ago
    Closed-source models have to deal with the current frontier being heavily regulated. Fable, at its old level, was "too good" to be released and they had to add an additional safety layer to sanitize the outputs. Lowering the quality of the models so they are safer and more steerable has been something all the closed-source models have been doing for a while, a requirement that many open source models don't need to deal with. If Kimi k3 really were above Fable 5 then there invariably the USG would have to consider their restrictions on model capabilities excessive, or one would have to admin closed source models are held to more restrictive safety standards than open source models. >Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competitively. How would you know this? How could you ascertain exactly how much performance is attributable to their unique engineering/research? If they really were so competitive they could surely make a model that isn't dependent on distilling Fable or other frontier models.
insanitybit 3mo ago
It is presumably against their ToS.
- applfanboysbgon 3mo ago
  And why, pray tell, would a cabinet member of the Trump administration be involving the US government in enforcing a private ToS?
  - thewebguyd 3mo ago
    Whoever Anthropic just bought when they doubled their political spending to $40 million just recently for "Pushing for AI safety"
nylonstrung 3mo ago
I wouldn't be surprised at all if US labs are also distilling Chinese models, except we'd never know since they can simply self-host them
- GuuD 3mo ago
  We do know, because we used to have some Claude models identifying as Deepseek when prompted in Chinese
xienze 3mo ago
> Does this matter? Distillation is not illegal by every definition of the word. Correct, but it at least helps answer the question of "how do they make such good models for a fraction of the price???" The answer is someone else spends the untold billions and Chinese labs do a little tweaking.
slibhb 3mo ago
Of course it matters. Regardless of whether distillation is legal, there is a difference between training a model with and without distillation. For one thing, the distilled model wouldn't exist without the model it distilled. Also, companies that use distillation may be competitive but seem unlikely to surpass the companies that are training these models from scratch.
- bilbo0s 3mo ago
  >but seem unlikely to surpass the companies that are training these models from scratch Then why is it a problem? Another serious question. Trying to get my head around what the root of the objection is here. There must be some fear, but if that fear is not a fear of being surpassed in the market, then what is the fear?
  - overfeed 3mo ago
    A fear of competition causing a failure to recoup the trillions invested in AI via sky-high margins, and starting a (short) chain-reaction that causes the bubble to pop (or fizzle). A lot of people have a lot riding on the AI bubble not popping.
  - slibhb 3mo ago
    I didn't say it was a problem, I said it "mattered" If I wanted to argue that it's a problem, I'd just say that companies investing billions in training frontier models should reap the rewards. And distillation is essentially theft.
Gajurgensen 3mo ago
It is incredibly important to whether the US can maintain its AI lead. If foreign competition is closing the gap only by distillation, then the frontier labs can focus on preventing distillation and maintain their lead that way. US dominance is also important for approaches to safety, especially political approaches. If the frontier models are all US-based, safety might be tackled via internal US policy. If other countries can independently train competitive models, international cooperation is required. Edit: It is also important for the business model. Companies won't be able to justify tremendous training costs if competitors can replicate their product much more cheaply via distillation.
- titanomachy 3mo ago
  Do Americans even believe that US policy is likely to steer development in a way that’s safe and beneficial for humanity? The rest of the world certainly doesn’t. The US currently seems to primarily use their superpower status to be the world’s number one shit disturber and geopolitical antagonist. I don’t think China’s necessarily any better, but I’d rather have the most powerful models be open rather than under the exclusive control of the US executive.
  - Gallows4574 3mo ago
    >Do Americans even believe that US policy is likely to steer development in a way that’s safe and beneficial for humanity? No, no we do not.
  - CuriouslyC 3mo ago
    China uses its power to make favorable deals and get people hooked on what it's slinging so it has captive customers. The US uses its power to bully and break rules that apply to everyone else for its own benefit. Kind of a big difference.
    - thesmtsolver2 3mo ago
      Not really. Go ask someone in Tibet/HongKong/Taiwan/Japan/India if China isn't breaking rules and bullying them. If China had US's powers/economy, it would be a much bigger bully. https://en.wikipedia.org/wiki/Wolf_warrior_diplomacy https://en.wikipedia.org/wiki/Wolf_warrior_diplomacy https://en.wikipedia.org/wiki/Chinese_police_overseas_service_stations https://en.wikipedia.org/wiki/Chinese_police_overseas_servic...
pgt 3mo ago
It matters because it means that lab could not train that model without distilling another frontier model, and their progress would slow once they get properly cut-off. If I funded that lab, I would want to know that.
XorNot 3mo ago
It certainly matters as familiar sounding words to their stock holders to please not drop the valuation. Because what they want them to think is "the AI factory has unique proprietary technology that cannot be replicated" What they don't want them to think is "it's relatively easy once you know the basics to bootstrap to near SOTA and so the commercial case for selling inference has an extremely short profitability horizon with little if any brand loyalty or lock in".
thewebguyd 3mo ago
> Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. They claim it because Anthropic are planning to push for protectionism. They just doubled their political spending to $40 million for the midterms to "push for AI regulation" Gee, I wonder what it is they are lobbying for. Certainly won't be OFAC sanctions right? ICTS import controls? US GOV, under lobbying pressure from Anthropic and OpenAI are going to go full protectionism and restrict Chinese models, I'd almost be willing to bet money on it. They can't really enforce for individuals, but they can definitely tell US based hpyerscalers they can't host them, make it illegal to host the weights, and government procurement restrictions.
GuB-42 3mo ago
> Distillation is not illegal by every definition of the word. I am waiting for a precedent on this one. In general, training on copyrighted material is legal, there is a lot of precedent there. But every now and then there is a case where the owner of the training material wins. I don't remember the details but I believe one of these instances was when one company trained its AI on the knowledge base of another company and turned it into a competing product. Fair use was denied because of that direct competition. Distilling a LLM to make a competing LLM looks kind of like this, or maybe not, I don't know. It would make sense for distillation to be legal in every way, LLMs are built on a broad interpretation of fair use, but sometimes, law is weird.
- unknownfuture 3mo ago
  > I am waiting for a precedent on this one. In general, training on copyrighted material is legal, there is a lot of precedent there. But every now and then there is a case where the owner of the training material wins. You're making a fundamental assumption: that model outputs are subject to copyright. In the US that's only the case if a human is part of the creative process: https://www.copyright.gov/newsnet/2025/1060.html https://www.copyright.gov/newsnet/2025/1060.html > It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements. This can include situations where a human-authored work is perceptible in an AI output, or a human makes creative arrangements or modifications of the output, but not the mere provision of prompts.
torginus 3mo ago
> The only argument they have here is that they use GB300 GPU's I don't follow events closely, but the US has constantly flipflopped on what sort of GPUs the Chinese are allowed to have, not in small part because much of the AI boom's valuation is based on demand for US-made hardware, for which the Chinese have inexhaustible and well-financed demand. So even this feels a bit hypocritical to me, but my understanding is that Chinese native AI hardware is getting good enough that labs dont feel a huge disadvantage by being forced to buy at home, even if they'd have preferred to buy US chips. Which is a situation that was manufactured by the constant thread of having their access to advanced GPUs revoked.

JKCalhoun 3mo ago

Legal, illegal… The word I would use is inevitable. It reminds me of the (PC) clones wars…

random_coder_nz 3mo ago

It doesn't matter. It is most likely a pretext for upcoming actions mostly likely executed via yet another retarded executive order. The guy that posted this looks like he's drowned himself in the MAGA Koolaid.

antisthenes 3mo ago

It also doesn't matter for a simpler, and much grander reason. All LLMs are trained on the corpus of humanity's knowledge, the legacy of everyone who's ever lived and our civilization as a whole. Anything that prevents or circumvents the accumulation or gatekeeping of this knowledge and puts it in the hands of more people (that are not AI company shareholders) is a good thing. Whether that is done by open sourcing the model weights, the training set, or by making the output better and cheaper, it is all fair game and is, as another poster mentioned, inevitable in the long run.

jaggederest 3mo ago

Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.

dylan604 3mo ago
This is why I don't give a shit that this is happening. It's actually kind of funny to me.
- azinman2 3mo ago
  Unless you’re from mainland China, you should.
  - bilbo0s 3mo ago
    Why? Serious question. I'm from the US, and I think it's hilarious.
    - breppp 3mo ago
      One example is that a totalitarian government known for erasing historical record of its crimes will control an arbitrator of truth.
      - dylan604 3mo ago
        so what? this has nothing to do with them taking from those that took before them. china censoring information they do not like is nothing new and seems irrelevant to this conversation
        breppp 3mo ago
        Fair enough you don't care, some people might want to know about the Uyghur happy camps, mass organ harvesting and such. In a world where information sources are only going to dwindle, it is not in anyone's interest to empower actors that will use these to manipulate perceptions

smeeth 3mo ago

Uh, what? > Distillation is not illegal by every definition of the word Note that Anthropic (and USG) alleges [0] not only that Kimi was distilled, but that they actively circumvented measures intended to stop distillation. There are multiple ways that's illegal, including: - Civil breach of contract. Anthropic's TOS explicitly say you can't do what Kimi is alleged to have done. - Economic espionage: 18 U.S.C. §1831 criminalizes obtaining a trade secret through theft, fraud, or deception while intending that it will benefit a foreign entity. - Trade-secret misappropriation: if Anthropic could argue industrial-scale querying reconstructed proprietary aspects of Fable (like by showing it produces similar outputs, as others have done) then it's illegal under 18 U.S.C. §1832. - California computer-access statute §502 bars knowingly accessing a computer system and, without permission, taking, copying, or using its data. - Computer Fraud and Abuse Act protects against the case where restrictions against an activity are circumvented (like Kimi is alleged to have done). > There are millions of samples available on huggingface and models explicitely trained on output produced by fable. There has been no action taken against them. A lack of prosecution does not make something legal. There is also the scale/commercialization thing, which isn't an issue with random tiny HF datasets/models. Remember: Kimi also sells K3 inference. > kimi architecture is vastly different than that of fable How do you know that? Do you work for Anthropic? Also, this has nothing to do with architecture, we are talking about data. > US AI labs are inspired by opensource advancements just as much as open source labs are inspired by traces from models such as fable. Cool. The difference is that one of those things is legal (because they chose to open-source) and one of those things is illegal theft of trade secrets (because it was stolen). > Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. 1) this has nothing to do with other labs, just Moonshot (and Z.ai, MiniMax, DS) 2) slandering or not it happens to be completely true, so, there's that [0] https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks https://www.anthropic.com/news/detecting-and-preventing-dist...

skippyfish 3mo ago
> Civil breach of contract. Anthropic's TOS explicitly say you can't do what Kimi is alleged to have done. Ah yes, I remember when Anthropic crawlers abided by the TOS of the websites they slurped up. All your other points are downstream from this, which makes them pretty tenuous. Labs don't think that ToS or other explicit wishes of content providers apply to them, but they expect everyone else to abide by theirs.

petilon 3mo ago

[dead]

linkregister 3mo ago

It matters because the closed-source frontier labs spend lots of money on human data (RLHF / RLAIF with human oversight). Moonshot is accused of circumventing these costs. Frontier labs add research costs into their inference pricing. If the market doesn't permit them to sustain sufficient pricing to have a positive cash flow, then their business prospects become weaker and they risk insolvency. Furthermore, other leveraged companies are at risk. The reason why the United States government is weighing in is because it's in the national interest of the US to have supremacy in "AI". Legality or lack thereof is one of many data points about whether a thing is noteworthy. Moonshot performing distillation is rational from their point of view. Reducing costs is in the interest of businesses. It's also rational for frontier labs and the US government to add obstacles to this process. As consumers this is probably a positive development.

noja 3mo ago
Isn’t that the same argument they are making for replacing human labour? Circumventing costs.
- SubiculumCode 3mo ago
  There are many frames that one can place upon this issue. They do not contradict the other. There are moral framings (stole the internet so go eff yourselves, is one), but so is national security, and so is the doomer recursive self improvement risk, and then there is the framing purely on what this implies for future AI training. I mainly focus on the last. It will be hard for a frontier lab to justify spending the compute and data curation needed to advance AI further if that expenditure can be assimilated into your competitor's products within months/weeks. So reality will present labs with three choices: A. Cease spending massive amounts of money and compute improving those models. B. make those improved models more difficult to distill from, either through some regulatory regime, or some technical solution, which seems unlikely to me. C. making the best models available only to select partners and government. In all these potential outcomes, China, which lacks compute that U.S. labs enjoy, will likely stop seeing massive improvements in their AI models. Improvements to be sure, but right now they are enjoying gains from distillation AND their own model innovations, and these potential outcomes would largely stop one of those sources.

mattertoast 3mo ago

It does matter in that these LLM companies need to be run into the ground, and every embarrassing clod working for them run out of town. It's showing that 'distillation' is a viable way to reclaim all of what they stole and hoard, and with enough luck their debts will come due in time for them to feel it.

kevinqi 3mo ago

I agree distillation isn't illegal; I also think Moonshot/Kimi is very impressive. But the more interesting question is whether labs like Moonshot can be a real competitor to OpenAI/Anthropic. If you can only play catchup (however quickly you do that), then you're never going to be at the frontier - I think that's why distillation matters.

mring33621 3mo ago
People that think the Chinese are only able to copy western tech are in for a wakeup call. Actually, that has already happened in many domains, it's just that most western people (USA especially) won't admit it.
- kevinqi 3mo ago
  my assertion isn't that china isn't able to surpass western AI. I think it may well happen. I've been to china many times and am well aware of how ahead they are in many technological/societal areas. at the same time, I don't buy the idea that distillation is unimportant in assessing what Chinese labs are capable of. If it wasn't, why did Kimi's release timing coincide so well with Fable's launch? and if Anthropic hadn't released Fable, would we have Kimi today? If the answer is no, then I think that's still a very important point to consider.
  - Daishiman 3mo ago
    > and if Anthropic hadn't released Fable, would we have Kimi today? If the answer is no, then I think that's still a very important point to consider. That works both ways, competition and performance spur new developments. You don't think the American labs are looking at Chinese research on how to reduce compute per token?
  - ericmay 3mo ago
    Spot-on and you're asking the right questions. And the other problem in these comments is that folks seem to think if China pulls ahead we can't just distill their models, provided that distillation is a key part of "this". If it's such a great strategy we'll just use it too if we want to. Boom roasted. For some reason folks seem to think that China can take action and then other countries can't also take action or respond to that action and it comes up again and again. China has hypersonic missiles! Pack it up boys time to go home. Nothing we can do. Dang shucks. China distilled American AI models, welp time to just close it all down and let's just write off those trillions of dollars and all the literal geniuses financing and building these things. Oh well China can just copy American models while we spend all the money! Ok we just stop developing models and we'll just copy their models. China will flood the market with their cheap products! Nope can't do anything like, oh, idk, not buy any of those products or just raise the prices on them in local markets. It's never-ending. I don't understand the lack of capacity to reason about other actors that takes commonly takes place. And that's just China, never mind other general issues.

mNovak 3mo ago

> The only argument they have here is that they use GB300 GPU's which for some reason should not be available to chinese citizens Note that Chinese companies are free to rent from GB300 clouds internationally. There are large datacenter hubs in Singapore and Malaysia serving chinese and other customers. Though there is also reported [1] significant smuggling of Nvidia chips into China as well. [1] https://epoch.ai/publications/chip-smuggling https://epoch.ai/publications/chip-smuggling

ryandvm 3mo ago

Boy I tell you, I am having an awful hard time summoning pity for the organizations that have themselves distilled all of humanity's knowledge into mysterious labor-market-masticating black boxes.

matheusmoreira 3mo ago
https://news.ycombinator.com/item?id=47567575 https://news.ycombinator.com/item?id=47567575 > protect our first-party products from abuse like bots, scraping Won't you think of the trillion dollar corporations?!

atleastoptimal 3mo ago

It matters because everyone imagines the inevitable "closing of the gap" between closed and open source, but the rate at which open source catches up with closed source seems to depend on being able to train on and distill the outputs of open source models. As long as performance of open source models is at least partially dependent on frontier-model outputs, then that gap will remain in place by definition. >Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. If the distillation is irrelevant to why it is competitive, why do they do it then? Obviously is helps improve their benchmarks/performance to some degree, otherwise they wouldn't need to do it.

himata4113 3mo ago
Never claimed that it is irrelevant. And kimi k3 is on the same level and sometimes outperforms fable 5 - that cannot be explained by distillation. The reason why gap is not closed is simply the fact that fable was trained months ago so in theory the frontier labs are still 1 (small) step ahead. Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competitively.
- atleastoptimal 3mo ago
  Closed-source models have to deal with the current frontier being heavily regulated. Fable, at its old level, was "too good" to be released and they had to add an additional safety layer to sanitize the outputs. Lowering the quality of the models so they are safer and more steerable has been something all the closed-source models have been doing for a while, a requirement that many open source models don't need to deal with. If Kimi k3 really were above Fable 5 then there invariably the USG would have to consider their restrictions on model capabilities excessive, or one would have to admin closed source models are held to more restrictive safety standards than open source models. >Although I will reiterate the fact that distillation is not the primary reason why these models are performing so competitively. How would you know this? How could you ascertain exactly how much performance is attributable to their unique engineering/research? If they really were so competitive they could surely make a model that isn't dependent on distilling Fable or other frontier models.

insanitybit 3mo ago

It is presumably against their ToS.

applfanboysbgon 3mo ago
And why, pray tell, would a cabinet member of the Trump administration be involving the US government in enforcing a private ToS?
- thewebguyd 3mo ago
  Whoever Anthropic just bought when they doubled their political spending to $40 million just recently for "Pushing for AI safety"

nylonstrung 3mo ago

I wouldn't be surprised at all if US labs are also distilling Chinese models, except we'd never know since they can simply self-host them

GuuD 3mo ago
We do know, because we used to have some Claude models identifying as Deepseek when prompted in Chinese

xienze 3mo ago

> Does this matter? Distillation is not illegal by every definition of the word. Correct, but it at least helps answer the question of "how do they make such good models for a fraction of the price???" The answer is someone else spends the untold billions and Chinese labs do a little tweaking.

slibhb 3mo ago

Of course it matters. Regardless of whether distillation is legal, there is a difference between training a model with and without distillation. For one thing, the distilled model wouldn't exist without the model it distilled. Also, companies that use distillation may be competitive but seem unlikely to surpass the companies that are training these models from scratch.

bilbo0s 3mo ago
>but seem unlikely to surpass the companies that are training these models from scratch Then why is it a problem? Another serious question. Trying to get my head around what the root of the objection is here. There must be some fear, but if that fear is not a fear of being surpassed in the market, then what is the fear?
- overfeed 3mo ago
  A fear of competition causing a failure to recoup the trillions invested in AI via sky-high margins, and starting a (short) chain-reaction that causes the bubble to pop (or fizzle). A lot of people have a lot riding on the AI bubble not popping.
- slibhb 3mo ago
  I didn't say it was a problem, I said it "mattered" If I wanted to argue that it's a problem, I'd just say that companies investing billions in training frontier models should reap the rewards. And distillation is essentially theft.

Gajurgensen 3mo ago

It is incredibly important to whether the US can maintain its AI lead. If foreign competition is closing the gap only by distillation, then the frontier labs can focus on preventing distillation and maintain their lead that way. US dominance is also important for approaches to safety, especially political approaches. If the frontier models are all US-based, safety might be tackled via internal US policy. If other countries can independently train competitive models, international cooperation is required. Edit: It is also important for the business model. Companies won't be able to justify tremendous training costs if competitors can replicate their product much more cheaply via distillation.

titanomachy 3mo ago
Do Americans even believe that US policy is likely to steer development in a way that’s safe and beneficial for humanity? The rest of the world certainly doesn’t. The US currently seems to primarily use their superpower status to be the world’s number one shit disturber and geopolitical antagonist. I don’t think China’s necessarily any better, but I’d rather have the most powerful models be open rather than under the exclusive control of the US executive.
- Gallows4574 3mo ago
  >Do Americans even believe that US policy is likely to steer development in a way that’s safe and beneficial for humanity? No, no we do not.
- CuriouslyC 3mo ago
  China uses its power to make favorable deals and get people hooked on what it's slinging so it has captive customers. The US uses its power to bully and break rules that apply to everyone else for its own benefit. Kind of a big difference.
  - thesmtsolver2 3mo ago
    Not really. Go ask someone in Tibet/HongKong/Taiwan/Japan/India if China isn't breaking rules and bullying them. If China had US's powers/economy, it would be a much bigger bully. https://en.wikipedia.org/wiki/Wolf_warrior_diplomacy https://en.wikipedia.org/wiki/Wolf_warrior_diplomacy https://en.wikipedia.org/wiki/Chinese_police_overseas_service_stations https://en.wikipedia.org/wiki/Chinese_police_overseas_servic...

pgt 3mo ago

It matters because it means that lab could not train that model without distilling another frontier model, and their progress would slow once they get properly cut-off. If I funded that lab, I would want to know that.

XorNot 3mo ago

It certainly matters as familiar sounding words to their stock holders to please not drop the valuation. Because what they want them to think is "the AI factory has unique proprietary technology that cannot be replicated" What they don't want them to think is "it's relatively easy once you know the basics to bootstrap to near SOTA and so the commercial case for selling inference has an extremely short profitability horizon with little if any brand loyalty or lock in".

thewebguyd 3mo ago

> Claiming in any shape or form that fable disillation is one of the primary reasons why kimi k3 is so competitive is slandering the work of other labs that cooperatively push the open-source models forward. They claim it because Anthropic are planning to push for protectionism. They just doubled their political spending to $40 million for the midterms to "push for AI regulation" Gee, I wonder what it is they are lobbying for. Certainly won't be OFAC sanctions right? ICTS import controls? US GOV, under lobbying pressure from Anthropic and OpenAI are going to go full protectionism and restrict Chinese models, I'd almost be willing to bet money on it. They can't really enforce for individuals, but they can definitely tell US based hpyerscalers they can't host them, make it illegal to host the weights, and government procurement restrictions.

GuB-42 3mo ago

> Distillation is not illegal by every definition of the word. I am waiting for a precedent on this one. In general, training on copyrighted material is legal, there is a lot of precedent there. But every now and then there is a case where the owner of the training material wins. I don't remember the details but I believe one of these instances was when one company trained its AI on the knowledge base of another company and turned it into a competing product. Fair use was denied because of that direct competition. Distilling a LLM to make a competing LLM looks kind of like this, or maybe not, I don't know. It would make sense for distillation to be legal in every way, LLMs are built on a broad interpretation of fair use, but sometimes, law is weird.

unknownfuture 3mo ago
> I am waiting for a precedent on this one. In general, training on copyrighted material is legal, there is a lot of precedent there. But every now and then there is a case where the owner of the training material wins. You're making a fundamental assumption: that model outputs are subject to copyright. In the US that's only the case if a human is part of the creative process: https://www.copyright.gov/newsnet/2025/1060.html https://www.copyright.gov/newsnet/2025/1060.html > It concludes that the outputs of generative AI can be protected by copyright only where a human author has determined sufficient expressive elements. This can include situations where a human-authored work is perceptible in an AI output, or a human makes creative arrangements or modifications of the output, but not the mere provision of prompts.

torginus 3mo ago

> The only argument they have here is that they use GB300 GPU's I don't follow events closely, but the US has constantly flipflopped on what sort of GPUs the Chinese are allowed to have, not in small part because much of the AI boom's valuation is based on demand for US-made hardware, for which the Chinese have inexhaustible and well-financed demand. So even this feels a bit hypocritical to me, but my understanding is that Chinese native AI hardware is getting good enough that labs dont feel a huge disadvantage by being forced to buy at home, even if they'd have preferred to buy US chips. Which is a situation that was manufactured by the constant thread of having their access to advanced GPUs revoked.