3 ms·

meanwhile in the real world https://www.science.org/content/article/china-tops-world-artificial-intelligence-publications-database-analysis-reveals https://www.

meanwhile in the real world https://www.science.org/content/article/china-tops-world-artificial-intelligence-publications-database-analysis-reveals https://www.science.org/content/article/china-tops-world-art...

felixgallo 6d ago
both things can be true. China's gearing up, but they also try to make progress by aggressively distilling Anthropic and OpenAI models, and this is currently where most of their progress comes from.
- yogthos 6d ago
  People really need to stop parroting this line uncritically. The process takes time because even when you're distilling answers, you still need to actually do reinforcement training on the model. And given that Fable and GPT 5.6 just came out there simply hasn't been much time to do that. However, models like Kimi also do better than Fable or GPT on a lot of tasks, which means it's not just distillation but also difference in architecture. You can watch this talk from Kimi founder to see how Kimi was actually trained and why it performs well. https://www.youtube.com/watch?v=5CkCW1P-g88 https://www.youtube.com/watch?v=5CkCW1P-g88 It's also absolutely hilarious that people think only Chinese companies use distillation, as if Anthropic or OpenAI are above that or something. Not to mention that they basically ignored copyrights on all the data the siphoned and are now crying that people aren't respecting their terms of use. Chinese labs have come up with a bunch of genuine innovations: GRPO, auxiliary loss free MoE load balancing, MLA, muon optimizer, and a bunch of other ones. The Deepseek papers are really well written, this isn’t just sneaking a peek at a peer. Anybody who thinks China is simply distilling glorious American models is not engaging with reality.
  - Zaraif13 6d ago
    This is actually brilliantly put.
  - 4d4m 5d ago
    Every model provider distills each other even if for benchmarking. Agree with all your points.
    - yogthos 5d ago
      For sure, everybody distills when they can, it would be stupid not to. I'm just pointing out that Chinese companies clearly do their own research and innovation just like American companies do. It's not that they just wait for American models to drop and then distill them.

felixgallo 6d ago

both things can be true. China's gearing up, but they also try to make progress by aggressively distilling Anthropic and OpenAI models, and this is currently where most of their progress comes from.

yogthos 6d ago
People really need to stop parroting this line uncritically. The process takes time because even when you're distilling answers, you still need to actually do reinforcement training on the model. And given that Fable and GPT 5.6 just came out there simply hasn't been much time to do that. However, models like Kimi also do better than Fable or GPT on a lot of tasks, which means it's not just distillation but also difference in architecture. You can watch this talk from Kimi founder to see how Kimi was actually trained and why it performs well. https://www.youtube.com/watch?v=5CkCW1P-g88 https://www.youtube.com/watch?v=5CkCW1P-g88 It's also absolutely hilarious that people think only Chinese companies use distillation, as if Anthropic or OpenAI are above that or something. Not to mention that they basically ignored copyrights on all the data the siphoned and are now crying that people aren't respecting their terms of use. Chinese labs have come up with a bunch of genuine innovations: GRPO, auxiliary loss free MoE load balancing, MLA, muon optimizer, and a bunch of other ones. The Deepseek papers are really well written, this isn’t just sneaking a peek at a peer. Anybody who thinks China is simply distilling glorious American models is not engaging with reality.
- Zaraif13 6d ago
  This is actually brilliantly put.
- 4d4m 5d ago
  Every model provider distills each other even if for benchmarking. Agree with all your points.
  - yogthos 5d ago
    For sure, everybody distills when they can, it would be stupid not to. I'm just pointing out that Chinese companies clearly do their own research and innovation just like American companies do. It's not that they just wait for American models to drop and then distill them.