5 ms·

assume you are a "second class lab" and you are in fact making progress by distilling the results of the frontier labs' efforts. what is the end game for this

assume you are a "second class lab" and you are in fact making progress by distilling the results of the frontier labs' efforts. what is the end game for this strategy? if the frontier labs shut down, or stop releasing to the public, and there's noting left to distill, how will you progress?

traverseda 3mo ago
In public with budgets that don't risk destroying the American economy presumably. Yes it may be slower.
- milkshakes 3mo ago
  > with budgets and what will fund these budgets exactly? inference is cheap, distillation is cheap, training is what's expensive.
  - Jtarii 3mo ago
    Presumably the US military / NSA.
    - milkshakes 3mo ago
      the USG/NSA will fund chinese labs? to what end?
      - Jtarii 3mo ago
        I was more thinking they would be funding US labs.
        milkshakes 3mo ago
        the question was: what is the endgame for the stated "second class labs" strategy of distilling their frontier competitors then undercutting them on price?
        nickysielicki 3mo ago
        Making lots of money?
        traverseda 3mo ago
        Yes yes, we all understand the game-theoretic race-to-the-bottom you're describing here. Somehow despite linux being FOSS it still powers most of the important computing in the world. Can you explain how that works despite it being free? Once you understand that case I think you'll understand the game-theory behind how large projects can exist in the absence of traditional IP protection.
        milkshakes 3mo ago
        the obvious difference is the massive scale of data and compute required to develop and evolve these models, and the costs they impose on those building them.
        traverseda 3mo ago
        Smaller budgets, slower improvement, less risk. They're not entitled to profits if that business model isn't sustainable. They're not entitled to a change in IP laws to protect their business model. They're not entitled to growing that fast.
        milkshakes 3mo ago
        who are you talking about? again, my question is concerned with the "second class labs" and the sustainability of the distillation-as-a-service model.
        Jtarii 3mo ago
        They will make a bunch of money then maybe go out of business eventually when the economics shift? Do they need an endgame? Some people are just happy to follow the money.
  - traverseda 3mo ago
    Same people who fund linux kernel development. A coalition of companies that find it useful.
InsideOutSanta 3mo ago
This line of thinking makes no sense because it assumes that labs that distill from frontier models are doing nothing else. It's the classic "the Chinese can only copy" mentality, and it's going to end poorly for American companies. I'm pretty sure that all labs are distilling each others' LLMs, maybe apart from Anthropic and OpenAI. It would be stupid not to do it, because it's cheap and effective. But that's not the only thing they're doing. If you think K3 and GLM-5.2 got this good only from distilling frontier models, you're not paying attention to Chinese labs' publications.
- milkshakes 3mo ago
  i never assumed that, and i do keep up with the publications. i'm also not saying it's a dumb thing to do! what i am saying is that empirically, it appears that distillation of a more advanced model is a required first step for them to train a borderline competitive, cheaper model. in effect, their training is subsidized by the frontier labs. if this were not the case, then we would be observing chinese models that far surpass frontier models in capabilities, rather than "almost as good, but much cheaper", and we would be having a very different conversation. what happens to these efforts when the subsidy is cut off?
  - InsideOutSanta 3mo ago
    > empirically, it appears that distillation of a more advanced model is a required first step I see no evidence for that. > if this were not the case, then we would be observing chinese models that far surpass frontier models It's pretty clear that the primary reason for the difference is budget and compute availability. Chinese labs have at least an order of magnitude less money than Anthropic and OpenAI. > what happens to these efforts when the subsidy is cut off? They will continue making progress as they do now, minus the benefits of distillation.
    - milkshakes 3mo ago
      https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks https://www.anthropic.com/news/detecting-and-preventing-dist... Moonshot AI Scale: Over 3.4 million exchanges The operation targeted: Agentic reasoning and tool use Coding and data analysis Computer-use agent development Computer vision Moonshot (Kimi models) employed hundreds of fraudulent accounts spanning multiple access pathways. Varied account types made the campaign harder to detect as a coordinated operation. We attributed the campaign through request metadata, which matched the public profiles of senior Moonshot staff. In a later phase, Moonshot used a more targeted approach, attempting to extract and reconstruct Claude’s reasoning traces.
IncreasePosts 3mo ago
There doesn't need to be progress at this point. Some models even from 1 or more years ago are useful for some purposes
tristanj 3mo ago
Distillation from a teacher model solves the self-start problem, that is, building a model to the point where it reason coherently. Without distillation, solving self-start is incredibly difficult since it requires millions of high quality training samples. Creating that kind of dataset takes an enormous amount of effort. Once a model becomes competent enough to perform complex reasoning, a teacher model is no longer necessary. The model can now reason about its own behavior and build a better version of itself through recursive self-improvement (RSI). Kimi K3 is capable of RSI.
- anon373839 3mo ago
  > Creating that kind of dataset takes an enormous amount of effort Can you imagine the amount of effort it takes to write 15 trillion tokens worth of art, literature, source code, textbooks, scientific papers, news articles, etc.? No wonder Anthropic just scooped it up and took it for free! Yet you don’t seem bothered by this. I wonder why.

traverseda 3mo ago

In public with budgets that don't risk destroying the American economy presumably. Yes it may be slower.

milkshakes 3mo ago
> with budgets and what will fund these budgets exactly? inference is cheap, distillation is cheap, training is what's expensive.
- Jtarii 3mo ago
  Presumably the US military / NSA.
  - milkshakes 3mo ago
    the USG/NSA will fund chinese labs? to what end?
    - Jtarii 3mo ago
      I was more thinking they would be funding US labs.
      - milkshakes 3mo ago
        the question was: what is the endgame for the stated "second class labs" strategy of distilling their frontier competitors then undercutting them on price?
        nickysielicki 3mo ago
        Making lots of money?
        traverseda 3mo ago
        Yes yes, we all understand the game-theoretic race-to-the-bottom you're describing here. Somehow despite linux being FOSS it still powers most of the important computing in the world. Can you explain how that works despite it being free? Once you understand that case I think you'll understand the game-theory behind how large projects can exist in the absence of traditional IP protection.
        milkshakes 3mo ago
        the obvious difference is the massive scale of data and compute required to develop and evolve these models, and the costs they impose on those building them.
        traverseda 3mo ago
        Smaller budgets, slower improvement, less risk. They're not entitled to profits if that business model isn't sustainable. They're not entitled to a change in IP laws to protect their business model. They're not entitled to growing that fast.
        milkshakes 3mo ago
        who are you talking about? again, my question is concerned with the "second class labs" and the sustainability of the distillation-as-a-service model.
        Jtarii 3mo ago
        They will make a bunch of money then maybe go out of business eventually when the economics shift? Do they need an endgame? Some people are just happy to follow the money.
- traverseda 3mo ago
  Same people who fund linux kernel development. A coalition of companies that find it useful.

InsideOutSanta 3mo ago

This line of thinking makes no sense because it assumes that labs that distill from frontier models are doing nothing else. It's the classic "the Chinese can only copy" mentality, and it's going to end poorly for American companies. I'm pretty sure that all labs are distilling each others' LLMs, maybe apart from Anthropic and OpenAI. It would be stupid not to do it, because it's cheap and effective. But that's not the only thing they're doing. If you think K3 and GLM-5.2 got this good only from distilling frontier models, you're not paying attention to Chinese labs' publications.

milkshakes 3mo ago
i never assumed that, and i do keep up with the publications. i'm also not saying it's a dumb thing to do! what i am saying is that empirically, it appears that distillation of a more advanced model is a required first step for them to train a borderline competitive, cheaper model. in effect, their training is subsidized by the frontier labs. if this were not the case, then we would be observing chinese models that far surpass frontier models in capabilities, rather than "almost as good, but much cheaper", and we would be having a very different conversation. what happens to these efforts when the subsidy is cut off?
- InsideOutSanta 3mo ago
  > empirically, it appears that distillation of a more advanced model is a required first step I see no evidence for that. > if this were not the case, then we would be observing chinese models that far surpass frontier models It's pretty clear that the primary reason for the difference is budget and compute availability. Chinese labs have at least an order of magnitude less money than Anthropic and OpenAI. > what happens to these efforts when the subsidy is cut off? They will continue making progress as they do now, minus the benefits of distillation.
  - milkshakes 3mo ago
    https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks https://www.anthropic.com/news/detecting-and-preventing-dist... Moonshot AI Scale: Over 3.4 million exchanges The operation targeted: Agentic reasoning and tool use Coding and data analysis Computer-use agent development Computer vision Moonshot (Kimi models) employed hundreds of fraudulent accounts spanning multiple access pathways. Varied account types made the campaign harder to detect as a coordinated operation. We attributed the campaign through request metadata, which matched the public profiles of senior Moonshot staff. In a later phase, Moonshot used a more targeted approach, attempting to extract and reconstruct Claude’s reasoning traces.

IncreasePosts 3mo ago

There doesn't need to be progress at this point. Some models even from 1 or more years ago are useful for some purposes

tristanj 3mo ago

Distillation from a teacher model solves the self-start problem, that is, building a model to the point where it reason coherently. Without distillation, solving self-start is incredibly difficult since it requires millions of high quality training samples. Creating that kind of dataset takes an enormous amount of effort. Once a model becomes competent enough to perform complex reasoning, a teacher model is no longer necessary. The model can now reason about its own behavior and build a better version of itself through recursive self-improvement (RSI). Kimi K3 is capable of RSI.

anon373839 3mo ago
> Creating that kind of dataset takes an enormous amount of effort Can you imagine the amount of effort it takes to write 15 trillion tokens worth of art, literature, source code, textbooks, scientific papers, news articles, etc.? No wonder Anthropic just scooped it up and took it for free! Yet you don’t seem bothered by this. I wonder why.