3 ms·
If we accept the premise that the top Chinese labs are simply distilling and can't compete otherwise: why don't US labs simply do the same thing? Distill their
by nvme0n1p1 1mo ago
If we accept the premise that the top Chinese labs are simply distilling and can't compete otherwise: why don't US labs simply do the same thing? Distill their own models and slash their costs by 99% while keeping the same quality output. It should be a piece of cake if even the open labs can figure it out, after all.
One way or another they're getting the same results as proprietary labs, with a fraction of the hardware for a fraction of the cost. OpenAI can't keep raising funding rounds of $100billions to subsidize their compute costs and get results by brute forcing parameter count. And if they're having trouble keeping up with Chinese labs' efficiency, maybe they should stop worrying about distilling and instead hire some of the smart people responsible.
- novok 1mo agoBecause distillation-only is a quick performance shortcut that only lets you get to the level of the thing your distilling for the most part or a little bit worse and does not allow you to actually progress past it. It's like only being able to make VHS copies of videos, and maybe do some basic video editing without being able to actually go out with cameras and make new movies. To actually have something competitive and improved within the next 3 months and not be perpetually behind, you need your own independent model creation process. So to extend the metaphor, a complete movie studio with cameras, actors, staff, sets, budgets, etc. It's the right strategic move to do when you are GPU constrained, which the Chinese labs are, but it won't let you get past it. A bunch of pedantic people will come out of the wood work citing a bunch of things saying that is not the case because of some detailed mechanics of how model training works and they will get fixated on some of the words I used, but zoom out to the level of what an AI lab is able to produce and this becomes evident.
- nvme0n1p1 1mo agoThat doesn't really answer the question. What they do internally for training the next model is a separate issue. I'm talking about the models they offer publicly. Per the article, companies are dropping OpenAI+Anthropic (partly) because of costs. If distilling is so simple and easy, why doesn't OpenAI take this "quick shortcut" and serve a self-distilled model externally, so they can charge reasonable prices and stop bleeding customers? Surely they can at least match the Chinese labs' efficiency, right? Wouldn't more customers and less opex look good for the IPO?
- linkregister 1mo agoAnthropic, OpenAI, GDM, and Meta spend more on training than other labs by an order of magnitude. If they felt safe reducing this spend they would. These labs fear getting outcompeted.
- nvme0n1p1 1mo agoAgain, I am talking about inference, not training. Please read.
- linkregister 1mo agoHow do you think they fund training? This is just as asinine as insisting that drug manufacturers only price medications based on production costs.
- nvme0n1p1 1mo agoLess opex = more profits. They can use that money to fund training. I don't see how that could possibly be a bad thing.
- linkregister 1mo agoI indeed failed to understand your point. Isn't that what they already do with Claude Haiku, GPT-5.6-Terra, etc?
- nvme0n1p1 1mo agoThat's the goal, but those smaller models don't match the price:performance of leading open-weight models, which is why companies are switching away (as explained in TFA). The open weight labs figured out some secret sauce that (so far) big name labs are unable to replicate, so instead of competing, they're going on the defensive with claims of distillation attacks.
- 27d ago