3 ms·
I took GP as asking for evidence that the reduced-cost Sol is actually a distillation of the previous-cost Sol. AFAIK, providers distilling or quantising models
by maleldil 1mo ago
I took GP as asking for evidence that the reduced-cost Sol is actually a distillation of the previous-cost Sol. AFAIK, providers distilling or quantising models and offering them as the same model have not been proven.
- sebzim4500 1mo agoI doubt he was claiming that. He's probably saying that the ability of Chinese companies to be able to distill frontier US models has put downwards pressure on the price of all models.
- colingauvin 1mo agoI'm saying that it is unclear that without distillation this wouldn't still be happening. There is a massive narrative that no one but OpenAI, Anthropic, and Google can make a model without distilling. But there's basically no evidence of that.
- byzantinegene 1mo agoI think the (non-definitive) evidence is that the chinese models are always just slightly behind the publically released US models, and never ahead.
- sebzim4500 1mo agoI think it's more that distilling is enormously cheaper than training from scratch to achieve the same results
- miki123211 1mo agoAlternatively, modern AI is good enough at optimizing its own kernels that it just keeps pushing costs down. Unlike the semi-decentralized inference provider community, OpenAI has both the talent and the compute to throw at the problem of making their models much more efficient to run. GPU kernel optimization is just the kind of well-bounded problem with clear success criteria that AI loves.