4 ms·
I'm saying that it is unclear that without distillation this wouldn't still be happening. There is a massive narrative that no one but OpenAI, Anthropic, and Go
by colingauvin 2mo ago
I'm saying that it is unclear that without distillation this wouldn't still be happening. There is a massive narrative that no one but OpenAI, Anthropic, and Google can make a model without distilling. But there's basically no evidence of that.
- byzantinegene 2mo agoI think the (non-definitive) evidence is that the chinese models are always just slightly behind the publically released US models, and never ahead.
- sebzim4500 2mo agoI think it's more that distilling is enormously cheaper than training from scratch to achieve the same results