3 ms·
They've always been good enough for double digit less money. Always. Anyone thinking "Chinese models fake models built using dirty distillation scam" don't know
by numpad0 2mo ago
They've always been good enough for double digit less money. Always. Anyone thinking "Chinese models fake models built using dirty distillation scam" don't know what they're talking about.
Distillation is just forcing the model to use an exam prep workbook for training instead of generic publicly available textbooks. The models themselves has to be smart enough for that to work. It's the exact same thing as Asian tiger mom double schoolwork strategy, to paint a picture.
- budsniffer952 2mo agoTheir best models are getting more and more expensive, and still aren't SOTA. It's almost like there's an actual cost to developing these models, and the Chinese don't have magic dirt that allows them to do it at a fraction of the cost.
- fwip 2mo agoTo carry on this analogy - do test prep workbooks make you meaningfully more competent in general, or is it benchmaxing? (Versus studying textbooks for a similar time, of course.)