Data is doing more of the work than it used to. Every source in our mixture is a curated artifact built with large models Training a model this small on them

Data is doing more of the work than it used to. Every source in our mixture is a curated artifact built with large models Training a model this small on them is distillation When models of this size were last studied seriously such corpora did not exist