3 ms·
Fine-tuning is done against a dataset. Distillation is done against a model.
by nerdsniper 2mo ago
Fine-tuning is done against a dataset. Distillation is done against a model.
- verdverm 2mo agoDistillation is used to build part of a data set for fine-tuning (loosely interpreted). Advanced model traces are useless if you don't have a base model that is good enough to be improved by them.