6 ms·

You can logit distill (full token probabilities) or one hot distill (chat logs), or even align hidden states. All are distillation methods.