https://huggingface.co/docs/trl/main/en/gkd_trainer https://huggingface.co/docs/trl/main/en/gkd_trainer Take a set of prompts, run it through the large model a

https://huggingface.co/docs/trl/main/en/gkd_trainer https://huggingface.co/docs/trl/main/en/gkd_trainer Take a set of prompts, run it through the large model and small model, calculate KL divergence between all the logits, then update the small model to minimize the loss. This gives the smaller model a much higher density signal than SFT which is usually cross entropy against the single correct logit. Edit: note that the term is abused quite often and SFT on just the traces from the big model is also sometimes referred to as distillation (Eg the Deepseek smaller models). IMO that is incorrect.