Having seen a bunch of people fail to make distillation work in the sense of training a better model than by training the smaller model from scratch, I don't th

Having seen a bunch of people fail to make distillation work in the sense of training a better model than by training the smaller model from scratch, I don't think it's really fool proof...