I think that the bigger conversation to be had here is about the environmental damage - if by using distillation we can really train new models at 1% of the cos

I think that the bigger conversation to be had here is about the environmental damage - if by using distillation we can really train new models at 1% of the cost in energy, it is ethically imperative that we do this.