Alan Turing actually talked about this way back in Computing Machinery and Intelligence and it’s already used for distillation of information from large models

Alan Turing actually talked about this way back in Computing Machinery and Intelligence and it’s already used for distillation of information from large models to more compact ones in the so called “student/teacher” training regime.