15 ms·

So they don’t even try to explain why splitting it into two steps works better? I wonder if anyone tried splitting knowledge distillation optimization like that

by p1esk 5y ago

So they don’t even try to explain why splitting it into two steps works better? I wonder if anyone tried splitting knowledge distillation optimization like that.