3 ms·
It's a whole subfield of machine learning called model distillation or model compression that concerns with shrinking large neural nets, or ensembles of them, t
by visarga 10y ago
It's a whole subfield of machine learning called model distillation or model compression that concerns with shrinking large neural nets, or ensembles of them, to fit on small devices like phones and tablets. By reducing the space needed by 10x, they only lose 1-2% in accuracy.
But what I like about it is that neural networks can be transferred between different architectures, and the whole process of training a neural net can be sped up by starting from a previous result.