5 ms·
The most effective "shrinking" of ML models that I've seen (very limited experience, YMMV) is through "pruning". Searching for "arxiv pruning" is an excellent s
by web007 8y ago
The most effective "shrinking" of ML models that I've seen (very limited experience, YMMV) is through "pruning". Searching for "arxiv pruning" is an excellent starting point, and a couple of those papers include metrics for accuracy vs size and the tradeoffs therein.
- mlthoughts2018 8y agoI came to the comments to say the same thing. Quantization and hashing tricks for embeddings are cool and all, but not really important for model compression. Rather, training companion models to prune away whole subnetworks of weight and layer combinations can allow you to remove tens of thousands of parameters from the model entirely— not wasting space on their quantized weights when they end up not being a contributing pathway to predictions.