4 ms·
The article's title is actually "New Method for Compressing Neural Networks Better Preserves Accuracy". Summary of a key result: "In one set of experiments, fo
by azhenley 8y ago
The article's title is actually "New Method for Compressing Neural Networks Better Preserves Accuracy".
Summary of a key result: "In one set of experiments, for instance, we showed that our system could shrink a neural network by 90 percent while reducing its accuracy by less than 1%. At the same compression rate, the best prior method reduced the accuracy by about 3.5%."
Link to the 2019 AAAI paper: https://arxiv.org/pdf/1811.00641.pdf https://arxiv.org/pdf/1811.00641.pdf
- trhway 8y agoit would be interesting if they also show whether those 90% are necessary or not to get that 1% back. I mean we kind of suspect that probably not, the interesting part here would be to show proof or at least some good foundation.
- irodov_rg 8y agoThis is shown in one of the plots, the less you compress the less you loose. While there is some analysis in the paper on how the computations reduce etc, the results are mostly emperical.
- sctb 8y agoWe've reverted the title from the submitted “Method shrinks NNs by 90% with sub-1% accuracy dropoff, vs. 3.5% for prior SOA”. Submitters: there's no need to try to pack the article into 80 characters! Titles and articles are two separate things.
- ralusek 8y agoDo they mean compressing the amount of neurons involved in computing, or do they mean compressing the outputted model?
- irodov_rg 8y agoIts mostly the lookup table which takes up the most space. This work is about breaking it into 2 layers and continuing to train to gain accuracy. The output model becomes 90% smaller compared to the original model.