4 ms·

NN = Neural Network SOA = State of the Art The rest should be clear: it's a new method that shrinks the size of neural nets by 90% while only dropping the acc

NN = Neural Network SOA = State of the Art The rest should be clear: it's a new method that shrinks the size of neural nets by 90% while only dropping the accuracy <1%. The previous state of the art for doing the same task of shrinking neural nets dropped the accuracy by 3.5%. Compressing/shrinking neural nets is an important task because most edge devices (i.e. smart devices like thermostats, security cams, etc.) don't have the compute power to run large neural networks and need more compact ways to do the same tasks. Currently the solution is to use internet connectivity to query an API but that's not ideal for a variety of reasons.

visarga 8y ago
Not neural nets in general, just the word embedding layer. This layer tends to be large, like 1,000,000 x 300, so there are problems deploying it to production, especially in mobile environments. They can reduce the second dimension to something like 30 using SVD, replacing the large layer with the smaller one, and then retraining. Note that whole networks can also be compressed to similar levels with a variety of techniques, this one is just slightly better for a specific type of layer.