3 ms·
Not neural nets in general, just the word embedding layer. This layer tends to be large, like 1,000,000 x 300, so there are problems deploying it to production,
by visarga 8y ago
Not neural nets in general, just the word embedding layer. This layer tends to be large, like 1,000,000 x 300, so there are problems deploying it to production, especially in mobile environments. They can reduce the second dimension to something like 30 using SVD, replacing the large layer with the smaller one, and then retraining.
Note that whole networks can also be compressed to similar levels with a variety of techniques, this one is just slightly better for a specific type of layer.