anishacharya

3 ms·

1. 1.▲
   by anishacharya 8y ago

   Overcomplete nature of DNN is a common phenomenon and often advocated to convert saddle points into local minima --- understanding the actual landscape is difficult due to the inherent nonlinearity.... this indeed leads to an interesting re
2. 2.▲
   by anishacharya 8y ago

   Just to clarify a bunch of questions below: The idea is more like : start from a full embedding layer train the network, after the training stabilizes break the first layer into two using the SVD formulation in an online fashion and continu
3. 3.▲
   by anishacharya 8y ago

   The paper is here: https://arxiv.org/pdf/1811.00641.pdf
4. 4.▲
   by anishacharya 8y ago

   The idea ismore like : start from a full embedding layer train the network, after the training stabilizes break the first layer into two using the SVD formulation in an online fashion and continue training - after only a few more epochs you