4 ms·
All of this seems to be using word embeddings, but most languages don't have all that many words. You're effectively trying to train with just a few thousand
by londons_explore 8y ago
All of this seems to be using word embeddings, but most languages don't have all that many words. You're effectively trying to train with just a few thousand data points, and will quickly overfit.
Wouldn't the method work better with n-gram embeddings, where n=3 or 4?
- guismay 8y agoThe method is straightforward to extend to n-grams, this is actually what they do (see table 1 in https://arxiv.org/abs/1804.07755 https://arxiv.org/abs/1804.07755). And since you are simply learning a rotation matrix, there is no risk of overfitting.