4 ms·
From LSA/SVD you get a V x K matrix as well - that's exactly what the factorisation is doing. The following two papers also go into detail about the mathematic
by eginhard 10y ago
From LSA/SVD you get a V x K matrix as well - that's exactly what the factorisation is doing.
The following two papers also go into detail about the mathematical similarities between LSA and neural embeddings and achieving similar performance with both:
Levy, O. and Goldberg, Y. (2014). Neural word embedding as implicit matrix
factorization. https://www.cs.bgu.ac.il/~yoavg/publications/nips2014pmi.pdf https://www.cs.bgu.ac.il/~yoavg/publications/nips2014pmi.pdf
Levy, O., Goldberg, Y., and Dagan, I. (2015). Improving distributional similarity
with lessons learned from word embeddings. http://www.anthology.aclweb.org/Q/Q15/Q15-1016.pdf http://www.anthology.aclweb.org/Q/Q15/Q15-1016.pdf
- twelfthnight 10y agoTrue, you could use the right matrix of the SVD on the term document matrix. As far as I know, the embeddings won't have the same interpretation as the left matrix from SVD on the word-context matrix. (By LSA I mean SVD on the term document matrix). Those are excellent papers, by the way.