SVD has been used for dimensionality reduction of co-occurrence matrices for ages [1], but the resulting word embeddings aren't as performant as those of word2v

SVD has been used for dimensionality reduction of co-occurrence matrices for ages [1], but the resulting word embeddings aren't as performant as those of word2vec/etc. The same is probably true of using PCA. Word2vec's popularity is the result of people valuing performance (i.e. accuracy) more than interpretability. [1] https://dl.acm.org/citation.cfm?id=148132 https://dl.acm.org/citation.cfm?id=148132