4 ms·
Do you not consider SVD/PCA to be "embeddings"? Latent Semantic Indexing long precedes word2vec (I believe the initial paper was 1988) but also attempted to de
by IKantRead 3y ago
Do you not consider SVD/PCA to be "embeddings"?
Latent Semantic Indexing long precedes word2vec (I believe the initial paper was 1988) but also attempted to derived semantic vectors (and was arguably successful) using SVD on the term-frequency matrix representing a corpus.
I would certainly consider this an example of using "embeddings" that was quite heavily used in practice prior to the explosion of deep learning techniques.
- simonw 3y agoSounds like you know a great deal more about the history of this field than I do!
- IKantRead 3y agoThe classic Information Retrieval, though out of date in many ways, does a great job covering the "old school" NLP approaches for IR and has an excellent section on Latent Semantic Indexing which you might find enjoyable: https://nlp.stanford.edu/IR-book/html/htmledition/latent-semantic-indexing-1.html https://nlp.stanford.edu/IR-book/html/htmledition/latent-sem...