3 ms·
It feels weird how word embedding models have come to refer to both the underlying model, as well as the implementation. word2vec is the implementation of two m
by herrkanin 10y ago
It feels weird how word embedding models have come to refer to both the underlying model, as well as the implementation. word2vec is the implementation of two models: the continuous bag-of-word and the skipgram models by Mikolov, while LexVec implements a version of the PPMI weighted count matrix as referenced in the README file. But the papers also discuss implementation details of LexVec that has no bearing on the final accuracy. I feel like we should make more effort to keep the models and reference implementations separate.
- tdj 10y agoAren't skip-grams equivalent to NMF of the PPMI matrix? https://papers.nips.cc/paper/5477-neural-word-embedding-as-implicit-matrix-factorization.pdf https://papers.nips.cc/paper/5477-neural-word-embedding-as-i....