The paper you link is about word2vec, which has taken quite a while to understand. This paper shows word2vec is a particular factorization of the word-context m

The paper you link is about word2vec, which has taken quite a while to understand. This paper shows word2vec is a particular factorization of the word-context matrix. Neural Word Embedding as Implicit Matrix Factorization https://papers.nips.cc/paper/5477-neural-word-embedding-as-implicit-matrix-factorization.pdf https://papers.nips.cc/paper/5477-neural-word-embedding-as-i... Which is some of the story. I found this paper also interesting. Towards a Better Understanding of Predict and Count Models https://arxiv.org/abs/1511.02024 https://arxiv.org/abs/1511.02024