Well, the thing that word2vec can (and should) be understood in terms of word coincidences (and pointwise mutual information) is important, but hardly new. I t

Well, the thing that word2vec can (and should) be understood in terms of word coincidences (and pointwise mutual information) is important, but hardly new. I tried to explain it here: http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html http://p.migdal.pl/2017/01/06/king-man-woman-queen-why.html. There is a temptation to use just the word pair counts, skipping SVD, but it won't yield in the best results. Creating vectors not only compresses data, but also finds general patterns. This compression is super important for less frequent words (otherwise we get a lot of overfitting). See "Why do low dimensional embeddings work better than high-dimensional ones?" from http://www.offconvex.org/2016/02/14/word-embeddings-2/ http://www.offconvex.org/2016/02/14/word-embeddings-2/.