In my experience it also works fairly well when training e.g. on newspaper text. It's not really that surprising when you think about how it works. Similar wor

In my experience it also works fairly well when training e.g. on newspaper text. It's not really that surprising when you think about how it works. Similar words cluster together in vector space in some dimensions. E.g. 'Paris' and 'Berlin' will both have capital-ish contexts. However, they also are different in some ways, e.g. Paris will have France-ish contexts and Berlin German-ish contexts. The 2-dimensional PCA projection in figure 2 of one of Mikolov's papers [1] gives an intuition why substraction/addition generally works. [1] http://arxiv.org/pdf/1310.4546.pdf http://arxiv.org/pdf/1310.4546.pdf