4 ms·
Is that really that surprising, though? Assuming you train it on encyclopedias and reference material, you'd get a ton of "X is the capital of Y" sentences.
by siegecraft 11y ago
Is that really that surprising, though? Assuming you train it on encyclopedias and reference material, you'd get a ton of "X is the capital of Y" sentences.
- danieldk 11y agoIn my experience it also works fairly well when training e.g. on newspaper text. It's not really that surprising when you think about how it works. Similar words cluster together in vector space in some dimensions. E.g. 'Paris' and 'Berlin' will both have capital-ish contexts. However, they also are different in some ways, e.g. Paris will have France-ish contexts and Berlin German-ish contexts. The 2-dimensional PCA projection in figure 2 of one of Mikolov's papers [1] gives an intuition why substraction/addition generally works. [1] http://arxiv.org/pdf/1310.4546.pdf http://arxiv.org/pdf/1310.4546.pdf