4 ms·
> Words with similar vectors have similar meanings. Can you explain WHY this is the case with word2vec? I have come across a paper which says "we don't know re
by denzil_correa 10y ago
> Words with similar vectors have similar meanings.
Can you explain WHY this is the case with word2vec? I have come across a paper which says "we don't know really". Is this true?
https://arxiv.org/abs/1402.3722 https://arxiv.org/abs/1402.3722
- rspeer 10y agoI think there have been some attempts to explain, none of them fully satisfying. But there have also been systems that create vectors with that property more directly instead of as a side-effect of a neural net. Examples include Stanford's GloVe, or Omer Levy's Hyperwords, which is made entirely from old-school ideas such as mutual information and dimensionality reduction, and was for a while the best system if you limited it to fixed training data (still Wikipedia). A gripe: reviewers don't even seem to like explanations. If you can explain your system with well-understood operations, it's boring and "not novel". But when Google publishes magical mystery vectors, they lap it up.