6 ms·
Excellent article, thank you. My snag in thinking about word2vec is how the vector model stores information about words with multiple, significantly different d
by rmbryan 8y ago
Excellent article, thank you. My snag in thinking about word2vec is how the vector model stores information about words with multiple, significantly different definitions, such as 'polish', Eastern Europe or glistening clean.
- yodon 8y agoIn a sufficiently high dimensional space, like used in word2vec, concepts can have lots of neighbors along wildly different dimensions.
- physicsyogi 8y agoWord2vec doesn't really address multiple meanings (polysemy). There has been some progress on this though. Sebastian Ruder has been tracking the state-of-the-art in this here: [1]. [1] https://nlpprogress.com/english/word_sense_disambiguation.html https://nlpprogress.com/english/word_sense_disambiguation.ht... Edit: formatting
- c256 8y agoClearly distinct meanings are usually fine, as there’s little chance of context overlap causing clashes. The subtle distinctions can cause more trouble, but those tend to occur in situations more refined than we expect systems like these to handle anyway. The section on interchangeability versus context might be helpful for illuminating this idea.