4 ms·
> Instead, each word should have a vector based on context, for disambiguation and nuance. You do realize even under vanilla Word2vec, each word embedding is a
by orthoganol 9y ago
> Instead, each word should have a vector based on context, for disambiguation and nuance.
You do realize even under vanilla Word2vec, each word embedding is a sum of both the word vector and a vector of a surrounding window of words? Each embedding results from maximizing the likelihood of a small surrounding context given the word. Thus, context is factored in.
- nl 9y agoThis is true, but the context implicit in the vector is the average of the training data. Vanilla Embedding based models don't handle word order well (as some of the examples upon this slide deck show).