3 ms·
that's a nice link but the excerpt you quote is kind of misleading; the word vectors in this case are not one-hot encodings. They are learned, continuous repres
by bllguo 9y ago
that's a nice link but the excerpt you quote is kind of misleading; the word vectors in this case are not one-hot encodings. They are learned, continuous representations. But one-hot representations are also a kind of word vector.
word vectors are vector representations of each word in the vocabulary. Here they are learned by a neural net. the length of the vector is the # of features. Just for intuition, one feature of a word the NN could learn is the gender of a word, and so on.
- v1n337 9y agoYes, quite correct! The vectors are dense representations. But the features aren't individually interpretable, in practice. For instance, the 'gender' of a word may have it's signal scattered over several features/dimensions of the learned vector.
- Choco31415 9y agoTo help with intuition, here are a few example word vectors that we might encounter: "fruit": {food: 0.99, gender: -0.05, size: 0.2} "king": {food: -0.9, gender: 0.92, size: 0.56} Building off of what v1n337 stated, though, axis can easily be skewed and rotated such that they're still interpretable, just not obviously so.