10 ms·
Besides what others have already said: word vectors are usually calculated from the context they appear in, the other words surrounding them. The vector represe
by slx26 9y ago
Besides what others have already said: word vectors are usually calculated from the context they appear in, the other words surrounding them. The vector representation can be intuitively understood as representation of the context the word appears in. The number of words we take into account for that context is called 'window'.
This has very interesting possibilities: you can complete sentences where a word is missing (you have a context, so you can search for the best matching word vector), use it in text autocorrection tasks, and other classic natural language processing problems.
Learned word representations are also coherent between them, so you can use them to make analogies (the distance from 'Spain' to 'Madrid' is similar to the distance from 'France' to 'Paris'), so they implicitly hold some of the semantic info between words.
It can also be used to find related words. Synonyms, antonyms and related words have similar representations. For example, 'facebook', 'twitter' and 'instagram' have similar vectors (vectors with similar directions). But you can also try with famous musician or band names, tech related terms, etc.
Finally, word vectors, unlike other language models, can store representations for large windows and vocabulary sizes in a few GB, which is another useful property in certain situations, and makes them easy to handle.