3 ms·

I'm not ML expert (especially in the natural language space), but I can't help thinking something is missing here. I know a ship 'is-a' boat. Cats and dogs are

I'm not ML expert (especially in the natural language space), but I can't help thinking something is missing here. I know a ship 'is-a' boat. Cats and dogs are unrelated animals etc. Maybe the colocated words are a very good approximation of a dictionary or real meaning?

sbpayne 11y ago
well the idea is that a word's meaning can be derived from its context (those context clues from grade school strike again). The co located words are simply the context. Since "ship" and "boat" frequently occur in similar contexts, they are similar words. Honestly, this blog post felt like they were trying to explain word2vec for the hundredth time, but tried real hard to explain it differently--so much so that it just made it more confusing.
mannykannot 11y ago
The author is refreshingly straightforward about what they are doing: "The key insight, made at the beginning of the information age, is to replace word meaning with something that machines can actually measure." Of course, if you do too much of this sort of meddling with usage, it will really mess with the machines' method for finding 'meaning' in the language. I expect humans would cope with such meddling much better, because they have the advantage of natural understanding.
slashcom 11y ago
So this is my research area. Distributional vectors are a proxy to word meaning. Words with similar vectors have similar meaning or are semantically related in some way. But usually, you just measure similarity by a single number: the cosine similarity between the two words' vectors. This number can tell you words are related or not, but it can't tell you how they're related [1]. There's been a good deal of work in automatically identifying that "ship is-a boat" (which is called hypernymy) or cats and dogs are unrelated animals (cohyponomy), but it's still being perfected. But it is useful. Words that are similar in meaning can be treated similarly. As a bad example: maybe I know that "anger" has negative sentiment, but I don't know what sentiment "furious" has, but I can infer it probably has negative sentiment since it's so similar to "anger". [1] There's a good deal of evidence that words that have high cosine similarity are more likely to be cohyponyms.

sbpayne 11y ago

well the idea is that a word's meaning can be derived from its context (those context clues from grade school strike again). The co located words are simply the context. Since "ship" and "boat" frequently occur in similar contexts, they are similar words. Honestly, this blog post felt like they were trying to explain word2vec for the hundredth time, but tried real hard to explain it differently--so much so that it just made it more confusing.

mannykannot 11y ago

The author is refreshingly straightforward about what they are doing: "The key insight, made at the beginning of the information age, is to replace word meaning with something that machines can actually measure." Of course, if you do too much of this sort of meddling with usage, it will really mess with the machines' method for finding 'meaning' in the language. I expect humans would cope with such meddling much better, because they have the advantage of natural understanding.

slashcom 11y ago

So this is my research area. Distributional vectors are a proxy to word meaning. Words with similar vectors have similar meaning or are semantically related in some way. But usually, you just measure similarity by a single number: the cosine similarity between the two words' vectors. This number can tell you words are related or not, but it can't tell you how they're related [1]. There's been a good deal of work in automatically identifying that "ship is-a boat" (which is called hypernymy) or cats and dogs are unrelated animals (cohyponomy), but it's still being perfected. But it is useful. Words that are similar in meaning can be treated similarly. As a bad example: maybe I know that "anger" has negative sentiment, but I don't know what sentiment "furious" has, but I can infer it probably has negative sentiment since it's so similar to "anger". [1] There's a good deal of evidence that words that have high cosine similarity are more likely to be cohyponyms.