4 ms·
Supervised disambiguation? Isn't the entire reason word2vec is exciting because it's unsupervised?
by habitue 11y ago
Supervised disambiguation? Isn't the entire reason word2vec is exciting because it's unsupervised?
- accurrent 11y agoNot the only reason though. Its exciting because it turns a word into a reasonably sized vector. After that the vector can be fed into neural networks (nueral networks take vectors for input) or operations can be performed. Like Paris - France ~= Beijing - China.
- ya3r 11y agoI have seen this many times that people claim the word2vec is unsupervised. But I think that is inaccurate. word2vec is using a very weak supervision which is the order in which words appear in a meaningful sentence. And I think it is fascinating to use this kind of weak supervision to build distributed embedding for words.
- Matumio 11y agoThis makes me wonder if the supervised/unsupervised distinction is useful at all. Maybe you could say instead that all learning algorithms simply need some way to measure similarity between the training examples, and it doesn't make a fundamental difference if you cluster examples by their target label ("supervised") or by their input vectors ("unsupervised") or by the context in which they appear (word2vec).
- fnl 11y agoSkip-grams only take focus word, context word pairs. So those pairs do not take order into account
- ya3r 11y agobeing in the same context is some kind of order.
- fnl 11y agoThat's still a bit far fetched. "Weakly supervised" refers to using a small amount of labeled data. This is not the case for word2vec and similar embedding methods. As a matter of fact, rather, the method presented here, sense2vec, would qualify, at it indeed is weakly supervised.
- ya3r 11y agoWhere we use "small amount of labeled data" is called "semi-supervised".
- gbrits 11y ago> word2vec is using a very weak supervision which is the order in which words appear in a meaningful sentence. What's supervised about that? Can't this be produced by a large enough corpus?
- hellameta 11y agoWord2Vec is exciting because of its result, not particularly its method. This paper is a pretty straight forward extension of word2vec by combining it's 'unsupervised' result with various labels to make it more useful. You can think of word2vec as a kind of independent representation of language (which is a stretch for the purists but for sake of conversation I think it's okay) that can be applied to various domains by 'supervising' it.