3 ms·
I have seen this many times that people claim the word2vec is unsupervised. But I think that is inaccurate. word2vec is using a very weak supervision which is
by ya3r 11y ago
I have seen this many times that people claim the word2vec is unsupervised. But I think that is inaccurate.
word2vec is using a very weak supervision which is the order in which words appear in a meaningful sentence. And I think it is fascinating to use this kind of weak supervision to build distributed embedding for words.
- Matumio 11y agoThis makes me wonder if the supervised/unsupervised distinction is useful at all. Maybe you could say instead that all learning algorithms simply need some way to measure similarity between the training examples, and it doesn't make a fundamental difference if you cluster examples by their target label ("supervised") or by their input vectors ("unsupervised") or by the context in which they appear (word2vec).
- fnl 11y agoSkip-grams only take focus word, context word pairs. So those pairs do not take order into account
- ya3r 11y agobeing in the same context is some kind of order.
- fnl 11y agoThat's still a bit far fetched. "Weakly supervised" refers to using a small amount of labeled data. This is not the case for word2vec and similar embedding methods. As a matter of fact, rather, the method presented here, sense2vec, would qualify, at it indeed is weakly supervised.
- ya3r 11y agoWhere we use "small amount of labeled data" is called "semi-supervised".
- gbrits 11y ago> word2vec is using a very weak supervision which is the order in which words appear in a meaningful sentence. What's supervised about that? Can't this be produced by a large enough corpus?