4 ms·

Indeed, the implementation is very simple. We had considered this idea some times as well, but like you didn't get around to implement/evaluate it yet. So, it's

Indeed, the implementation is very simple. We had considered this idea some times as well, but like you didn't get around to implement/evaluate it yet. So, it's good to hear that it works. One thing notably absent from the paper is a discussion of the trade-off between augmenting tokens with annotations in this way for sense disambiguation vs. data sparseness. Their approach may make the embeddings for frequent senses better, but the difficulty in WSD is typically in low-frequency senses. I think that particularly in disambiguation using part-of-speech tags, there is still a high semantic relatedness between senses, especially in languages with frequent nominalization or verbalization. I can imagine that an model that predicts a target (or context) in decomposed form (token and label) might improve embeddings for low-frequent senses.

syllogism 11y ago
This isn't really WSD though, or at least, only very weakly. Rare words are usually pretty unambiguous for part-of-speech. I would guess this mostly has an effect on the top 5,000 items of the vocabulary, and most of the rest of the lexicon only has a single "sense".
- danieldk 11y ago
  This isn't really WSD though, or at least, only very weakly. Sure. I was pointing to real WSD where sparseness becomes even a stronger problem than when your definition of sense is restricted to part-of-speech tag or sentiment. Rare words are usually pretty unambiguous for part-of-speech. I was talking about (possibly) frequent words where some part-of-speech are infrequent, not about rare words. To take five more or less random examples form the Brown corpus (yes, we train on large corpora, but I think similar distributions could hold for less frequent forms in languages with e.g. frequent nomalization, not everyone speaks English!): mother NN 173 VB 1 code NN 20 VB 1 hanging NN 1 VBG 20 level JJ 14 NN 172 VB 2 services NNS 115 VBZ 1 If your learning method is as coarse-grained as simply throwing the token plus part-of-speech into word2vec or wang2vec, some will be below the frequency cut-offs (or will be to sparse to learn good embeddings), while other 'senses' may in reality be semantically similar.
  - syllogism 11y ago
    Thanks for the explanation. I see what you're saying now.

syllogism 11y ago

This isn't really WSD though, or at least, only very weakly. Rare words are usually pretty unambiguous for part-of-speech. I would guess this mostly has an effect on the top 5,000 items of the vocabulary, and most of the rest of the lexicon only has a single "sense".

danieldk 11y ago
This isn't really WSD though, or at least, only very weakly. Sure. I was pointing to real WSD where sparseness becomes even a stronger problem than when your definition of sense is restricted to part-of-speech tag or sentiment. Rare words are usually pretty unambiguous for part-of-speech. I was talking about (possibly) frequent words where some part-of-speech are infrequent, not about rare words. To take five more or less random examples form the Brown corpus (yes, we train on large corpora, but I think similar distributions could hold for less frequent forms in languages with e.g. frequent nomalization, not everyone speaks English!): mother NN 173 VB 1 code NN 20 VB 1 hanging NN 1 VBG 20 level JJ 14 NN 172 VB 2 services NNS 115 VBZ 1 If your learning method is as coarse-grained as simply throwing the token plus part-of-speech into word2vec or wang2vec, some will be below the frequency cut-offs (or will be to sparse to learn good embeddings), while other 'senses' may in reality be semantically similar.
- syllogism 11y ago
  Thanks for the explanation. I see what you're saying now.