4 ms·
One method to partially rectify the problem you mention is to add additional preprocessing to the text to identify ngrams and add part of speech tags to words.
by morelandjs 9y ago
One method to partially rectify the problem you mention is to add additional preprocessing to the text to identify ngrams and add part of speech tags to words. For example [The, river, bank] might be restructed as [The, river bank (bigram)] where "river bank" is embedded as its own word vector.
This can also be used to disambiguate words like "hit" which can be used as a verb and a noun. You just replace "hit" with "hit|noun" and "hit|verb".
- loxias 9y agoYes! And, also, I swear I've seen this method done before. I can't remember the paper. Can anyone help? Basically, instead of making neural word embeddings, much like you describe, the objects being embedded in a vector space were "hit|noun(1)" "hit|noun(2)" "hit|verb", and so on. I believe they used WordNet or some ontology, or maybe a POS corpus like Penn Treebank....
- jdonaldson 9y agoI think you're talking about sense2vec
- loxias 9y agoYes!!! It was! Thanks so much!