2 ms·
Just replace a random subset of words with their nearest neighbors computed using a fancy word embedding model like BERT or GPT-2. If you cannot do that due to
by Der_Einzige 7y ago
Just replace a random subset of words with their nearest neighbors computed using a fancy word embedding model like BERT or GPT-2.
If you cannot do that due to them not including a vocabulary file, you're stuck using something like word2vec/FastText which is fine but not ideal if you're looking for grammatical correctness...
- amitness 7y agoYeah, since BERT has been trained specifically on this task of replacing random masked words, the resulting augmented sentence should sound natural compared to something like Word2Vec.