4 ms·
I wish there was a way to augment data as easily for free text, and other business data. I always see these few-shot learning papers for images, I suspect becau
by starpilot 7y ago
I wish there was a way to augment data as easily for free text, and other business data. I always see these few-shot learning papers for images, I suspect because it's easy to augment image datasets and because image-recognition is interesting to laypeople. The vast majority of data we deal with in business is text/numerical which is much harder to use in these approaches.
- amitness 7y agoAgree with you on this. For text data, there was a paper called "UDA"(https://arxiv.org/abs/1904.12848 https://arxiv.org/abs/1904.12848) that did some work on this direction. They augmented text by using backtranslation. Basic idea is you take text in English, translate that to some other language say French and then translate back the French text to English. Usually, you get back an English sentence that is different that the original English sentence but has the same meaning. Another approach they use to augment is to randomly replace stopword/low tf-idf(intuitively say very frequent words like a, an, the) with random words. You will find implementation of UDA on GitHub and try that out. I am learning these existing image semi-supervised technique right now and the plan is to do research on how we can transfer those ideas to text data. Let's see how it goes.
- codegladiator 7y agoHaha that's how we used to generate blog spam content and comments :p
- deleted 7y ago[deleted]
- amitness 7y agoWhat are other techniques you use to generate spam? Maybe the research community can learn from you guys
- Der_Einzige 7y agoJust replace a random subset of words with their nearest neighbors computed using a fancy word embedding model like BERT or GPT-2. If you cannot do that due to them not including a vocabulary file, you're stuck using something like word2vec/FastText which is fine but not ideal if you're looking for grammatical correctness...
- amitness 7y agoYeah, since BERT has been trained specifically on this task of replacing random masked words, the resulting augmented sentence should sound natural compared to something like Word2Vec.