3 ms·
I argue that you need both domain-general and domain-specific data. Most domains you could apply NLP to don't have billions of words to train on, especially if
by rspeer 9y ago
I argue that you need both domain-general and domain-specific data. Most domains you could apply NLP to don't have billions of words to train on, especially if the language of the data isn't English.
Pre-trained data is valuable, and you really aren't going to re-learn it all from your data, so why throw it out?
- laughingman2 9y agoYes, recent work on transfer learning https://arxiv.org/abs/1801.06146 https://arxiv.org/abs/1801.06146 " Fine-tuned Language Models for Text Classification" suggests that training a lstm on a large corpora for language modeling, then using it as sentence encoder after tuning it in specific dataset gives a considerable perfomance. This is covered in fast.ai 's new course 1.