My 2 favorites are 1. BERT - easy to understand, SOTA results on most NLP tasks, code and pretrained models readily available. What more can you ask! 2. ULMFit

My 2 favorites are 1. BERT - easy to understand, SOTA results on most NLP tasks, code and pretrained models readily available. What more can you ask! 2. ULMFit - transfer learning for NLP, have been waiting for something like this since a long time. Both these mean getting better results/models for most NLP tasks with lesser training data, effort and time. Here is another link explaining these topics in a little more detail (a must read if you are in the NLP field) -- https://jalammar.github.io/illustrated-bert/ https://jalammar.github.io/illustrated-bert/