"Inference, question-answering, NER detection/disambiguation are pretty important NLP tasks" Yes indeed. "While GPT-2 has gained mindshare for its generative c

"Inference, question-answering, NER detection/disambiguation are pretty important NLP tasks" Yes indeed. "While GPT-2 has gained mindshare for its generative capabilities, BERT and other pre-trained Transformer Encoder models are used for production workloads" You rephrased my point pretty well, while openAI search for "fun" tasks, deepmind and others allow progress on real world tasks. You use BERT which is nice but do you consider using it's successor: XLnet? "take a look at the work by HuggingFace, the Flair project, Spark-NLP and others." I was aware of Flair (from Zalando) but thank you for Huggingface and Spark-NLP, I will take a look!