But BERT isn't a generative model? Unless you mean it's trained similar to a generative model by predicting the missing tokens? Also an interesting paper about

But BERT isn't a generative model? Unless you mean it's trained similar to a generative model by predicting the missing tokens? Also an interesting paper about hybrid energy based models: https://arxiv.org/pdf/1912.03263.pdf https://arxiv.org/pdf/1912.03263.pdf It's possible to have a model be both a classifier + generative, and the performance is comparable (although lower) to the SOTA of single-purpose models.