I thought so, too. But then I asked it to define fake words that were portmanteaus I made up. Believe me, my understanding of BERT and discriminant models align

I thought so, too. But then I asked it to define fake words that were portmanteaus I made up. Believe me, my understanding of BERT and discriminant models aligned perfectly with what you're saying. But testing out the theory that it can break down and make meaning of fake words with accurate depictions of what words I'm combining proved me wrong. Generative models must work differently than you and I thought.