This feels like it's a mish-mash of concepts that are used in different settings (pre-trained word embeddings, unmasked attention, beam search, ...) but that ar

This feels like it's a mish-mash of concepts that are used in different settings (pre-trained word embeddings, unmasked attention, beam search, ...) but that aren't really used in things like ChatGPT. I guess if you're reading this kind of article, the details don't really matter that much, though.