I'm not sure what distinction you're getting at, but transformers do use "fill in the missing word" training and text generation chooses the next word (token ac

I'm not sure what distinction you're getting at, but transformers do use "fill in the missing word" training and text generation chooses the next word (token actually) one at a time. Once it chooses a word, it doesn't go back.