2 ms·Yes, transformers allow attention to flow to all previous tokens, even while the phrase is being generated.by visarga 6y agoYes, transformers allow attention to flow to all previous tokens, even while the phrase is being generated.