Technically it is only taking gradients from the training text, not the text itself. And those gradients are all added up so the contribution of any one of them

Technically it is only taking gradients from the training text, not the text itself. And those gradients are all added up so the contribution of any one of them is diluted to almost nothing.