3 ms·GPTs also get gradients from all tokens, BERT only on 15% masked tokens. GPTs are more effective.by visarga 2y agoGPTs also get gradients from all tokens, BERT only on 15% masked tokens. GPTs are more effective.