4 ms·
The original T5 models were trained with masked language modelling similar to BERT and afterwards also trained on a autoregressive task similar to GPT.
by navjordj 3y ago
The original T5 models were trained with masked language modelling similar to BERT and afterwards also trained on a autoregressive task similar to GPT.