3 ms·
They used more computer power than BERT and much more data for training. Also they still under-perform comparing to the best BERT-base model in SQuaD 2.0 benchm
by riku_iki 7y ago
They used more computer power than BERT and much more data for training.
Also they still under-perform comparing to the best BERT-base model in SQuaD 2.0 benchmark.
The big value of BERT is that it is publicly available, and can be modified and improved.
- roseway4 7y agoAn English pretrained model is available. Also, the SQuAD2.0 EM numbers are higher than the BERT results published in the BERT paper. Subsequent work has improved on these and no doubt XLNet will see similar activity.