3 ms·

I am sure a lot of work (and time) went into this but what's the point? If the model is not as good and the original then the applications are limited and the t

I am sure a lot of work (and time) went into this but what's the point? If the model is not as good and the original then the applications are limited and the training time is moot.

TacticalCoder 3y ago
> I am sure a lot of work (and time) went into this but what's the point? If the model is not as good and the original then the applications are limited and the training time is moot. If the several orders of magnitude (two in this case) improvement can be used to train other models, that's quite a thing no?
potatoman22 3y ago
It might not be as good on the same benchmark as the original BERT, but pretraining means you can expose it to entirely new corpuses of text e.g. medical or obscure programming languages. This has the potential to make it much easier to create domain-specific adaptations of BERT.