3 ms·
That's not what's impressive here. > While we can see that BERT-Base performed better at every task; the results for this model would have been very good (poss
by nighthawk454 3y ago
That's not what's impressive here.
> While we can see that BERT-Base performed better at every task; the results for this model would have been very good (possibly SOTA for a few tasks) in early 2018.
SOTA - meaning no models of _any_ size at that time could do this. And they're achieving that now on a single consumer GPU with 1/30th the data and 1/40th the epochs. That's huge for improvements to training efficiency.