3 ms·
BERT is a transformer! The unlock happened within transformers, yes, but they were not exactly super new or innovative architectures at that time. The scale was
by paytonjjones 4mo ago
BERT is a transformer! The unlock happened within transformers, yes, but they were not exactly super new or innovative architectures at that time. The scale was the main innovation that brought us from BERT to today's frontier.