4 ms·
BERT was not large (it was under a billion parameters). And it wasn't an autoregressive language model like GPT
by rolisz 2y ago
BERT was not large (it was under a billion parameters). And it wasn't an autoregressive language model like GPT
- refulgentis 2y agoRelative to today's models, it is small. For the time, it was large. Re: that it's not auto regressive, that's correct. Things built on eachother smoothly. Transformers to BERT to GPT.