4 ms·
To some approximation, if you understood what BERT was at the time it was released, you'd consider it the first modern LLM. GPT-1 was OpenAI's BERT. Timeline w
by refulgentis 2y ago
To some approximation, if you understood what BERT was at the time it was released, you'd consider it the first modern LLM. GPT-1 was OpenAI's BERT.
Timeline would be viewed as:
2017: transformers
2018: bert
2018: gpt-1
2019: gpt-2
2020: gpt-3
2022: gpt-3.5 (chatgpt)
- rolisz 2y agoBERT was not large (it was under a billion parameters). And it wasn't an autoregressive language model like GPT
- refulgentis 2y agoRelative to today's models, it is small. For the time, it was large. Re: that it's not auto regressive, that's correct. Things built on eachother smoothly. Transformers to BERT to GPT.