4 ms·
We are not “a year or two into LLMs”. To name one example, BERT is more than 5 years old.
by l33t233372 4y ago
We are not “a year or two into LLMs”. To name one example, BERT is more than 5 years old.
- FeepingCreature 4y agoIs 110 million parameters really a "large" language model though? Especially since the models gain novel skills as they scale up.
- srcreigh 4y agoSo if a new model is trained with 10x parameters of gpt3, is gpt now no longer an llm?
- blagie 4y agoIt's a little bit weird. In semiconductors, we ran into the same problem. "large scale integration" happened at 500 transistors, and "very large scale integration" at 20,000. For a while, "ultra large scale integration" happened at a million and more qualifiers were added, until everyone decided it was ridiculous, and we went back to VLSI. From my perspective, LLMs are about where the language models start to behave in ways which feel sentient and replace mainstream human tasks, such as making first drafts of emails, code, or legal filings. That breakpoint was around GPT-3. I can't predict the future. When we have 3T parameter models, we might: - Call them LLMs, and group them with GPT-3 - Call them LLMs, but shift the goal posts to where GPT-3 is no longer one - Call them VLLM - Call them AGI However, what's clear to me is that state-of-the-art models with e.g. 3B parameters are qualitatively different from GPT-3 and friends. I don't consider those to be LLMs.
- not_a_shill 4y agoNah, the new ones can just be called Xllms. Then we can just keep adding Xs every generation.