3 ms·
So if a new model is trained with 10x parameters of gpt3, is gpt now no longer an llm?
by srcreigh 4y ago
So if a new model is trained with 10x parameters of gpt3, is gpt now no longer an llm?
- blagie 4y agoIt's a little bit weird. In semiconductors, we ran into the same problem. "large scale integration" happened at 500 transistors, and "very large scale integration" at 20,000. For a while, "ultra large scale integration" happened at a million and more qualifiers were added, until everyone decided it was ridiculous, and we went back to VLSI. From my perspective, LLMs are about where the language models start to behave in ways which feel sentient and replace mainstream human tasks, such as making first drafts of emails, code, or legal filings. That breakpoint was around GPT-3. I can't predict the future. When we have 3T parameter models, we might: - Call them LLMs, and group them with GPT-3 - Call them LLMs, but shift the goal posts to where GPT-3 is no longer one - Call them VLLM - Call them AGI However, what's clear to me is that state-of-the-art models with e.g. 3B parameters are qualitatively different from GPT-3 and friends. I don't consider those to be LLMs.
- not_a_shill 4y agoNah, the new ones can just be called Xllms. Then we can just keep adding Xs every generation.