I agree that larger models have more memorization capacity, but that isn't what find exciting about LLMs, personally. You can ask GPT-4 some esoteric question a

I agree that larger models have more memorization capacity, but that isn't what find exciting about LLMs, personally. You can ask GPT-4 some esoteric question and get back an answer that's more likely to be truthful than Llama 3's, but either way, it's the same result: you have to look it up for yourself. What I think is much more interesting about LLMs is their ability to transform natural language inputs. Applications like open-book question answering, summarization, data extraction, and so forth. For these things, small models are at the point where they are commercially useful now. And if you do fine-tune them on your own domain, they will work better than larger general-purpose models.