4 ms·
The reason we're seeing so many models right now is that the field is WIDE open for big leaps forward. Prior to LLaMA in February there were almost no openly l
by simonw 3y ago
The reason we're seeing so many models right now is that the field is WIDE open for big leaps forward.
Prior to LLaMA in February there were almost no openly licensed language models that were any good at all - BLOOM and GPT-J were pretty much the best of the bunch.
Then LLaMA happened and the flood gates opened - there was now a base model which people could fine-tune against to get really great results.
Llama 2 accelerated things even more because it's licensed for commercial use - so now it's worth spending real money building on top of it.
Llama 2 isn't even 2.5 months old at this point. I'm not at all surprised at the rate of innovation around it - there is SO MUCH scope for improvement, and so many new techniques to be explored.
I expect we'll see a slow down in the release of models at some point, once the low hanging fruit has all been picked off - but it's going to be a while away yet.
- NBJack 3y agoYou could even argue it's not so much the models themselves (LLM or diffusion), but rather the fact that fine tuning is remarkably accessible. Research is hot on ways to quantize parameters, swap portions for low memory, and the like. It's amazing to watch and use them.