Fine-tuning is a solved problem using transformer adapters which are fast, small and match regular fine-tuning, so that's not an issue. As for smaller models, t

Fine-tuning is a solved problem using transformer adapters which are fast, small and match regular fine-tuning, so that's not an issue. As for smaller models, their usefulness will quickly wane when they will be seen as producing trivial gibberish in the light of cloud models available in 5 years.