Small pretrained models are often too “dumb” to be useful, but, if you have a task in which you can tune a model, their pretraining means you can produce an eff

Small pretrained models are often too “dumb” to be useful, but, if you have a task in which you can tune a model, their pretraining means you can produce an effective model with a shockingly small corpus. These can be more reliable than off the shelf models in an automated process, because even the large pretrained models have a lot of “behaviors” you can trigger with surprising inputs. Most recently I retrained SmolLM2 to translate intents to SDXL prompts.