LoRAs are, like most fine-tuning, a spectrum. LoRAs can be nearly the same size as the original model, with nearly the same representation capacity/trainabilit

LoRAs are, like most fine-tuning, a spectrum. LoRAs can be nearly the same size as the original model, with nearly the same representation capacity/trainability, or they can be a tiny tiny fraction of the original model size, with correspondingly fewer learnable parameters. As such, they are suitable for nearly all tasks. We should be asking if they are better than regular fine-tuning, or soft-prompts (aka textual inversion), or slapping new trainable layers on the end (aka hypernetworks). The stable diffusion community seems to think that they are.