Somewhat yes. See "LoRA": https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685 They're not composable in the sense that you can take these adaptat

Somewhat yes. See "LoRA": https://arxiv.org/abs/2106.09685 https://arxiv.org/abs/2106.09685 They're not composable in the sense that you can take these adaptation layers and arbitrarily combine them, but training different models while sharing a common base of weights is a solved problem.