not strictly true I think - you could add new units throughout and train those while freezing existing units (adapter-based fine-tuning) - you could train all

not strictly true I think - you could add new units throughout and train those while freezing existing units (adapter-based fine-tuning) - you could train all units and use e.g. low-rank adaptation to limit how much they can change - you could do prefix tuning and train an input to add at every layer see e.g. - https://lightning.ai/pages/community/article/understanding-llama-adapters/ https://lightning.ai/pages/community/article/understanding-l...