Just look at instruct fine tuning that is being done to completion models to turn them into assistant models. A few thousand examples are enough to alter the be

Just look at instruct fine tuning that is being done to completion models to turn them into assistant models. A few thousand examples are enough to alter the behavior of the model and what and how it outputs things significantly and thoroughly. Mechanisms like LoRA (a very efficient fine-tuning mechanism that has a accuracy penalty) change only a few layers at the top to alter the model considerably.