For "full fine tuning", mathematically there's no difference. Fine tuning is just extending the training on new data. What you are suggesting is called "curric

For "full fine tuning", mathematically there's no difference. Fine tuning is just extending the training on new data. What you are suggesting is called "curriculum learning", and though it hasn't been applied to LLMs yet to the best of my knowledge, it has proven to improve learning and decrease training times in other areas of ML.