IT can be, though. High quality synthetic data for training is a thing, see Phi-2 [0] (and many more examples, people having been using GPT-4 outputs for finetu

IT can be, though. High quality synthetic data for training is a thing, see Phi-2 [0] (and many more examples, people having been using GPT-4 outputs for finetuning a LOT to great avail). [0] https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/ https://www.microsoft.com/en-us/research/blog/phi-2-the-surp...