This is a well-known problem that has been an issue for years now. You can't use models to generate data for models because it leads to "model collapse" where i

This is a well-known problem that has been an issue for years now. You can't use models to generate data for models because it leads to "model collapse" where it amplifies quirks in the generated data until it's all quirks. Here is a random university press release about it (grain of salt etc) https://www.utoronto.ca/news/training-ai-machine-generated-text-could-lead-model-collapse-researchers-warn https://www.utoronto.ca/news/training-ai-machine-generated-t... In practice you can do it a bit (generated data from a better / different model is fine, some generated data might be useful if there is non generated data etc.)