There have been some much publicised studies showing poor performance of training from scratch on purely undiscriminated synthetic data. Curated synthetic data

There have been some much publicised studies showing poor performance of training from scratch on purely undiscriminated synthetic data. Curated synthetic data has yielded excellent results. Even when the curation is AI There is no requirement to train from scratch in order to get better, you can start from where you are. You may not be able to design a living human being, but random changes and keeping the bits that performed better can.