TL;DR I think its complementary. Synthetic data is particularly valuable when even the unlabelled data is expensive to obtain. For example if you want to train

TL;DR I think its complementary. Synthetic data is particularly valuable when even the unlabelled data is expensive to obtain. For example if you want to train a driverless car, you may never see an ambulance driving at night in the rain even if you drive for thousands of miles. In that case, being able to synthesise data makes a lot of sense and we have lots of tools for computer graphics that make this easy. Synthetic data can also be useful to share data when there are privacy concerns but my own feeling here is that there are better approaches to privacy preservation, like federated learning and learning via randomised response (https://arxiv.org/pdf/2001.04942.pdf https://arxiv.org/pdf/2001.04942.pdf). In general though, outside of some vision applications, I'm pretty sceptical of synthetic data. For synthetic data to work well, you need a really good class conditional generator. E.g "generate a tweet that is a negative sentiment review" but if you have a sufficiently good model to do this, then you can probably use that model to solve your classification task anyway. For most settings, I think synthetic data will work for data augmentation as a regulariser but will not be a substitute for all labelled data. For the labelled data, active learning should still help.