You could also consider some kinds of sampling of the dataset - basically not including images that are too close together in whatever space. There are also dat

You could also consider some kinds of sampling of the dataset - basically not including images that are too close together in whatever space. There are also data distillation methods that are supposed to generate a smaller training set - the ones I've seen don't generate images that look like real training samples though. I think it's definitely an interesting thing to study - it would be worth exploring what specifically the generative model can add. I've actually done something like this with GANs, but the gains came largely from generating plausible training images that were augmentations around some invariance, as I mentioned above.