You don't need humans. 'Self-distillation' and related approaches work surprisingly well. Like with GPT-3, you can get much better question-answering or French

You don't need humans. 'Self-distillation' and related approaches work surprisingly well. Like with GPT-3, you can get much better question-answering or French translation simply by generating a bunch of examples, ranking them with the model's own internal metrics (a simple ranking would be to accept an answer if the majority of one's samples agree on the same answer), and train further on that - with no humans or additional 'real' data involved. Remember that DALL-E 1 got a big quality boost by generating like 50 samples and using CLIP to rank them by similarity to the original text input. (You can also retrain the DALL-E 1 model to run in reverse - image to text - to score images. CogView did this.) And if you are training on Internet data, then you are presumably training on human-curated selections. Even a minimal level of selection, like picking the best out of a dozen, by a DALL-E 2 API user, or by downstream humans selectively resharing higher-quality images, is a (slow) bootstrap.