I think we are seeing Knowledge Distillation at a large scale. OpenAI trains this megamodel which has an impressive amount of real-world knowledge packed into i

I think we are seeing Knowledge Distillation at a large scale. OpenAI trains this megamodel which has an impressive amount of real-world knowledge packed into it, and annotation tasks such as these are effectively extracting a small portion of the knowledge to pass onto other specialized models.