I think the real reason they created GPTs is to generate synthetic data, with LLM, reference materials and tools, while also having human in the loop. This kind

I think the real reason they created GPTs is to generate synthetic data, with LLM, reference materials and tools, while also having human in the loop. This kind of data would be great to maximize the quality of results. But as a side effect it would surface the weak spots of the model. I estimated that, at 100M users, and let's say 10K tokens per user per month, they would collect 1T tokens per month. In one year they double the size of the training set (13T tokens). May be off by an order of magnitude, I don't know the average user token count per month. But it is not weird to think that GPT has generated as much chat text as its training set. tl;dr They're just chasing better data.