But there are a lot of selection mechanisms that filter out the bad generated content. For one, people will publish a fraction of the content they generate, and

But there are a lot of selection mechanisms that filter out the bad generated content. For one, people will publish a fraction of the content they generate, and it most likely will go through some more editing and selection steps by real people. Then the Internet itself becomes like a decentralized labeling service, where there are various signals you could detect to identify different qualities of content. What ends up being crawled for the next training iteration is a societally processed version of the raw model output that might have contributed to generating it. It's kind of interesting to think we might all be contributing to the training data of AGI not just through generating content but also what we choose to share or upvote.