I think you are not taking into consideration the new content being created by AI with human in the loop, such as in the chatGPT interface. With 100M users Ope

I think you are not taking into consideration the new content being created by AI with human in the loop, such as in the chatGPT interface. With 100M users OpenAI might be generating on the order of 1 trillion tokens per month. These logs are mixed AI and human text, the human part containing tasks and feedback. By transforming these chat logs into training examples and fine-tuning the model there is a way to integrate LLM and human signals. Of course it is necessary to be mindful of copyright and PII during this process, not everything is generally useful to be included in the model. But having hundreds of millions of people inputting feedback into the model can scale even more than unassisted content publishing. Besides the AI users, we have the social networks. Billions of people comment on the news, if you scrape a comment thread such as this one you can readily generate an article from it, grounded in human feedback (sample based on this thread https://pastebin.com/R229b41s https://pastebin.com/R229b41s) Ultimately I don't believe publishers will stop creating content because there are many reasons to do it besides collecting advertising fees, but as a backup we have LLMs continuously learning by assisting their users or reshaping informal communications into well formed content. It's a two way street - both the LLM and human get help from each other. In turn, LLMs learn from other LLMs, usually Mistral and LLaMA would learn from GPT-4, this process works very very well. So any skills learned by a SOTA model with massive human chat logs would eventually be extracted for small models. Feedback percolates back to all AI agents. tl;dr Turning comment threads or human-AI chat logs into articles is a possible solution.