The old truism is that "in the short term things change less than you expect, and in the long term things change more than you expect". I'm wondering if we're

The old truism is that "in the short term things change less than you expect, and in the long term things change more than you expect". I'm wondering if we're going to get into a circular problem of information "purity". At the moment these models are trained on entirely human created content (because that's the only thing that exists). That training data is therefore roughly as true as it can be. But what happens when significant portions of the internet have been generated by LLMs? What happens when other models are unwittingly trained on them? Do they, very subtly just become worse and worse? Does the prevalence of these models mean that people write material less and less - exacerbating the signal to noise problem even more. Basically, these models are just really efficient at recycling things that people have written. What happens when nobody writes much and it's just recycling things produced by a model?