> For better or worse, the quality of output LLMs can produce based on such publicly available data appears to be peaking. Recent work has documented that GPT,

> For better or worse, the quality of output LLMs can produce based on such publicly available data appears to be peaking. Recent work has documented that GPT, for example, has begun to exhibit declines in the quality of its output.3 It suggests no such thing, and the problems with citation 3 (https://arxiv.org/abs/2307.09009 https://arxiv.org/abs/2307.09009) have been extensively discussed. > It has been suggested this decline in performance is the expected result of a feedback loop, wherein data collected for training is increasingly contaminated by GPT itself, as users leverage the technology to produce and post content online.12 And the reasons why #12 (https://arxiv.org/abs/2305.17493 https://arxiv.org/abs/2305.17493) are totally inapplicable to the real world, as opposed to their artificial toy model where no kind of curation or selection whatsoever is done as synthetic samples approach 100% of the training data, have also been extensively discussed.