2 ms·
With all these models trained on a corpus of texts from the Internet[1], what's going to happen in 5 years when the internet is full of text generated after Jan
by cratermoon 3y ago
With all these models trained on a corpus of texts from the Internet[1], what's going to happen in 5 years when the internet is full of text generated after January 2022, the current cutoff data for ChatGPT? As an increasing proportion of training data is either pre-2002 or LLM-generated confabulations, when will it all collapse in on itself? Do these companies have any plans for how they are going to get accurate and factual input after their output dominates?
1. Corpora: https://commoncrawl.org/ https://commoncrawl.org/, https://www.english-corpora.org/ https://www.english-corpora.org/