3 ms·
Absolutely right. First gen models are trained on 'virgin' non-LLM output. Subsequent models are tainted by ingesting AI replies. Rinse and repeat, and all you
by dnemmers 2mo ago
Absolutely right. First gen models are trained on 'virgin' non-LLM output. Subsequent models are tainted by ingesting AI replies. Rinse and repeat, and all you end up with is AI copies of AI replies, and the noise will completely overtake the signal.
- literalAardvark 2mo agoI'd have been more worried about that had the models not gotten incredibly good over time. But they did get good and this seems like a non-issue.