3 ms·
Absolutely right. First gen models are trained on 'virgin' non-LLM output. Subsequent models are tainted by ingesting AI replies. Rinse and repeat, and all you
by dnemmers 4mo ago
Absolutely right. First gen models are trained on 'virgin' non-LLM output. Subsequent models are tainted by ingesting AI replies. Rinse and repeat, and all you end up with is AI copies of AI replies, and the noise will completely overtake the signal.
- literalAardvark 4mo agoI'd have been more worried about that had the models not gotten incredibly good over time. But they did get good and this seems like a non-issue.