26 ms·
As it seems to be getting worse over time (with 4.8 being worse than 4.6 and 5 worse than 4.8 again), could this simply be signs of model collapse? I mean more
by mort96 2mo ago
As it seems to be getting worse over time (with 4.8 being worse than 4.6 and 5 worse than 4.8 again), could this simply be signs of model collapse?
I mean more and more training data you find on the web is generated by previous models. The only reliable way to find human-generated text is to find text written before 2022, and they've used up all of that already. And AFAIU, these companies are using more and more synthetic data or semi-synthetic data.