3 ms·
At this point, they all using each other because so much of the new content they are scraping for data is generated. These models will converge and plateau bec
by parineum 1y ago
At this point, they all using each other because so much of the new content they are scraping for data is generated.
These models will converge and plateau because the datasets are only going to get worse as more of their content is incestuous.
- wkat4242 1y agoYes indeed some studies were already done on this.
- sovietmudkipz 1y agoI recall that AI trained on AI output over many cycles eventually becomes something akin to noise texture as the output degrades rapidly. Won’t most AI produced content put out into the public be human curated, thus heavily mitigating this degradation effect? If we’re going to see a full length AI generated movie it seems like humans will be heavily involved, hand holding the output and throwing out the AI’s nonsense.
- AstroBen 1y agoSome will be heavily curated, by those who care about quality. This is a lot slower to produce, requires some expertise to do right, so there will be far less of it The vast majority of content will be (is) the fastest and easiest to create - AI slop
- jsheard 1y agoThe default Llama 4 system prompt even instructs it to avoid using various ChatGPT-isms, presumably because they've already scraped so much GPT-generated material that it noticably skews their models output.
- zackangelo 1y agoThere might be a plateau coming but I’m not sure that will be the reason. It seems counterintuitive but there is some research suggesting that using synthetic data might actually be productive.
- jsheard 1y agoI think there's probably a distinction to be made between deliberate, careful use of synthetic data, as opposed to blindly scraping 1PB of LLM generated SEO spam and force-feeding it into a new model. Maybe the former is useful, but the latter... probably not.