4 ms·
> AI trained on AI generated data tend to perform worse Citation needed. To the best of my knowledge, synthetic data is a solid way to train models and isn't
by sebastiennight 2y ago
> AI trained on AI generated data tend to perform worse
Citation needed.
To the best of my knowledge, synthetic data is a solid way to train models and isn't going away anytime soon.
- dotnet00 2y agohttps://www.nature.com/articles/s41586-024-07566-y https://www.nature.com/articles/s41586-024-07566-y IIRC it had a pretty big thread here a few weeks ago
- sebastiennight 2y agoThanks. I remember that thread: https://news.ycombinator.com/item?id=41058194 https://news.ycombinator.com/item?id=41058194 What I took from the discussion is that there is very little chance that our next step for training SOTA models (eg. LLMs) will be "scraping the whole web including increasing volumes of ChatGPT-generated content". Instead, the synthetic content used to train new models is (from recent papers I've seen) mostly curated - not "indiscriminate" as the Nature paper discusses.