4 ms·
So, if AI trained on AI generated data tend to perform worse, and you're trying to have AI generate the data that ultimately props up our civilization... Clear
by dotnet00 2y ago
So, if AI trained on AI generated data tend to perform worse, and you're trying to have AI generate the data that ultimately props up our civilization...
Clearly this can only end well
- sebastiennight 2y ago> AI trained on AI generated data tend to perform worse Citation needed. To the best of my knowledge, synthetic data is a solid way to train models and isn't going away anytime soon.
- dotnet00 2y agohttps://www.nature.com/articles/s41586-024-07566-y https://www.nature.com/articles/s41586-024-07566-y IIRC it had a pretty big thread here a few weeks ago
- sebastiennight 2y agoThanks. I remember that thread: https://news.ycombinator.com/item?id=41058194 https://news.ycombinator.com/item?id=41058194 What I took from the discussion is that there is very little chance that our next step for training SOTA models (eg. LLMs) will be "scraping the whole web including increasing volumes of ChatGPT-generated content". Instead, the synthetic content used to train new models is (from recent papers I've seen) mostly curated - not "indiscriminate" as the Nature paper discusses.