2 ms·
https://www.nature.com/articles/s41586-024-07566-y https://www.nature.com/articles/s41586-024-07566-y IIRC it had a pretty big thread here a few weeks ago
by dotnet00 2y ago
https://www.nature.com/articles/s41586-024-07566-y https://www.nature.com/articles/s41586-024-07566-y
IIRC it had a pretty big thread here a few weeks ago
- sebastiennight 2y agoThanks. I remember that thread: https://news.ycombinator.com/item?id=41058194 https://news.ycombinator.com/item?id=41058194 What I took from the discussion is that there is very little chance that our next step for training SOTA models (eg. LLMs) will be "scraping the whole web including increasing volumes of ChatGPT-generated content". Instead, the synthetic content used to train new models is (from recent papers I've seen) mostly curated - not "indiscriminate" as the Nature paper discusses.