3 ms·
Closing the barn door after the horse has already bolted. Pulling data off Reddit now will likely give you a very large amount of polluted data from LLMs. I me
by pixl97 4y ago
Closing the barn door after the horse has already bolted.
Pulling data off Reddit now will likely give you a very large amount of polluted data from LLMs. I mean, yea it could be useful for some broad topics at this point, but still likely to contain a lot of GPTs own feedback.
It's likely that companies like OpenAI will just use their old reddit dataset, and then move to scraping things like YouTube for not just text, but audio and imagery too.
- welshwelsh 4y agoIt doesn't really matter if some of the data is generated by LLMs. It's possible for LLMs to improve themselves by training on their own output, there is no strict requirement for "fresh" content. If it's mixed with human content and gets human feedback through voting that's great training data.
- vctrm67 4y agoWhat if an LLM generates absolute crap that then gets upvoted by bots?