2 ms·
Common Crawl is heavily populated with reddit threads So a massive chunk of LLMs are Reddit data
by AndrewKemendo 3mo ago
Common Crawl is heavily populated with reddit threads
So a massive chunk of LLMs are Reddit data
- ccgreg 3mo agoThat isn't true. You're welcome to peruse our index to prove or disprove your claim.
- AndrewKemendo 3mo agoI appreciate the response because it made me go back and look at my sources. What I conflated was that WebText utilizes Reddit links and data (mostly prior to 2023/4) and that I combined WebText and common crawl for the original GPT2 bootstrap into one dataset I was incorrectly connecting Reddit-mediated WebText pipeline to Common Crawl. So thanks for the correction!
- ccgreg 3mo agoAppreciate you double-checking.