4 ms·
Isn’t this before any curation has happened? I looked at it, I can see why it looks bad, but if they’re really being open about the whole pipeline, they have to
by andy99 11mo ago
Isn’t this before any curation has happened? I looked at it, I can see why it looks bad, but if they’re really being open about the whole pipeline, they have to include everything. Giving them a hard time for it only promotes keeping models closed.
That said I like to think of it was my dataset I would have shuffled that part down in the list so it didn’t show up on the hf preview
- Oras 11mo agoHard time? What value does adult videos description, views and comments add to small (7,32B) models?
- andy99 11mo agoIt says it’s common crawl, I interpret it to mean this is a generic web scrape dataset, presumably they filter stuff out they don’t want before pretraining. You’d have to do do some ablation testing to know what value it adds
- ccgreg 11mo agoCommon Crawl is a particular dataset. commoncrawl.org
- khimaros 11mo agowhat if that's where they learned how to utilize the double entendre? hard times indeed.