2 ms·

Lovely! Right there in the first page of example strings: > Huge selection of exclusive animal sex videos is what makes 3AnimalSexTube the best animal porn tu

• yorwba 3y ago
  " by carefully adding additional filtering steps, we managed to push the performance of FineWeb well above that of the original RefinedWeb, and models trained on our dataset also outperform models trained on other commonly used high quality web datasets (like C4, Dolma-v1.6, The Pile, SlimPajama) on our aggregate group of benchmark tasks. That said, we think there is still room for additional filtering and improvement and intend to continue exploring how to improve the dataset quality in coming versions of FineWeb. " I think future advances in model quality are more likely to come from better filtering to decrease the amount of bad training data, rather than training on an ever growing share of the entire internet.
• yogorenapan 3y ago
  I don’t think that’s bad to be in the raw training data per-se. Such content will be RLHF’ed out. Some users will certainly ask it for such tasks and it will somewhat deliver