I think that's basically what models such as Mistral are doing. They mentioned that they use very sanitized datasets for training as compared to simply scraping

I think that's basically what models such as Mistral are doing. They mentioned that they use very sanitized datasets for training as compared to simply scraping the web.