> Getting humans to refine your data is the best solution right now Source ? All those big models are trained with data for which the source is not known or v

> Getting humans to refine your data is the best solution right now Source ? All those big models are trained with data for which the source is not known or vetted. The amount of data needed is not human-refinable. For example for language models we train mostly on subsets of CommonCrawl + other things. CommonCrawl data is “cleaned” by filtering out known bad sources and with some heuristics such as ratio of text to other content, length of sentences etc. The final result is a not too dirty but not clean huge pile of data that comes from millions of sources that no human as vetted and that no one in the team using the data knows about. The same applies to large images dataset, e.g. Laon 400m that also comes from CommonCrawl and is not curated.