4 ms·
How was the data collected in the first place? Why was the data considered useful for training if no one had evaluated it?
by sp332 3y ago
How was the data collected in the first place? Why was the data considered useful for training if no one had evaluated it?
- orbital-decay 3y agoIt's useful because it includes most of the web. In enormous datasets like this, introspection and filtering is done with statistical/ML techniques. This was done with both LAION (obviously not enough to filter CSAM out) and the models trained on it - statistical analysis, 3rd party model classification and captioning, aesthetic scores, perceptual hashes etc. Only the filtered part of the dataset makes it into the actual training set, as LAION has notoriously poor quality.
- PeterisP 3y agoThe point of various unsupervised and 'self-supervised' training methods is the ability to use unlabeled data, which enables you to use much more data than it's feasible for people to evaluate. Some crude automated filtering (e.g. deduplication) is still helpful, but removing the bottleneck of having a person look at every single item can enable you to use thousand or million times larger datasets, which is what's required to give models a 'general understanding' of a wide domain, by taking an (automated) look at literally everything that is available.