3 ms·
The article summarizes how LAION was assembled: > This dataset was built by taking a snapshot of the Common Crawl repository, downloading images referenced in
by duskwuff 3y ago
The article summarizes how LAION was assembled:
> This dataset was built by taking a snapshot of the Common Crawl repository, downloading images referenced in the HTML, reading the “alt” attributes of the images and using CLIP interrogation to discard images that did not sufficiently match the captions.
In other words, it was based a random subset of the entire public Internet and there was no human in the loop -- and it's not clear how there could ever be, at the scale this project was operating. (LAION-5B contains 5.8B images.) If anything, it's surprising that the Stanford researchers only found a few thousand CSAM images in the data set.