5 ms·
So the core of the dataset is image _URLs_ and text captions. 1. From a reproducibility perspective, isn't this kinda brittle in that even without malicious in
by abeppu 4y ago
So the core of the dataset is image _URLs_ and text captions.
1. From a reproducibility perspective, isn't this kinda brittle in that even without malicious intent, some of those images will no longer be available when other researchers attempt to download them?
2. From a resilience perspective, if your site has some of the images in the dataset, could you swap in another image with the correct dimensions. Could you poison or skew the model in any interesting ways?
- alsodumb 4y agoThe key is the scale of the dataset. Both the points you mention become irrelevant for a large dataset because 1) The chance that a significant percentage of the images become unavailable is low. Also, training on such a big dataset means your model generalizes well and is usually robust. 2) Again, you would need to inject adversarial/malicious images to a significant number of those links in the dataset for it to have actual impact on trained model. Again, unlikely.
- abeppu 4y agoFor point 1 ... it depends on the timescale. In the fullness of time, surely a significant portion of images will be unavailable. From the perspective of allowing other researchers to work from the "same" baseline "today", this is likely good enough. In a generation from now, if someone wants to reproduce results from some landmark model trained against this dataset, we'd have problems. In other fields where people publish or share their datasets, would this be considered sufficient? For point 2, I think it's possible that for some narrow topics, some domains have a significant share of images. I think these can affect the model, which is in part why they give special attention to watermarking. Suppose instead of merely watermarking images, for every image on my large collegiate track and field website I make sure someone is wearing a garment with a visible Nike swoosh. Can I skew the model towards associating Nike with the sport? I think this kind of thing may be achievable for niche areas.
- deleted 4y ago[deleted]
- version_five 4y agoImagenet (arguably the most used image dataset of the last 10 years) is the same, it's a list of URLs with full archives of the downloaded images available under some conditions.
- tbalsam 4y agoFair enough, but Imagenet is sort of a nightmare right now. I get it's a crowd funded and sourced effort, but hopefully at some point some brave soul(s) will step up to archive the data as-is in a very reproducible kind of way. :D :))))
- 8n4vidtmkvmk 4y agoisn't this a perfect use case for torrents? might be too expensive to host for 1 nonprofit company, but collectively.. there might be a few dozen people willing to host it. don't know about the legality too.. does laion check all the licenses? or they skirt that by using urls?
- whiplash451 4y agoAnd from the storing perspective? The full image dataset weighs dozens of PB. How convenient is that to share?
- astrange 4y agoSince artists already appear to believe LAION is “stolen content”, actually downloading everything wouldn’t help the case that it’s fine.