5 ms·
Wit: Wikipedia-Based Image Text Dataset
- singularity2001 6y agothat is a pretty smart data augmentation idea
- abhayhegde 6y agoKudos to the effort. But where could this dataset be used specifically?
- gwern 6y agoYou would use this to train your own CLIP/ALIGN model, most obviously, to give you image classifiers on steroids. DALL-E/M6 would also be trainable with this. (All at a smaller scale, but this is a good starting point.)
- sillysaurusx 6y ago11.6M images with 37.6M captions isn't exactly small scale. Isn't it within an order of magnitude of DALL-E? I seem to recall something like ~47M images, but my memory is often off.
- Imnimo 6y agoI think the DALL-E dataset is 250M images. I think a possibly bigger issue is that the captions in this dataset are spread across many languages, which isn't ideal for doing something like DALL-E.
- gwern 6y agoDALL-E is 250m: https://arxiv.org/pdf/2102.12092.pdf#page=14 https://arxiv.org/pdf/2102.12092.pdf#page=14 (so 20x larger). CLIP (https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf https://cdn.openai.com/papers/Learning_Transferable_Visual_M...) is 400m (34x larger). And ALIGN (https://arxiv.org/abs/2102.05918#google https://arxiv.org/abs/2102.05918#google) is 1800m (155x). (For 47m maybe you're thinking of a cleaned subset from YFCC100M or something that one of them used, but they all use total images an order or two larger than Wit.)
- akhilcacharya 6y agoI'm not familiar with ALIGN, what is this?
- sillysaurusx 6y agoPlease stay tuned and we will share the details about how to download WIT dataset. But.. why post it at all if you're not going to give a download link? Arg, such a tease. (This is also technically an "announcement of an announcement" since there's no substance here, aside from the paper at https://arxiv.org/abs/2103.01913 https://arxiv.org/abs/2103.01913.)