3 ms·
11.6M images with 37.6M captions isn't exactly small scale. Isn't it within an order of magnitude of DALL-E? I seem to recall something like ~47M images, but my
by sillysaurusx 6y ago
11.6M images with 37.6M captions isn't exactly small scale. Isn't it within an order of magnitude of DALL-E? I seem to recall something like ~47M images, but my memory is often off.
- Imnimo 6y agoI think the DALL-E dataset is 250M images. I think a possibly bigger issue is that the captions in this dataset are spread across many languages, which isn't ideal for doing something like DALL-E.
- gwern 6y agoDALL-E is 250m: https://arxiv.org/pdf/2102.12092.pdf#page=14 https://arxiv.org/pdf/2102.12092.pdf#page=14 (so 20x larger). CLIP (https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf https://cdn.openai.com/papers/Learning_Transferable_Visual_M...) is 400m (34x larger). And ALIGN (https://arxiv.org/abs/2102.05918#google https://arxiv.org/abs/2102.05918#google) is 1800m (155x). (For 47m maybe you're thinking of a cleaned subset from YFCC100M or something that one of them used, but they all use total images an order or two larger than Wit.)