4 ms·

Kudos to the effort. But where could this dataset be used specifically?

• gwern 6y ago
  You would use this to train your own CLIP/ALIGN model, most obviously, to give you image classifiers on steroids. DALL-E/M6 would also be trainable with this. (All at a smaller scale, but this is a good starting point.)
  • sillysaurusx 6y ago
    11.6M images with 37.6M captions isn't exactly small scale. Isn't it within an order of magnitude of DALL-E? I seem to recall something like ~47M images, but my memory is often off.
    • Imnimo 6y ago
      I think the DALL-E dataset is 250M images. I think a possibly bigger issue is that the captions in this dataset are spread across many languages, which isn't ideal for doing something like DALL-E.
    • gwern 6y ago
      DALL-E is 250m: https://arxiv.org/pdf/2102.12092.pdf#page=14 https://arxiv.org/pdf/2102.12092.pdf#page=14 (so 20x larger). CLIP (https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdf https://cdn.openai.com/papers/Learning_Transferable_Visual_M...) is 400m (34x larger). And ALIGN (https://arxiv.org/abs/2102.05918#google https://arxiv.org/abs/2102.05918#google) is 1800m (155x). (For 47m maybe you're thinking of a cleaned subset from YFCC100M or something that one of them used, but they all use total images an order or two larger than Wit.)
  • akhilcacharya 6y ago
    I'm not familiar with ALIGN, what is this?