4 ms·

From the article: "CLIP is trained on hundreds of millions of images and their associated captions..." Does anyone have any insight as to which images were tr

From the article: "CLIP is trained on hundreds of millions of images and their associated captions..." Does anyone have any insight as to which images were trained on? Was it all open-domain stuff? And if not were the original authors of those images made aware their work was being use to train an AI that would likely put them out of work? Were they compensated appropriately?

SleekEagle 5y ago
As far as I know, OpenAI has not made this dataset publicly available. IIRC the dataset is images scraped from instagram and their corresponding captions. Check out the CLIP paper for more details: https://arxiv.org/abs/2103.00020 https://arxiv.org/abs/2103.00020 Theoretically, you could build a web-scraping tool to do something like this, but even storing that data would take an absolutely insane amount of storage. I would assume OpenAI has some deal with Meta to make the creation of datasets like this easier.
- mokchira 5y ago
  Thanks for the link. I hope they do make the data set publicly available at some point so that the artists whose work helped train this can know. I think, while it is absolutely impressive on a technical level what the OpenAI team has been able to do, it is also important to consider what damage it will do to artists and their livelihood. Many professional artists stake their career on one unique style of art that they have honed and developed over many years. It's this unique style that clients generally pay for, and that now faces a very real threat of being stolen from them by a technology that frankly no human can hope to compete with. Without artist compensation, this can only lead to artists terminating their careers early once the AI has co-opted all work from them. Or future artists never beginning their careers in the first place. This is a net loss for humanity, as it will deprive us of works and styles of art that have yet to be imagined. I'm not saying AI like this needs to go away. There is no putting that genie back in the bottle, of course. But it needs to be something that artists opt into. If someone's style is worth it for OpenAI to train on, then that style obviously should have a price tag. And it ought to be up to the artist whether they want to sell or not. Anything short of that is theft in my eyes.
  - manquer 5y ago
    Very unlikely they will share even if they really wished to, for one they don't want competition to have the same data, for another it will just expose them to lawsuits, some images are bound to be problematic and would slip their filters. Openai just uses the word Open in their name, they are commercial company like any other, they are a business first.
    - mokchira 5y ago
      To be honest I don't see how this does not lead to lawsuits anyway. Its only a matter of time before the AI spits out something that is a bit too close to an artist's actual work. Withholding their data, which undoubtedly contains copyrighted material if they were just blindly scraping, will only forestall things. The earlier they start being transparent the better it will be for them in the long run.
      - manquer 5y ago
        Lawsuits are cost of doing business in the U.S. They don't need to increase potential surface and invite them though. Knowing that there is potentially copyright material does not give you standing to sue them, unless you can show reasonable cause that they could have used your copyrighted content no court will take it to discovery to prove it conclusively, your case will be thrown out for lack of standing. However if they share their data set, then you can show that they actually took it and have standing to sue them. Making any potential case more complex and expensive.
- jazzyjackson 5y ago
  Whoa, I guess that court precedent about legalizing web scraping has bigger ramifications for AI than for humans, I can't imagine Meta is very happy with having provided the training set for Microsoft's AI research (via OpenAI)
  - kingcharles 5y ago
    There was a note somewhere saying they had paid to license all the images they used.

SleekEagle 5y ago

As far as I know, OpenAI has not made this dataset publicly available. IIRC the dataset is images scraped from instagram and their corresponding captions. Check out the CLIP paper for more details: https://arxiv.org/abs/2103.00020 https://arxiv.org/abs/2103.00020 Theoretically, you could build a web-scraping tool to do something like this, but even storing that data would take an absolutely insane amount of storage. I would assume OpenAI has some deal with Meta to make the creation of datasets like this easier.

mokchira 5y ago
Thanks for the link. I hope they do make the data set publicly available at some point so that the artists whose work helped train this can know. I think, while it is absolutely impressive on a technical level what the OpenAI team has been able to do, it is also important to consider what damage it will do to artists and their livelihood. Many professional artists stake their career on one unique style of art that they have honed and developed over many years. It's this unique style that clients generally pay for, and that now faces a very real threat of being stolen from them by a technology that frankly no human can hope to compete with. Without artist compensation, this can only lead to artists terminating their careers early once the AI has co-opted all work from them. Or future artists never beginning their careers in the first place. This is a net loss for humanity, as it will deprive us of works and styles of art that have yet to be imagined. I'm not saying AI like this needs to go away. There is no putting that genie back in the bottle, of course. But it needs to be something that artists opt into. If someone's style is worth it for OpenAI to train on, then that style obviously should have a price tag. And it ought to be up to the artist whether they want to sell or not. Anything short of that is theft in my eyes.
- manquer 5y ago
  Very unlikely they will share even if they really wished to, for one they don't want competition to have the same data, for another it will just expose them to lawsuits, some images are bound to be problematic and would slip their filters. Openai just uses the word Open in their name, they are commercial company like any other, they are a business first.
  - mokchira 5y ago
    To be honest I don't see how this does not lead to lawsuits anyway. Its only a matter of time before the AI spits out something that is a bit too close to an artist's actual work. Withholding their data, which undoubtedly contains copyrighted material if they were just blindly scraping, will only forestall things. The earlier they start being transparent the better it will be for them in the long run.
    - manquer 5y ago
      Lawsuits are cost of doing business in the U.S. They don't need to increase potential surface and invite them though. Knowing that there is potentially copyright material does not give you standing to sue them, unless you can show reasonable cause that they could have used your copyrighted content no court will take it to discovery to prove it conclusively, your case will be thrown out for lack of standing. However if they share their data set, then you can show that they actually took it and have standing to sue them. Making any potential case more complex and expensive.
jazzyjackson 5y ago
Whoa, I guess that court precedent about legalizing web scraping has bigger ramifications for AI than for humans, I can't imagine Meta is very happy with having provided the training set for Microsoft's AI research (via OpenAI)
- kingcharles 5y ago
  There was a note somewhere saying they had paid to license all the images they used.