3 ms·

Thanks for your thoughtful comment, akozak! Great idea to add model provenance. I'm not sure if Dalle-2 embeds any metadata, but openAI also has this CLIP mod

Thanks for your thoughtful comment, akozak! Great idea to add model provenance. I'm not sure if Dalle-2 embeds any metadata, but openAI also has this CLIP model https://openai.com/blog/clip/ https://openai.com/blog/clip/ that lets you get embeddings. So we will be using this for our search functionality. It works amazingly well. Stay tuned for the search functionality. It is super powerful for prompt engineering. Thanks for pointing out the copyright status. Most of the images in Discover are crawled on the web, and you can see all the sources under each image. However for the images that people post on our platform, we should definitely add the clarification for copyright status.

akozak 4y ago
For more context about my 'metadata' comment, there is the PNG metadata (should be easy to check) but there is also the watermark in the bottom right. Here is a reddit thread (one of many) investigating the watermark, seems to conclude its innocuous: https://www.reddit.com/r/dalle2/comments/uaq7bf/discovery_dalle_2_watermarks_contain_pixels_that/ https://www.reddit.com/r/dalle2/comments/uaq7bf/discovery_da...
- cocokechun 4y ago
  Ah got it! Thanks for sharing the reddit thread. Very helpful :)
- qiaoshun8888 4y ago
  Thanks for the link. There is another thread talking about similar topic https://www.reddit.com/r/dalle2/comments/uro3js/i_compared_the_dalle_2_watermarks_of_2_images/ https://www.reddit.com/r/dalle2/comments/uro3js/i_compared_t... "b) Exploration of the bolded part of OpenAI's comment "Each generated image includes a signature in the lower right corner, with the goal of indicating when DALL·E 2 helped generate a certain image." (source)." (source link: https://github.com/openai/dalle-2-preview/blob/main/system-card.md#signature-and-image-provenance https://github.com/openai/dalle-2-preview/blob/main/system-c...) I feel the DALL-E 2 watermark signature could be a seed or something.
krisoft 4y ago
> I'm not sure if Dalle-2 embeds any metadata, but openAI also has this CLIP model https://openai.com/blog/clip/ https://openai.com/blog/clip/ that lets you get embeddings. Hah! Same word, two totally different meaning. Metadata embedding: store extra information in non-visible fields in the file. For example some phones save the gps location where the picture was made in the image. CLIP embedding: a vector which describes the semantic content of an image. It is the output of an AI model which maps the image into a semantic space (aka embeds the image in said space.) This vectorial description then typically can be used for image content classification or textual label generation.

akozak 4y ago

For more context about my 'metadata' comment, there is the PNG metadata (should be easy to check) but there is also the watermark in the bottom right. Here is a reddit thread (one of many) investigating the watermark, seems to conclude its innocuous: https://www.reddit.com/r/dalle2/comments/uaq7bf/discovery_dalle_2_watermarks_contain_pixels_that/ https://www.reddit.com/r/dalle2/comments/uaq7bf/discovery_da...

cocokechun 4y ago
Ah got it! Thanks for sharing the reddit thread. Very helpful :)
qiaoshun8888 4y ago
Thanks for the link. There is another thread talking about similar topic https://www.reddit.com/r/dalle2/comments/uro3js/i_compared_the_dalle_2_watermarks_of_2_images/ https://www.reddit.com/r/dalle2/comments/uro3js/i_compared_t... "b) Exploration of the bolded part of OpenAI's comment "Each generated image includes a signature in the lower right corner, with the goal of indicating when DALL·E 2 helped generate a certain image." (source)." (source link: https://github.com/openai/dalle-2-preview/blob/main/system-card.md#signature-and-image-provenance https://github.com/openai/dalle-2-preview/blob/main/system-c...) I feel the DALL-E 2 watermark signature could be a seed or something.

krisoft 4y ago

> I'm not sure if Dalle-2 embeds any metadata, but openAI also has this CLIP model https://openai.com/blog/clip/ https://openai.com/blog/clip/ that lets you get embeddings. Hah! Same word, two totally different meaning. Metadata embedding: store extra information in non-visible fields in the file. For example some phones save the gps location where the picture was made in the image. CLIP embedding: a vector which describes the semantic content of an image. It is the output of an AI model which maps the image into a semantic space (aka embeds the image in said space.) This vectorial description then typically can be used for image content classification or textual label generation.