3 ms·

Uploading works by real human artists gives you a batch of results that resemble a reference board (mood board, inspiration board, etc) the artist could have be

by fwlr 4y ago

Uploading works by real human artists gives you a batch of results that resemble a reference board (mood board, inspiration board, etc) the artist could have been looking at while creating their original work of art. Obviously it’s not the actual reference board, the only way to get that is to ask the artist yourself, but it sure looks like what you’d expect their reference board to look like. This site is grift, of course, and I doubt its creators expect it to sway anyone who knows it’s just doing a nearest neighbor search of image embedding vectors. But it’s oddly humanizing to see that the AI uses reference boards too.

kmeisthax 4y ago
The AI does NOT build or use reference boards for specific prompts. The only reference it has is the prompt itself, which gets distilled down into a list of 512 numbers, each one of which the AI associates with a particular image feature (or set of features). The only reference material it has is the training set. Most images are not actually retained in the model; but certain statistically significant or repeated images will be. So if the model is regurgitating a training set image, this will alert you to that fact. But you are correct that it is not strictly speaking "attribution". You still need a human to say whether or not the images are just "in the same style" or close enough to actually be a straight copy. This is also only talking about the "type prompt, insert bacon" kind of AI art. There are plenty of people who are feeding other people's work into an AI as a sort of automated tracing tool, and this won't catch them if they're not tracing images that were in the training set. Unfortunately these are also the most egregious and awful abuses of AI art generators.
- fwlr 4y ago
  In a literal sense, no, it does not use reference boards. I was being glib, perhaps too glib. A less objectionable rephrasing might be “the AI is composing a bunch of visual features together into a coherent image; it’s cool that this tool can show you images in the training set where the AI might have learned those visual features from.” It may still be inaccurate to say “reference boards” at all, because the temporality is reversed: human artist has a reference board, then does some black box process in their brain to draw inspiration from them, then outputs the final art piece. The AI draws on all the training data to produce the final image and it’s only by analyzing the final image that you can reconstruct which images in the training set were important. Mostly I was just tickled by the fact that you can now get something that looks like a reference board for the generated image. There are some parallels to the human process, but maybe there are not enough parallels or the parallels don’t run deep enough to make it sensible to call this “the AI is using a reference board”.
- astrange 4y ago
  > Most images are not actually retained in the model; but certain statistically significant or repeated images will be. This isn't true. The distribution of images seen by the model is not the distribution of images in LAION; the base models were trained with deduplication (or tried to) and custom ones could've been forcibly overfitted. So "repeated" as seen in clip-front doesn't prove anything.
- minimaxir 4y ago
  > The AI does NOT build or use reference boards for specific prompts. The only reference it has is the prompt itself, which gets distilled down into a list of 512 numbers, each one of which the AI associates with a particular image feature (or set of features). Minor technical clarification: For SD 1.X, CLIPText encodes a prompt and passes a (77, 768) [edit: up to 77] matrix to the core UNet. For SD 2.X, OpenCLIP passes a (77, 1024) matrix. The black-box interaction between the text-encoded matrices and the UNet is how Stable Diffusion "learns" text associations.
  - kmeisthax 4y ago
    Huh. I thought those got projected into the (512) shared CLIP space before getting passed to the conditional blocks.
    - minimaxir 4y ago
      Only the text portion is included, otherwise that would be big. Also, I am slightly wrong in that the first dimension will many not always be 77, since apparently there is no padding in the tokenizer. Test notebook here: https://colab.research.google.com/drive/192PDIbc2XiI1HgJQSdNpbOlbZh49xygU?usp=sharing https://colab.research.google.com/drive/192PDIbc2XiI1HgJQSdN...
    - HammadB 4y ago
      Nope, this is why you cannot use images as prompts without some workarounds! SD doesn’t use the shared CLIP space but the text encoded before projection
- gwern 4y ago
  > The AI does NOT build or use reference boards for specific prompts. Interestingly, even if it did build reference boards, that doesn't mean that the generated image would 'copy' or 'look like' the references all that much. We know this because there are diffusion generative models which do exactly that: they do retrievals first, and include the 'examples' along with the prompt before generating. Here is a visualization: https://arxiv.org/pdf/2204.02849.pdf#page=19 https://arxiv.org/pdf/2204.02849.pdf#page=19 (the generated sample on the left is clearly semantically correct and high-quality but also very different from all the exemplars it was generated using).

by fwlr 4y ago

kmeisthax 4y ago

The AI does NOT build or use reference boards for specific prompts. The only reference it has is the prompt itself, which gets distilled down into a list of 512 numbers, each one of which the AI associates with a particular image feature (or set of features). The only reference material it has is the training set. Most images are not actually retained in the model; but certain statistically significant or repeated images will be. So if the model is regurgitating a training set image, this will alert you to that fact. But you are correct that it is not strictly speaking "attribution". You still need a human to say whether or not the images are just "in the same style" or close enough to actually be a straight copy. This is also only talking about the "type prompt, insert bacon" kind of AI art. There are plenty of people who are feeding other people's work into an AI as a sort of automated tracing tool, and this won't catch them if they're not tracing images that were in the training set. Unfortunately these are also the most egregious and awful abuses of AI art generators.

fwlr 4y ago
In a literal sense, no, it does not use reference boards. I was being glib, perhaps too glib. A less objectionable rephrasing might be “the AI is composing a bunch of visual features together into a coherent image; it’s cool that this tool can show you images in the training set where the AI might have learned those visual features from.” It may still be inaccurate to say “reference boards” at all, because the temporality is reversed: human artist has a reference board, then does some black box process in their brain to draw inspiration from them, then outputs the final art piece. The AI draws on all the training data to produce the final image and it’s only by analyzing the final image that you can reconstruct which images in the training set were important. Mostly I was just tickled by the fact that you can now get something that looks like a reference board for the generated image. There are some parallels to the human process, but maybe there are not enough parallels or the parallels don’t run deep enough to make it sensible to call this “the AI is using a reference board”.
astrange 4y ago
> Most images are not actually retained in the model; but certain statistically significant or repeated images will be. This isn't true. The distribution of images seen by the model is not the distribution of images in LAION; the base models were trained with deduplication (or tried to) and custom ones could've been forcibly overfitted. So "repeated" as seen in clip-front doesn't prove anything.
minimaxir 4y ago
> The AI does NOT build or use reference boards for specific prompts. The only reference it has is the prompt itself, which gets distilled down into a list of 512 numbers, each one of which the AI associates with a particular image feature (or set of features). Minor technical clarification: For SD 1.X, CLIPText encodes a prompt and passes a (77, 768) [edit: up to 77] matrix to the core UNet. For SD 2.X, OpenCLIP passes a (77, 1024) matrix. The black-box interaction between the text-encoded matrices and the UNet is how Stable Diffusion "learns" text associations.
- kmeisthax 4y ago
  Huh. I thought those got projected into the (512) shared CLIP space before getting passed to the conditional blocks.
  - minimaxir 4y ago
    Only the text portion is included, otherwise that would be big. Also, I am slightly wrong in that the first dimension will many not always be 77, since apparently there is no padding in the tokenizer. Test notebook here: https://colab.research.google.com/drive/192PDIbc2XiI1HgJQSdNpbOlbZh49xygU?usp=sharing https://colab.research.google.com/drive/192PDIbc2XiI1HgJQSdN...
  - HammadB 4y ago
    Nope, this is why you cannot use images as prompts without some workarounds! SD doesn’t use the shared CLIP space but the text encoded before projection
gwern 4y ago
> The AI does NOT build or use reference boards for specific prompts. Interestingly, even if it did build reference boards, that doesn't mean that the generated image would 'copy' or 'look like' the references all that much. We know this because there are diffusion generative models which do exactly that: they do retrievals first, and include the 'examples' along with the prompt before generating. Here is a visualization: https://arxiv.org/pdf/2204.02849.pdf#page=19 https://arxiv.org/pdf/2204.02849.pdf#page=19 (the generated sample on the left is clearly semantically correct and high-quality but also very different from all the exemplars it was generated using).