3 ms·

So wait they only found 109 matches after generating 175 milion images using the prompt from the most duplicated samples from the dataset and SD v1.4? Also almo

by GaggiX 4y ago

So wait they only found 109 matches after generating 175 milion images using the prompt from the most duplicated samples from the dataset and SD v1.4? Also almost all of them have more than 300 copies in the dataset, so with a model with the same size and trained on a dedup dataset like SD 2.0/2.1 there will be almost no matches, even after generating 175 mln images and knowing the prompts used in the dataset. Finally Google at el need to explain how an attacker that want to extract images from a trained model somehow has the prompts for the top X duplicated images in the dataset but not the images themselves, and thus will going to spent an incredible amount of money to generate something like 175 mln samples and test them together to find the matches. Edit: I also want to add that google seems to try really hard to show themselves as the good guys by not releasing its models because it's not safe enough, but in this paper they used an incredible amount of computation and show me otherwise.

babel_ 4y ago
The argument here is more that this attack serves to just "find" these, and while it uses significant compute to do so, it's also not using many of the complex and more efficient extraction approaches that have been used on GANs and such in prior research. They're using a simple method, and exploring what it does; most of their 175 million images comes from effectively "retrying" each prompt 500 times, which could easily be cut down when others are trying attacks, since they're usually much more targetted than this at what they're trying to extract. Also, "almost no matches" is still problematic, because they still occur, and since they don't seem prevalent, they're unlikely to have been accounted for in any other way (SD 2 being finally based on a deduplicated dataset). So, in a sense, they're "still in there", it's just harder to find by a casual or simple approach as above. Again, one of the central themes is that diffusion seems to be less resistant to attacks than, say, GANs, and that may mean more efficient and complex extraction attacks may translate over even more effectively.
- GaggiX 4y ago
  >it's also not using many of the complex and more efficient extraction approaches that have been used on GANs and such in prior research Some links? >most of their 175 million images comes from effectively "retrying" each prompt 500 times Prompts from the most duplicated samples in the dataset, a really important aspect if you actually want to used this method in the wild, this is also one of the reasons why I said that this attack seems so implausible. >they're usually much more targetted than this Even if you target some images you would still need an absurd amount of luck if with the most duplicated sample you only get 109, we can be generous and think that with the whole dataset will have something like 200 matches, the probability of finding an image with a direct attack is still less than a million (even if you know the prompt) and we're not talking about a model trained on a deduped dataset.
  - lelandfe 4y ago
    Is it implausible if they've done it in this paper? This paper seems to answer the question of, "can SD, even just in theory, produce copyright-infringing work?" with "yes, it can." For other images that are a product of thousands - if not millions - of source images, it becomes murkier.
    - GaggiX 4y ago
      >Is it implausible if they've done it in this paper? Extracting images in the wild yes, the authors of the paper have access to the dataset, they could sort prompts and images based on their presence in the dataset and they have an incredible amount of computation to do so, generating 175 mln using a diffusion model is an extremely resource-intensive task.
      - lelandfe 4y ago
        I believe everyone has access to their dataset, no? https://laion.ai/blog/laion-5b/ https://laion.ai/blog/laion-5b/ Anyway, I don't think the point of this was to indicate that people can stumble on these incidents, but rather that it is possible. It's hard to see how this won't affect the ongoing suit.
        GaggiX 4y ago
        In the case of Stable Diffusion yes the dataset publicly available but these types of attacks would make much more sense if the attacker wants to extract some private data.
        lelandfe 4y ago
        Ah, I understand what you're getting at. True.
dang 4y ago
(I've merged the extra bit from https://news.ycombinator.com/item?id=34612005 https://news.ycombinator.com/item?id=34612005 into this comment so people can read everything you wrote. If you don't like that, let me know and I'll reopen the comment so you can change it. I will move the replies here as well. But please see https://news.ycombinator.com/item?id=34614429 https://news.ycombinator.com/item?id=34614429. This sort of surgery is time-consuming!)

by GaggiX 4y ago

babel_ 4y ago

The argument here is more that this attack serves to just "find" these, and while it uses significant compute to do so, it's also not using many of the complex and more efficient extraction approaches that have been used on GANs and such in prior research. They're using a simple method, and exploring what it does; most of their 175 million images comes from effectively "retrying" each prompt 500 times, which could easily be cut down when others are trying attacks, since they're usually much more targetted than this at what they're trying to extract. Also, "almost no matches" is still problematic, because they still occur, and since they don't seem prevalent, they're unlikely to have been accounted for in any other way (SD 2 being finally based on a deduplicated dataset). So, in a sense, they're "still in there", it's just harder to find by a casual or simple approach as above. Again, one of the central themes is that diffusion seems to be less resistant to attacks than, say, GANs, and that may mean more efficient and complex extraction attacks may translate over even more effectively.

GaggiX 4y ago
>it's also not using many of the complex and more efficient extraction approaches that have been used on GANs and such in prior research Some links? >most of their 175 million images comes from effectively "retrying" each prompt 500 times Prompts from the most duplicated samples in the dataset, a really important aspect if you actually want to used this method in the wild, this is also one of the reasons why I said that this attack seems so implausible. >they're usually much more targetted than this Even if you target some images you would still need an absurd amount of luck if with the most duplicated sample you only get 109, we can be generous and think that with the whole dataset will have something like 200 matches, the probability of finding an image with a direct attack is still less than a million (even if you know the prompt) and we're not talking about a model trained on a deduped dataset.
- lelandfe 4y ago
  Is it implausible if they've done it in this paper? This paper seems to answer the question of, "can SD, even just in theory, produce copyright-infringing work?" with "yes, it can." For other images that are a product of thousands - if not millions - of source images, it becomes murkier.
  - GaggiX 4y ago
    >Is it implausible if they've done it in this paper? Extracting images in the wild yes, the authors of the paper have access to the dataset, they could sort prompts and images based on their presence in the dataset and they have an incredible amount of computation to do so, generating 175 mln using a diffusion model is an extremely resource-intensive task.
    - lelandfe 4y ago
      I believe everyone has access to their dataset, no? https://laion.ai/blog/laion-5b/ https://laion.ai/blog/laion-5b/ Anyway, I don't think the point of this was to indicate that people can stumble on these incidents, but rather that it is possible. It's hard to see how this won't affect the ongoing suit.
      - GaggiX 4y ago
        In the case of Stable Diffusion yes the dataset publicly available but these types of attacks would make much more sense if the attacker wants to extract some private data.
        lelandfe 4y ago
        Ah, I understand what you're getting at. True.

dang 4y ago

(I've merged the extra bit from https://news.ycombinator.com/item?id=34612005 https://news.ycombinator.com/item?id=34612005 into this comment so people can read everything you wrote. If you don't like that, let me know and I'll reopen the comment so you can change it. I will move the replies here as well. But please see https://news.ycombinator.com/item?id=34614429 https://news.ycombinator.com/item?id=34614429. This sort of surgery is time-consuming!)