4 ms·

The paper shows that Stable Diffusion and Google's Imagen regenerate individual images from their training sets. They show it is very rare, but can be found rel

The paper shows that Stable Diffusion and Google's Imagen regenerate individual images from their training sets. They show it is very rare, but can be found reliably.

pavlov 4y ago
Seems relevant to the Getty Images lawsuit against Stable Diffusion.
- sdenton4 4y ago
  Eh... The mp3 decoder can generate copyrighted music if you feed it the right inputs... Likewise, in this work they prime the pump by using exact training prompts of highly duplicated training images. And then you have to generate 500 images from that prompt to find 10 duplications. You've really gotta want to find the duplicates, which indicates that these are going to be extremely rare in practice, and even more rare once the training data is hardened against the attack by deduplication.
  - BaseballPhysics 4y ago
    > Eh... The mp3 decoder can generate copyrighted music if you feed it the right inputs... Good analogy! An MP3 decoder takes an input and produces an output. If the output is copyrighted material, it's well understood that the inputs is simply a transformed version of that same copyrighted material and is similarly copyrighted. The SD model is very much analogous. The prompt causes the algorithm to extract some output from the input model. If the output is copyrighted material then similarly the input model must carry a transformed version of that same copyrighted material and is therefore also subject to copyright. Right? By the way, I pose this, but I highly doubt this is actually how the courts will rule. I think they'll find the model itself is fine, that the training is subject to a fair use defense, but that the outputs may be subject to copyright if there's substantial similarly to an existing work in the training set.
- PartiallyTyped 4y ago
  The probability that synthesis of similar enough training samples given that the dataset does not contain duplicates is astronomically small, in this work they purposefully manipulated the dataset, used specific prompts and gave all sorts of advantages to an adversary, and created many images per prompt to be able to find such cases.
- roenxi 4y ago
  No doubt, but how relevant? If you could somehow go through my brain you'd also find the occasional piece of art and literature - I've memorised a few good poems and songs. And I can recognise certain paintings on sight which would be difficult if they weren't accurately encoded in my mind somewhere. The fact that I have memorised them doesn't mean that I'm violating anyone's copyright if I attempted to compose poems and songs.
  - haswell 4y ago
    The question is, will the legal system see this software as equivalent to your brain? Anthropomorphizing this software seems problematic.
  - freilanzer 4y ago
    I can't load your brain into RAM with a Python library yet, though.
    - roenxi 4y ago
      Absolutely, but that is what makes the relevancy of the fact interesting. If you could load my brain into Python, would it then become copyright infringement, despite the content being logically unchanged? Memorising a picture using a fleshy model is fine, so the raw fact that art has been found in a black box model here isn't necessarily relevant. Might be. Might not be.
      - haswell 4y ago
        I understand what you’re saying, but if you could load Python into your brain, I suspect the laws would quickly change to reflect that new reality. The biological/evolutionary limits of humans are core assumptions of current laws, and I’d argue that the operating environment has changed enough to make those assumptions outdated. > Memorising a picture using a fleshy model is fine I imagine the fleshy model is fine not because it’s fleshy, but because it’s the model that people were targeting when writing current laws. Even if no memorization occurred, there are still big questions about why such a model should be treated like anything other than just another computer program from a legal perspective.
amelius 4y ago
What do you mean by the last sentence?
- nickthegreek 4y ago
  I imagine reliability means that with the exact prompt, seed, cfg scale, model checkpoint, blah blah and on the same hardware, they can continue to get an image that they consider close enough to the original.

The paper shows that Stable Diffusion and Google's Imagen regenerate individual images from their training sets. They show it is very rare, but can be found reliably.

pavlov 4y ago

Seems relevant to the Getty Images lawsuit against Stable Diffusion.

sdenton4 4y ago
Eh... The mp3 decoder can generate copyrighted music if you feed it the right inputs... Likewise, in this work they prime the pump by using exact training prompts of highly duplicated training images. And then you have to generate 500 images from that prompt to find 10 duplications. You've really gotta want to find the duplicates, which indicates that these are going to be extremely rare in practice, and even more rare once the training data is hardened against the attack by deduplication.
- BaseballPhysics 4y ago
  > Eh... The mp3 decoder can generate copyrighted music if you feed it the right inputs... Good analogy! An MP3 decoder takes an input and produces an output. If the output is copyrighted material, it's well understood that the inputs is simply a transformed version of that same copyrighted material and is similarly copyrighted. The SD model is very much analogous. The prompt causes the algorithm to extract some output from the input model. If the output is copyrighted material then similarly the input model must carry a transformed version of that same copyrighted material and is therefore also subject to copyright. Right? By the way, I pose this, but I highly doubt this is actually how the courts will rule. I think they'll find the model itself is fine, that the training is subject to a fair use defense, but that the outputs may be subject to copyright if there's substantial similarly to an existing work in the training set.
PartiallyTyped 4y ago
The probability that synthesis of similar enough training samples given that the dataset does not contain duplicates is astronomically small, in this work they purposefully manipulated the dataset, used specific prompts and gave all sorts of advantages to an adversary, and created many images per prompt to be able to find such cases.
roenxi 4y ago
No doubt, but how relevant? If you could somehow go through my brain you'd also find the occasional piece of art and literature - I've memorised a few good poems and songs. And I can recognise certain paintings on sight which would be difficult if they weren't accurately encoded in my mind somewhere. The fact that I have memorised them doesn't mean that I'm violating anyone's copyright if I attempted to compose poems and songs.
- haswell 4y ago
  The question is, will the legal system see this software as equivalent to your brain? Anthropomorphizing this software seems problematic.
- freilanzer 4y ago
  I can't load your brain into RAM with a Python library yet, though.
  - roenxi 4y ago
    Absolutely, but that is what makes the relevancy of the fact interesting. If you could load my brain into Python, would it then become copyright infringement, despite the content being logically unchanged? Memorising a picture using a fleshy model is fine, so the raw fact that art has been found in a black box model here isn't necessarily relevant. Might be. Might not be.
    - haswell 4y ago
      I understand what you’re saying, but if you could load Python into your brain, I suspect the laws would quickly change to reflect that new reality. The biological/evolutionary limits of humans are core assumptions of current laws, and I’d argue that the operating environment has changed enough to make those assumptions outdated. > Memorising a picture using a fleshy model is fine I imagine the fleshy model is fine not because it’s fleshy, but because it’s the model that people were targeting when writing current laws. Even if no memorization occurred, there are still big questions about why such a model should be treated like anything other than just another computer program from a legal perspective.

amelius 4y ago

What do you mean by the last sentence?

nickthegreek 4y ago
I imagine reliability means that with the exact prompt, seed, cfg scale, model checkpoint, blah blah and on the same hardware, they can continue to get an image that they consider close enough to the original.