4 ms·

Using img2img to "royalty free"-ize an image is clearly copyright infringement, and the output is a clear derivative work of the original. If you described the

by ronsor 4y ago

Using img2img to "royalty free"-ize an image is clearly copyright infringement, and the output is a clear derivative work of the original. If you described the original image (or made CLIP do it) and fed the description to txt2img generation, then you'd probably be fine.

gamegoblin 4y ago
Seems like it's a matter of time (or maybe it's already done and I haven't seen it yet) before you get optimizers that takes an image and produces a seed+prompt that yields a super super conceptually similar image. I think being able to selectively optimize for "magic" random seeds in the diffusion algorithm will be kind of critically important here. Different seeds can produce very different images given the same prompt. What if an optimizer can find a seed+prompt combos that are just as good as cloning as img2img?
- GaggiX 4y ago
  If you want to clone an image (for some reason) just encode it in latent space and retrieve it using a deterministic sampler such as DDIM. Or you can simply right-click and copy. If you instead don't want to clone an image, you can just extract the CLIP image embeddings from it and use them to condition a generative model like Dalle 2, Midjourney or Karlo (open source). The CLIP embeddings extract really well the semantic meaning of an image.
  - gamegoblin 4y ago
    Going image -> vector space -> image feels very much like compression for the purpose of copyright. Like a lower quality JPEG of some image still has the same copyright properties of the original. Something about going image -> prompt -> image feels like it "subverts" this somehow, even if the prompt is hyper-optimized to recreate the original image. Obviously, this is just my feel/impression of it, the real test is how a jury feels about it. The next few years will be really interesting in exposing that this is a really massive gray area.
    - Ensorceled 4y ago
      > Something about going image -> prompt -> image feels like it "subverts" this somehow, That's only because you understand the algorithm in the first example. At a jury trial most, if not all, of the jurors will find both methods equally opaque and, I believe, will treat them as equivalent.
      - joxel 4y ago
        This is a great example of how dumb our legal system is.
      - int_19h 4y ago
        I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. That is something a good lawyer could make a show of, even to the point of having an artist draw a picture from such a description. The vectors are much more opaque because there's no straightforward human equivalent.
        oldgradstudent 4y ago
        > I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. Describing to a human. Describing it to stable Diffusion is very different. If you ask for a starry night, you get van Gogh's starry night, sometime with the original frame. A prompt can easily trigger it to make a derivative work from something in its training set, and it often does. Any competent expert will tell that to the jury and easily demonstrate it again and again.
        Ensorceled 4y ago
        My point is that if there is a prompt that results in a picture that is a nearly identical copy, the average jury member is going to think "yep, that's a copy". Trying to explain how that "isn't really a copy" by explaining AI concepts isn't going to win the day, not when they can SEE the copy.
saurik 4y ago
If you describe an image with sufficient fidelity on the specific expression to another human--someone who has never seen the original photo, mind you--that they are able to produce something that feels "substantially similar" to the original photo, that is still going to be copyright infringement. There is a reason that, if you want to reverse engineer and reimplement some piece of technology, even if you are using humans, you are supposed to have not only the team which doesn't get to see the original device and will implement it based on a description from a second team whose job is to carefully analyze the device to build documentation for that first team... but you then take that documentation and you run it through lawyers who carefully attempt to verify that the description is only of the factual content required for interoperability / behavior and doesn't accidentally include any of the expression inherent in the design of the original product. These examples from this demo video aren't just "image of a pile of gemstones" but stuff like "image of exactly six gems in this specific orientation with this specific color palette and texture and lighting and..."; there's just no way you are going to get a lawyer to sign off on a description as not including any of the expressive elements of the original image.
michalf6 4y ago
So mapping the image to text space would be OK, but mapping the image to the model's latent space would not be OK?
- shagie 4y ago
  Note that there are non-infringing models to use with img2img. If I drew a line sketch of the scene that I wanted and then used img2img to fill it out - that's not infringing. On the other hand, if I was to take an existing image had and then used that to transform it, it is a derivative work. If this was my image created with a camera rather than mspaint, its still a derivative work but I, as the creator of the original, can create derivative works of my own (I do it all the time). The idea that I could use someone else's image and run it through img2img to make it "not a copy" misses the point of copyright and the exclusive right of the copyright holder. https://www.copyright.gov/circs/circ14.pdf https://www.copyright.gov/circs/circ14.pdf > Only the owner of copyright in a work has the right to prepare, or to authorize someone else to create, an adaptation of that work. The owner of a copyright is generally the author or someone who has obtained the exclusive rights from the author. In any case where a copyrighted work is used without the permission of the copyright owner, copyright protection will not extend to any part of the work in which such material has been used unlawfully. The unauthorized adaptation of a work may constitute copyright infringement. --- With starting from text, a description of the Mona Lisa is difficult to get generative art to create without significant effort to get the Mona Lisa (without saying "I want that image"). The question is then "is a description of the Mona Lisa a derivative work" and that is likely sufficiently far away from the original to be "no."

by ronsor 4y ago

gamegoblin 4y ago

Seems like it's a matter of time (or maybe it's already done and I haven't seen it yet) before you get optimizers that takes an image and produces a seed+prompt that yields a super super conceptually similar image. I think being able to selectively optimize for "magic" random seeds in the diffusion algorithm will be kind of critically important here. Different seeds can produce very different images given the same prompt. What if an optimizer can find a seed+prompt combos that are just as good as cloning as img2img?

GaggiX 4y ago
If you want to clone an image (for some reason) just encode it in latent space and retrieve it using a deterministic sampler such as DDIM. Or you can simply right-click and copy. If you instead don't want to clone an image, you can just extract the CLIP image embeddings from it and use them to condition a generative model like Dalle 2, Midjourney or Karlo (open source). The CLIP embeddings extract really well the semantic meaning of an image.
- gamegoblin 4y ago
  Going image -> vector space -> image feels very much like compression for the purpose of copyright. Like a lower quality JPEG of some image still has the same copyright properties of the original. Something about going image -> prompt -> image feels like it "subverts" this somehow, even if the prompt is hyper-optimized to recreate the original image. Obviously, this is just my feel/impression of it, the real test is how a jury feels about it. The next few years will be really interesting in exposing that this is a really massive gray area.
  - Ensorceled 4y ago
    > Something about going image -> prompt -> image feels like it "subverts" this somehow, That's only because you understand the algorithm in the first example. At a jury trial most, if not all, of the jurors will find both methods equally opaque and, I believe, will treat them as equivalent.
    - joxel 4y ago
      This is a great example of how dumb our legal system is.
    - int_19h 4y ago
      I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. That is something a good lawyer could make a show of, even to the point of having an artist draw a picture from such a description. The vectors are much more opaque because there's no straightforward human equivalent.
      - oldgradstudent 4y ago
        > I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. Describing to a human. Describing it to stable Diffusion is very different. If you ask for a starry night, you get van Gogh's starry night, sometime with the original frame. A prompt can easily trigger it to make a derivative work from something in its training set, and it often does. Any competent expert will tell that to the jury and easily demonstrate it again and again.
      - Ensorceled 4y ago
        My point is that if there is a prompt that results in a picture that is a nearly identical copy, the average jury member is going to think "yep, that's a copy". Trying to explain how that "isn't really a copy" by explaining AI concepts isn't going to win the day, not when they can SEE the copy.

saurik 4y ago

If you describe an image with sufficient fidelity on the specific expression to another human--someone who has never seen the original photo, mind you--that they are able to produce something that feels "substantially similar" to the original photo, that is still going to be copyright infringement. There is a reason that, if you want to reverse engineer and reimplement some piece of technology, even if you are using humans, you are supposed to have not only the team which doesn't get to see the original device and will implement it based on a description from a second team whose job is to carefully analyze the device to build documentation for that first team... but you then take that documentation and you run it through lawyers who carefully attempt to verify that the description is only of the factual content required for interoperability / behavior and doesn't accidentally include any of the expression inherent in the design of the original product. These examples from this demo video aren't just "image of a pile of gemstones" but stuff like "image of exactly six gems in this specific orientation with this specific color palette and texture and lighting and..."; there's just no way you are going to get a lawyer to sign off on a description as not including any of the expressive elements of the original image.

michalf6 4y ago

So mapping the image to text space would be OK, but mapping the image to the model's latent space would not be OK?

shagie 4y ago
Note that there are non-infringing models to use with img2img. If I drew a line sketch of the scene that I wanted and then used img2img to fill it out - that's not infringing. On the other hand, if I was to take an existing image had and then used that to transform it, it is a derivative work. If this was my image created with a camera rather than mspaint, its still a derivative work but I, as the creator of the original, can create derivative works of my own (I do it all the time). The idea that I could use someone else's image and run it through img2img to make it "not a copy" misses the point of copyright and the exclusive right of the copyright holder. https://www.copyright.gov/circs/circ14.pdf https://www.copyright.gov/circs/circ14.pdf > Only the owner of copyright in a work has the right to prepare, or to authorize someone else to create, an adaptation of that work. The owner of a copyright is generally the author or someone who has obtained the exclusive rights from the author. In any case where a copyrighted work is used without the permission of the copyright owner, copyright protection will not extend to any part of the work in which such material has been used unlawfully. The unauthorized adaptation of a work may constitute copyright infringement. --- With starting from text, a description of the Mona Lisa is difficult to get generative art to create without significant effort to get the Mona Lisa (without saying "I want that image"). The question is then "is a description of the Mona Lisa a derivative work" and that is likely sufficiently far away from the original to be "no."