5 ms·

Seems like it's a matter of time (or maybe it's already done and I haven't seen it yet) before you get optimizers that takes an image and produces a seed+prompt

Seems like it's a matter of time (or maybe it's already done and I haven't seen it yet) before you get optimizers that takes an image and produces a seed+prompt that yields a super super conceptually similar image. I think being able to selectively optimize for "magic" random seeds in the diffusion algorithm will be kind of critically important here. Different seeds can produce very different images given the same prompt. What if an optimizer can find a seed+prompt combos that are just as good as cloning as img2img?

GaggiX 4y ago
If you want to clone an image (for some reason) just encode it in latent space and retrieve it using a deterministic sampler such as DDIM. Or you can simply right-click and copy. If you instead don't want to clone an image, you can just extract the CLIP image embeddings from it and use them to condition a generative model like Dalle 2, Midjourney or Karlo (open source). The CLIP embeddings extract really well the semantic meaning of an image.
- gamegoblin 4y ago
  Going image -> vector space -> image feels very much like compression for the purpose of copyright. Like a lower quality JPEG of some image still has the same copyright properties of the original. Something about going image -> prompt -> image feels like it "subverts" this somehow, even if the prompt is hyper-optimized to recreate the original image. Obviously, this is just my feel/impression of it, the real test is how a jury feels about it. The next few years will be really interesting in exposing that this is a really massive gray area.
  - Ensorceled 4y ago
    > Something about going image -> prompt -> image feels like it "subverts" this somehow, That's only because you understand the algorithm in the first example. At a jury trial most, if not all, of the jurors will find both methods equally opaque and, I believe, will treat them as equivalent.
    - joxel 4y ago
      This is a great example of how dumb our legal system is.
    - int_19h 4y ago
      I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. That is something a good lawyer could make a show of, even to the point of having an artist draw a picture from such a description. The vectors are much more opaque because there's no straightforward human equivalent.
      - oldgradstudent 4y ago
        > I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. Describing to a human. Describing it to stable Diffusion is very different. If you ask for a starry night, you get van Gogh's starry night, sometime with the original frame. A prompt can easily trigger it to make a derivative work from something in its training set, and it often does. Any competent expert will tell that to the jury and easily demonstrate it again and again.
      - Ensorceled 4y ago
        My point is that if there is a prompt that results in a picture that is a nearly identical copy, the average jury member is going to think "yep, that's a copy". Trying to explain how that "isn't really a copy" by explaining AI concepts isn't going to win the day, not when they can SEE the copy.
        int_19h 4y ago
        If the textual description in and of itself is already "not a copy", though, the question of whether the image produced from that textual description is a copy shouldn't arise, no? It's not like the jury is ruling on random questions; they're going to give answers to whatever the judge puts in front of them.
    - 4y ago

GaggiX 4y ago

If you want to clone an image (for some reason) just encode it in latent space and retrieve it using a deterministic sampler such as DDIM. Or you can simply right-click and copy. If you instead don't want to clone an image, you can just extract the CLIP image embeddings from it and use them to condition a generative model like Dalle 2, Midjourney or Karlo (open source). The CLIP embeddings extract really well the semantic meaning of an image.

gamegoblin 4y ago
Going image -> vector space -> image feels very much like compression for the purpose of copyright. Like a lower quality JPEG of some image still has the same copyright properties of the original. Something about going image -> prompt -> image feels like it "subverts" this somehow, even if the prompt is hyper-optimized to recreate the original image. Obviously, this is just my feel/impression of it, the real test is how a jury feels about it. The next few years will be really interesting in exposing that this is a really massive gray area.
- Ensorceled 4y ago
  > Something about going image -> prompt -> image feels like it "subverts" this somehow, That's only because you understand the algorithm in the first example. At a jury trial most, if not all, of the jurors will find both methods equally opaque and, I believe, will treat them as equivalent.
  - joxel 4y ago
    This is a great example of how dumb our legal system is.
  - int_19h 4y ago
    I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. That is something a good lawyer could make a show of, even to the point of having an artist draw a picture from such a description. The vectors are much more opaque because there's no straightforward human equivalent.
    - oldgradstudent 4y ago
      > I think the jurors would be aware (or at least acceptive) of the notion that describing a picture using words does not create a derived work. Describing to a human. Describing it to stable Diffusion is very different. If you ask for a starry night, you get van Gogh's starry night, sometime with the original frame. A prompt can easily trigger it to make a derivative work from something in its training set, and it often does. Any competent expert will tell that to the jury and easily demonstrate it again and again.
    - Ensorceled 4y ago
      My point is that if there is a prompt that results in a picture that is a nearly identical copy, the average jury member is going to think "yep, that's a copy". Trying to explain how that "isn't really a copy" by explaining AI concepts isn't going to win the day, not when they can SEE the copy.
      - int_19h 4y ago
        If the textual description in and of itself is already "not a copy", though, the question of whether the image produced from that textual description is a copy shouldn't arise, no? It's not like the jury is ruling on random questions; they're going to give answers to whatever the judge puts in front of them.
  - 4y ago