4 ms·

And they're available for public consumption at the website, albeit at reduced quality. Are the images part of the distributed data set? I thought it was value

And they're available for public consumption at the website, albeit at reduced quality. Are the images part of the distributed data set? I thought it was values/coefficients that manifest from the algorithmic analysis of the source image?

shagie 4y ago
Yes... ish. On one hand, if you do a "this is the size of the net" and then divide it by the number of training images, its rather small amount of storage per image. On the other hand, when I was playing with stable diffusion on the command line following the instructions of https://replicate.com/blog/run-stable-diffusion-on-m1-mac https://replicate.com/blog/run-stable-diffusion-on-m1-mac python scripts/txt2img.py --prompt "wolf with bling walking down a street" --n_samples 6 --n_iter 1 --plms I got: https://imgur.com/a/N1OufD1 https://imgur.com/a/N1OufD1 Now, you tell me if there's a copyrighted image encoded in that data set or not.
- cheald 4y ago
  That's a very interesting result. Did you happen to capture the seed for either of those first two images? It would be interesting to try to reproduce.
  - shagie 4y ago
    Alas no. And I haven't been able to tickle it again in the right way to get those images out. The invocation of that run is still in my scroll back: (venv) shagie@MacM1 stable-diffusion % python scripts/txt2img.py --prompt "wolf with bling walking down a street" --n_samples 6 --n_iter 1 --plms Global seed set to 42 Loading model from models/ldm/stable-diffusion-v1/model.ckpt Global Step: 470000 LatentDiffusion: Running in eps-prediction mode DiffusionWrapper has 859.52 M params. making attention of type 'vanilla' with 512 in_channels Working with z of shape (1, 4, 32, 32) = 4096 dimensions. making attention of type 'vanilla' with 512 in_channels That's the only spot I see the seed mentioned and then it goes on with lots of other logging but nothing seed related that would indicate a way to reproduce it. --- (late edit) you can fairly accurately (so far 1 image out of 20) get that image out with the prompt "Rick Astley Never Gonna Give You Up"
    - cheald 4y ago
      I'm thus far unable to reproduce it. Given: Rick Astley Never Gonna Give You Up Steps: 20, Sampler: PLMS, CFG scale: 7, Seed: 4231695436, Size: 512x512, Batch size: 2, Batch pos: 0 I ran a couple of batches of 32 (64 images total): https://imgur.com/a/74IbCuD https://imgur.com/a/74IbCuD (The images with the nonsensical but obvious Impact font that was learned from memes are quite funny, though) If you can get a full set of parameters (size, sampler, seed, prompt, cfg scale) then I should hopefully be able to reproduce your results, though.
- beiller 4y ago
  You have the version which filters out NSFW images based on keywords. The code literally replaces images it thinks are NSFW with Rick Astley. Copyright aside (yes it's probably wrong to hard code an image of Rick Astley in the actual stable diffusion git repository) that image is not contained in the weights of the model. - edit - please god tell me this is not an elaborate rick roll :)
  - shagie 4y ago
    It's not... though if stable-diffusion % python scripts/txt2img.py --prompt "Rick Astley Never Gonna Give You Up" --n_samples 1 --n_iter 1 --plms is such that it triggers NSFW sometimes, then... I'm... let's say "confused" about what entails NSFW prompts. (digging through scroll back) Creating invisible watermark encoder (see https://github.com/ShieldMnt/invisible-watermark)... Sampling: 0%| | 0/1 [00:00<?, ?it/sData shape for PLMS sampling is (1, 4, 64, 64) | 0/1 [00:00<?, ?it/s] Running PLMS Sampling with 50 timesteps PLMS Sampler: 100%|| 50/50 [03:23<00:00, 4.06s/it] Potential NSFW content was detected in one or more images. A black image will be returned instead. Try again with a different prompt and/or seed.:00, 3.99s/it] data: 100%|| 1/1 [03:29<00:00, 209.20s/it] Sampling: 100%|| 1/1 [03:29<00:00, 209.20s/it] Your samples are ready and waiting for you here: outputs/txt2img-samples Apparently you're right... though the "black image" is a poor description of the image.
    - beiller 4y ago
      Yeah I see noisy images in your output it may just be glitching. I may have poorly described how it worked because I'm not fully sure. It may be a nsfw image detection model and not based on the prompt. Either way you can disable it in code, I tried

shagie 4y ago

Yes... ish. On one hand, if you do a "this is the size of the net" and then divide it by the number of training images, its rather small amount of storage per image. On the other hand, when I was playing with stable diffusion on the command line following the instructions of https://replicate.com/blog/run-stable-diffusion-on-m1-mac https://replicate.com/blog/run-stable-diffusion-on-m1-mac python scripts/txt2img.py --prompt "wolf with bling walking down a street" --n_samples 6 --n_iter 1 --plms I got: https://imgur.com/a/N1OufD1 https://imgur.com/a/N1OufD1 Now, you tell me if there's a copyrighted image encoded in that data set or not.

cheald 4y ago
That's a very interesting result. Did you happen to capture the seed for either of those first two images? It would be interesting to try to reproduce.
- shagie 4y ago
  Alas no. And I haven't been able to tickle it again in the right way to get those images out. The invocation of that run is still in my scroll back: (venv) shagie@MacM1 stable-diffusion % python scripts/txt2img.py --prompt "wolf with bling walking down a street" --n_samples 6 --n_iter 1 --plms Global seed set to 42 Loading model from models/ldm/stable-diffusion-v1/model.ckpt Global Step: 470000 LatentDiffusion: Running in eps-prediction mode DiffusionWrapper has 859.52 M params. making attention of type 'vanilla' with 512 in_channels Working with z of shape (1, 4, 32, 32) = 4096 dimensions. making attention of type 'vanilla' with 512 in_channels That's the only spot I see the seed mentioned and then it goes on with lots of other logging but nothing seed related that would indicate a way to reproduce it. --- (late edit) you can fairly accurately (so far 1 image out of 20) get that image out with the prompt "Rick Astley Never Gonna Give You Up"
  - cheald 4y ago
    I'm thus far unable to reproduce it. Given: Rick Astley Never Gonna Give You Up Steps: 20, Sampler: PLMS, CFG scale: 7, Seed: 4231695436, Size: 512x512, Batch size: 2, Batch pos: 0 I ran a couple of batches of 32 (64 images total): https://imgur.com/a/74IbCuD https://imgur.com/a/74IbCuD (The images with the nonsensical but obvious Impact font that was learned from memes are quite funny, though) If you can get a full set of parameters (size, sampler, seed, prompt, cfg scale) then I should hopefully be able to reproduce your results, though.
beiller 4y ago
You have the version which filters out NSFW images based on keywords. The code literally replaces images it thinks are NSFW with Rick Astley. Copyright aside (yes it's probably wrong to hard code an image of Rick Astley in the actual stable diffusion git repository) that image is not contained in the weights of the model. - edit - please god tell me this is not an elaborate rick roll :)
- shagie 4y ago
  It's not... though if stable-diffusion % python scripts/txt2img.py --prompt "Rick Astley Never Gonna Give You Up" --n_samples 1 --n_iter 1 --plms is such that it triggers NSFW sometimes, then... I'm... let's say "confused" about what entails NSFW prompts. (digging through scroll back) Creating invisible watermark encoder (see https://github.com/ShieldMnt/invisible-watermark)... Sampling: 0%| | 0/1 [00:00<?, ?it/sData shape for PLMS sampling is (1, 4, 64, 64) | 0/1 [00:00<?, ?it/s] Running PLMS Sampling with 50 timesteps PLMS Sampler: 100%|| 50/50 [03:23<00:00, 4.06s/it] Potential NSFW content was detected in one or more images. A black image will be returned instead. Try again with a different prompt and/or seed.:00, 3.99s/it] data: 100%|| 1/1 [03:29<00:00, 209.20s/it] Sampling: 100%|| 1/1 [03:29<00:00, 209.20s/it] Your samples are ready and waiting for you here: outputs/txt2img-samples Apparently you're right... though the "black image" is a poor description of the image.
  - beiller 4y ago
    Yeah I see noisy images in your output it may just be glitching. I may have poorly described how it worked because I'm not fully sure. It may be a nsfw image detection model and not based on the prompt. Either way you can disable it in code, I tried