5 ms·

Tangent discussion: What are people's here experiences with running Stable Diffusion locally? I've installed it and haven't had time to play around, but I also

by danso 4y ago

Tangent discussion: What are people's here experiences with running Stable Diffusion locally? I've installed it and haven't had time to play around, but I also have a RTX 3060 8GB GPU -- IIRC, the official SD docs say that 10GB is the minimum, but I've seen posts/articles saying it could be done with 8GB. Mostly I'm interested in the processing time. Like, using a midrange desktop, what's the average time to expect SD to produce an image from a prompt? Minutes/Tens of minutes/Hours?

Morgawr 4y ago
I have a Titan X (Pascal) from like 2015 with 12GB of vram and I've had no trouble running it locally. I'd say it takes me about 30 seconds maybe to generate a single image on a 30ddim (which is like the bare minimum I consider for quick iterations), when I want to get more quality images after I focus on a proper prompt, I set it to like 100 or 200 ddim and that maybe takes 1 minute for one picture (I didn't accurately measure). I usually just let it run for a few minutes in bulk of 10 or 20 pictures while I go do something else then come back half 15-20 minutes later. It runs pretty well but the most I can get is a 768x512 image, but it's pretty good for stuff like visual novel background art[0] and similar things. [0] - https://twitter.com/xMorgawr/status/1564271156462440448 https://twitter.com/xMorgawr/status/1564271156462440448
chrismorgan 4y ago
ASUS Zephyrus G15 (GA503QM) with a laptop 3060 (95W, I think) with 6GB of VRAM, basujindal fork, does 512×512 at about 3.98 iterations per second in turbo mode (for which there’s plenty of memory at that size). That’s under 15 seconds per image on even small batches at the default 50 steps, and I think it was only using around 4.5GB of VRAM. (I say “I think” because I’ve uninstalled the nvidia-dkms package again while I’m not using it because having a functional NVIDIA dual-GPU system in Linux is apparently too annoying: Alacritty takes a few seconds to start because it blocks on spinning up the dGPU for a bit for some reason even though it doesn’t use it, wake from sleep takes five or ten seconds instead of under one second, Firefox glyph and icon caches for individual windows occasionally (mostly on wake) get blatted (that’s actually mildly concerning, though so long as the memory corruption is only in GPU memory it’s probably OK), and if the nvidia modules are loaded at boot time Sway requires --unsupported-gpu and my backlight brightness keys break because the device changes in the /sys tree and I end up with an 0644 root:root brightness file instead of the usual 0664 root:video, and I can’t be bothered figuring it out or arranging a setuid wrapper or whatever. Yeah, now I’m remembering why I would have preferred a single-GPU laptop, to say nothing of the added expense of a major component that had gone completely unused until this week. But no one sells what I wanted without a dedicated GPU for some reason.)
cube2222 4y ago
RTX 3080 (10GB) here Keep in mind to have the batch-size low (equal to 1, probably), that was my main issue when I first installed this. Then, there's lot's of great forks already which add an interactive repl or web ui [0][1]. They also run with half-precision which saves a few bytes. Additionally, they optionally integrate with upscaling neural networks, which means you can generate 512x512 images with stable diffusion and then scale them up to 1024x1024 easily. Moreover, they optionally integrate with face-fixing neural networks, which can also drastically improve the quality of images. There's also this ultra-optimized repo, but it's a fair bit slower [2]. [0]: https://github.com/lstein/stable-diffusion https://github.com/lstein/stable-diffusion [1]: https://github.com/hlky/stable-diffusion https://github.com/hlky/stable-diffusion [2]: https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion
deleted 4y ago
[deleted]
wccrawford 4y ago
I had to get a different repo with "optimized commands" on the first day, but my 3070 8GB has been happily processing images in decent time.
- nbutyllithium 4y ago
  I decided to set up a local instance yesterday with my 3070 TI 8GB and had similar success, about 10 seconds per image at the default settings. Like you I also opted for a different repo [0] which emphasized adding a GUI but I think also opts out of the watermark addition/other checks. Sounds like it reduces memory usage from what others have said. Had more trouble coming up with creative prompts then getting set up surprisingly (to me anyway). [0] https://github.com/hlky/stable-diffusion https://github.com/hlky/stable-diffusion
  - wccrawford 4y ago
    I found it very easy to set up, too. I had a previous couple things I set up that were a lot harder to set up. Stable Diffusion has been dreamy. I'm already tempted to upgrade my setup to one of these with the GUIs, but I think if I wait just a bit longer, it's going to get even better. So I'm resisting the urge.
  - nickthegreek 4y ago
    This helped me with prompt generation: https://promptomania.com/stable-diffusion-prompt-builder/ https://promptomania.com/stable-diffusion-prompt-builder/
cbozeman 4y ago
Removing the NSFW and watermark modules from the model will easily allow you to run it with 8 GB VRAM (usually takes around 6.9 GB for 512x512 generations). With an RTX 3060, your average image generation time is going to be around 7-11 seconds if I recall correctly. This swings wildly based on how you adjust different settings, but I doubt you'll ever require more than 70 seconds to generate an image.
orangecat 4y ago
I'm using the fork at https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion which is optimized for lower VRAM usage. My RTX 2070 (8 GB) takes about 90 seconds to generate a batch of 4 images.
- uwuemu 4y ago
  Just did a thousand images (different seeds for the same prompt) with the base (not fork) model at 50 iterations in a couple of hours (RTX 3080). Gpu vram usage seemed to hover around 9 gigs. This model is pretty damn fast for what it does. I quickly threw together a folder structure where I have a md5'd prompt as a folder name, into that goes _promp.txt with the actual text of the prompt and the images i generate in a loop with the seed used and iteration number in the image's file name. That way I can generate like 20 seed-based images for a prompt and if the model bites I let it run with a much higher number of seed-based images. When you have a 1000 to pick from, some of the results are freaking amazing.
mlsu 4y ago
I have a dated 1070 with 8gb of vram, some of which also renders my desktop. I was able to obtain 256x512 images with this card using the standard model, but ran into OOM issues. I don't mind waiting, so now I am using the "fast" repo: https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion With this, it takes 30s to generate a 768x512 image (any larger and I am experiencing OOM issues again). I think you should expect a bit faster at the same resolution with your 3060 because it's a faster card with the same amount of memory.
sarsway 4y ago
It's pretty fast on a RTX 3070 (8GB), a few seconds per image. My first impression is it seems a lot more useful then DALL-E, because you can quickly iterate on prompts, and also generate many batches, picking the best ones. To get something that's actually usable, you'll have to tinker around a bit and give it a few tries. With DALL-E, feedback is slower, and there's reluctance to just hammer prompts because of credits.
- foobiekr 4y ago
  I was blown away when I got DallE access, but now it seems almost silly by comparison. I really wonder why the DallE team chose to expose so few controls.

by danso 4y ago

Morgawr 4y ago

I have a Titan X (Pascal) from like 2015 with 12GB of vram and I've had no trouble running it locally. I'd say it takes me about 30 seconds maybe to generate a single image on a 30ddim (which is like the bare minimum I consider for quick iterations), when I want to get more quality images after I focus on a proper prompt, I set it to like 100 or 200 ddim and that maybe takes 1 minute for one picture (I didn't accurately measure). I usually just let it run for a few minutes in bulk of 10 or 20 pictures while I go do something else then come back half 15-20 minutes later. It runs pretty well but the most I can get is a 768x512 image, but it's pretty good for stuff like visual novel background art[0] and similar things. [0] - https://twitter.com/xMorgawr/status/1564271156462440448 https://twitter.com/xMorgawr/status/1564271156462440448

chrismorgan 4y ago

ASUS Zephyrus G15 (GA503QM) with a laptop 3060 (95W, I think) with 6GB of VRAM, basujindal fork, does 512×512 at about 3.98 iterations per second in turbo mode (for which there’s plenty of memory at that size). That’s under 15 seconds per image on even small batches at the default 50 steps, and I think it was only using around 4.5GB of VRAM. (I say “I think” because I’ve uninstalled the nvidia-dkms package again while I’m not using it because having a functional NVIDIA dual-GPU system in Linux is apparently too annoying: Alacritty takes a few seconds to start because it blocks on spinning up the dGPU for a bit for some reason even though it doesn’t use it, wake from sleep takes five or ten seconds instead of under one second, Firefox glyph and icon caches for individual windows occasionally (mostly on wake) get blatted (that’s actually mildly concerning, though so long as the memory corruption is only in GPU memory it’s probably OK), and if the nvidia modules are loaded at boot time Sway requires --unsupported-gpu and my backlight brightness keys break because the device changes in the /sys tree and I end up with an 0644 root:root brightness file instead of the usual 0664 root:video, and I can’t be bothered figuring it out or arranging a setuid wrapper or whatever. Yeah, now I’m remembering why I would have preferred a single-GPU laptop, to say nothing of the added expense of a major component that had gone completely unused until this week. But no one sells what I wanted without a dedicated GPU for some reason.)

cube2222 4y ago

RTX 3080 (10GB) here Keep in mind to have the batch-size low (equal to 1, probably), that was my main issue when I first installed this. Then, there's lot's of great forks already which add an interactive repl or web ui [0][1]. They also run with half-precision which saves a few bytes. Additionally, they optionally integrate with upscaling neural networks, which means you can generate 512x512 images with stable diffusion and then scale them up to 1024x1024 easily. Moreover, they optionally integrate with face-fixing neural networks, which can also drastically improve the quality of images. There's also this ultra-optimized repo, but it's a fair bit slower [2]. [0]: https://github.com/lstein/stable-diffusion https://github.com/lstein/stable-diffusion [1]: https://github.com/hlky/stable-diffusion https://github.com/hlky/stable-diffusion [2]: https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion

deleted 4y ago

[deleted]

wccrawford 4y ago

I had to get a different repo with "optimized commands" on the first day, but my 3070 8GB has been happily processing images in decent time.

nbutyllithium 4y ago
I decided to set up a local instance yesterday with my 3070 TI 8GB and had similar success, about 10 seconds per image at the default settings. Like you I also opted for a different repo [0] which emphasized adding a GUI but I think also opts out of the watermark addition/other checks. Sounds like it reduces memory usage from what others have said. Had more trouble coming up with creative prompts then getting set up surprisingly (to me anyway). [0] https://github.com/hlky/stable-diffusion https://github.com/hlky/stable-diffusion
- wccrawford 4y ago
  I found it very easy to set up, too. I had a previous couple things I set up that were a lot harder to set up. Stable Diffusion has been dreamy. I'm already tempted to upgrade my setup to one of these with the GUIs, but I think if I wait just a bit longer, it's going to get even better. So I'm resisting the urge.
- nickthegreek 4y ago
  This helped me with prompt generation: https://promptomania.com/stable-diffusion-prompt-builder/ https://promptomania.com/stable-diffusion-prompt-builder/

cbozeman 4y ago

Removing the NSFW and watermark modules from the model will easily allow you to run it with 8 GB VRAM (usually takes around 6.9 GB for 512x512 generations). With an RTX 3060, your average image generation time is going to be around 7-11 seconds if I recall correctly. This swings wildly based on how you adjust different settings, but I doubt you'll ever require more than 70 seconds to generate an image.

orangecat 4y ago

I'm using the fork at https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion which is optimized for lower VRAM usage. My RTX 2070 (8 GB) takes about 90 seconds to generate a batch of 4 images.

uwuemu 4y ago
Just did a thousand images (different seeds for the same prompt) with the base (not fork) model at 50 iterations in a couple of hours (RTX 3080). Gpu vram usage seemed to hover around 9 gigs. This model is pretty damn fast for what it does. I quickly threw together a folder structure where I have a md5'd prompt as a folder name, into that goes _promp.txt with the actual text of the prompt and the images i generate in a loop with the seed used and iteration number in the image's file name. That way I can generate like 20 seed-based images for a prompt and if the model bites I let it run with a much higher number of seed-based images. When you have a 1000 to pick from, some of the results are freaking amazing.

mlsu 4y ago

I have a dated 1070 with 8gb of vram, some of which also renders my desktop. I was able to obtain 256x512 images with this card using the standard model, but ran into OOM issues. I don't mind waiting, so now I am using the "fast" repo: https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion With this, it takes 30s to generate a 768x512 image (any larger and I am experiencing OOM issues again). I think you should expect a bit faster at the same resolution with your 3060 because it's a faster card with the same amount of memory.

sarsway 4y ago

It's pretty fast on a RTX 3070 (8GB), a few seconds per image. My first impression is it seems a lot more useful then DALL-E, because you can quickly iterate on prompts, and also generate many batches, picking the best ones. To get something that's actually usable, you'll have to tinker around a bit and give it a few tries. With DALL-E, feedback is slower, and there's reluctance to just hammer prompts because of credits.

foobiekr 4y ago
I was blown away when I got DallE access, but now it seems almost silly by comparison. I really wonder why the DallE team chose to expose so few controls.