5 ms·

Gonna have to pick up one of those cheap RTX 3090s that miners are getting rid of so I can try this myself. With 8GB I'm just getting out of memory errors.

14 4y ago
I was hoping an online service would pop up where you can use someone else's hardware for a small fee and generate images. I really want to give it a try but only have a gtx970 so don't think it will work.
- kcartlidge 4y ago
  > I was hoping an online service would pop up where you can use someone else's hardware for a small fee and generate images. This one charges but gives pretty good results at 512x512 - and in only a couple of seconds at that resolution. For more logo/cartoony stuff you can generally get a 10 step done in less than a second. When I signed up a few weeks ago they gave 200 credits free (IIRC). After that it is $10 for another 1,000 credits (a credit gives 5 images at 10 step 512x512 or 1 image at 50 step 512x512). https://beta.dreamstudio.ai https://beta.dreamstudio.ai Only thing to note is that after a while it slows down, but I realised that is because every image generated is going into an array in local storage in the browser. Using the browser's Inspect -> Application area to clear that array every 100 images or so sorted that out.
- pbhjpbhj 4y ago
  Places remote computer providers like Vultr, Digital Ocean, Linode, etc., have servers with GPUs you can run up. You pay for uptime, and storage (including machines that are not up).
montebicyclelo 4y ago
You can 100% run this on an 8gb card. Make sure to load the model weights in half precision (and optionally enable attention slicing). E.g., using the diffusers library [1]: from diffusers import StableDiffusionPipeline from torch import autocast pipe = StableDiffusionPipeline.from_pretrained( "CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16, use_auth_token=True ) pipe = pipe.to("cuda") pipe.enable_attention_slicing() prompt = "a photo of an astronaut riding a horse on mars" with autocast("cuda"): image = pipe(prompt).images[0] [1] https://github.com/huggingface/diffusers https://github.com/huggingface/diffusers
cercatrova 4y ago
Use a fork with optimized memory usage such as automatic1111's. You can even run it on a 4 GB VRAM card.
pdntspa 4y ago
There is a fork at https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion which is optimized for 8gb cards, I have been using it to run SD on my 8gb 1080 That fork released around when SD released, I've been wondering if anyone's integrated those optimizations into some of the more prominent forks that are floating around
- macrolime 4y ago
  Does it work with textual inversion? I've got Stable Diffusion set up and its works fine, but I get out of memory error on the textual inversion feature.
  - pdntspa 4y ago
    What is textual inversion?
    - macrolime 4y ago
      Textual inversion is teaching it a new concept, such as "Ugly Sonic", from a couple images. Then you get a file you can use in SD to use this new concept in prompts.
  - montebicyclelo 4y ago
    No, not currently. Textual inversion requires training the model, which requires gradients to be computed and stored in VRAM, which doesn't happen for normal inference, so larger than usual VRAM is required. You can do it in around 4 hours on Colab, using HuggingFace's notebook. https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/sd_textual_inversion_training.ipynb https://colab.research.google.com/github/huggingface/noteboo... Then you should be able to take the result (a single text token embedding) and use that locally. (I didn't realise you were talking about textual inversion in your original post.)
    - macrolime 4y ago
      I tried with these pretrained embeddings, but haven't gotten them to run in inference locally without crashing yet. https://github.com/hlky/sd-embeddings https://github.com/hlky/sd-embeddings
LanternLight83 4y ago
Got the grado webui running (unstably, thx to a memory leak) on my GTX 1050Ti 4GB; you /can/ do it! Be sure to check-out the "optimizedSD" repo if you follow up

14 4y ago

I was hoping an online service would pop up where you can use someone else's hardware for a small fee and generate images. I really want to give it a try but only have a gtx970 so don't think it will work.

kcartlidge 4y ago
> I was hoping an online service would pop up where you can use someone else's hardware for a small fee and generate images. This one charges but gives pretty good results at 512x512 - and in only a couple of seconds at that resolution. For more logo/cartoony stuff you can generally get a 10 step done in less than a second. When I signed up a few weeks ago they gave 200 credits free (IIRC). After that it is $10 for another 1,000 credits (a credit gives 5 images at 10 step 512x512 or 1 image at 50 step 512x512). https://beta.dreamstudio.ai https://beta.dreamstudio.ai Only thing to note is that after a while it slows down, but I realised that is because every image generated is going into an array in local storage in the browser. Using the browser's Inspect -> Application area to clear that array every 100 images or so sorted that out.
pbhjpbhj 4y ago
Places remote computer providers like Vultr, Digital Ocean, Linode, etc., have servers with GPUs you can run up. You pay for uptime, and storage (including machines that are not up).

montebicyclelo 4y ago

You can 100% run this on an 8gb card. Make sure to load the model weights in half precision (and optionally enable attention slicing). E.g., using the diffusers library [1]: from diffusers import StableDiffusionPipeline from torch import autocast pipe = StableDiffusionPipeline.from_pretrained( "CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=torch.float16, use_auth_token=True ) pipe = pipe.to("cuda") pipe.enable_attention_slicing() prompt = "a photo of an astronaut riding a horse on mars" with autocast("cuda"): image = pipe(prompt).images[0] [1] https://github.com/huggingface/diffusers https://github.com/huggingface/diffusers

cercatrova 4y ago

Use a fork with optimized memory usage such as automatic1111's. You can even run it on a 4 GB VRAM card.

pdntspa 4y ago

There is a fork at https://github.com/basujindal/stable-diffusion https://github.com/basujindal/stable-diffusion which is optimized for 8gb cards, I have been using it to run SD on my 8gb 1080 That fork released around when SD released, I've been wondering if anyone's integrated those optimizations into some of the more prominent forks that are floating around

macrolime 4y ago
Does it work with textual inversion? I've got Stable Diffusion set up and its works fine, but I get out of memory error on the textual inversion feature.
- pdntspa 4y ago
  What is textual inversion?
  - macrolime 4y ago
    Textual inversion is teaching it a new concept, such as "Ugly Sonic", from a couple images. Then you get a file you can use in SD to use this new concept in prompts.
- montebicyclelo 4y ago
  No, not currently. Textual inversion requires training the model, which requires gradients to be computed and stored in VRAM, which doesn't happen for normal inference, so larger than usual VRAM is required. You can do it in around 4 hours on Colab, using HuggingFace's notebook. https://colab.research.google.com/github/huggingface/notebooks/blob/main/diffusers/sd_textual_inversion_training.ipynb https://colab.research.google.com/github/huggingface/noteboo... Then you should be able to take the result (a single text token embedding) and use that locally. (I didn't realise you were talking about textual inversion in your original post.)
  - macrolime 4y ago
    I tried with these pretrained embeddings, but haven't gotten them to run in inference locally without crashing yet. https://github.com/hlky/sd-embeddings https://github.com/hlky/sd-embeddings

LanternLight83 4y ago

Got the grado webui running (unstably, thx to a memory leak) on my GTX 1050Ti 4GB; you /can/ do it! Be sure to check-out the "optimizedSD" repo if you follow up