4 ms·

You're going to need at least 10GB VRAM. My SFF pc with 4GB VRAM can only run dalle mini / craiyon :(

• mempko 4y ago
  Not if you change the precision to float16. Should work on a smaller card. Tried on a 1080 with 8GB and it works well.
  • krisoft 4y ago
    How would one do that? ----- Sorry my bad, found the answer. One simply adds the following flags to the StableDiffusionPipeline.from_pretrained call in the example: revision="fp16", torch_dtype=torch.float16 Found it in this blogpost: https://huggingface.co/blog/stable_diffusion https://huggingface.co/blog/stable_diffusion mempko thank you for your hint! I was about to drop a not insignificant amount of money on a new GPU. What does one lose by using float16 representation? Does it make the images visually less detailed? Or how can one reason about this?
    • ShamelessC 4y ago
      Zero loss. All upside. Only causes issues when training. 32-bit ships by default because it is compatible with cpu and GPU’s that might not have native fp16 support. Edit: Just to be clear, your intuition that it could cause issues is certainly merited - and not _all_ models can be trivially converted from fp32 to fp16 without some new error accumulating (during inference). Variational autoencoders like VQGAN and GAN's are particularly prone to such issues. But in this case, it's all upside.
• deleted 4y ago
  [deleted]