11 ms·

A Web UI for Stable Diffusion

284 points by feross 4y ago

nowandlater 4y ago
This is the one I've been using https://github.com/sd-webui/stable-diffusion-webui https://github.com/sd-webui/stable-diffusion-webui . docker-compose up , works great.
- TekMol 4y ago
  Is there a way to run this in the cloud? On Google Colab or elsewhere?
  - fragmede 4y ago
    Colab: https://colab.research.google.com/github/WASasquatch/StableDiffusionUI-TomRiddle/blob/main/StableDiffusionUI_Tom_Riddle_Edition.ipynb https://colab.research.google.com/github/WASasquatch/StableD... To run it elsewhere in the cloud, grab a GPU (spot) instance and SSH in.
    - TekMol 4y ago
      You mean a VM on a machine with a GPU? Or does it have to be a bare metal machine? What is a good provider of suitable VMs/machines? And what do you do after you SSHed in? The installation instructions seem to be for windows users (click here, then click there ...) is there a linux script that does the installation automatically?
      - fragmede 4y ago
        Just a VM with a GPU, doesn't need to be bare metal. AWS/GCP/Azure has em, but for GPU cloud instances, boutique vendors like CoreWeave, runpod, lambdalabs.com, vast.ai, paperspace may be more competitive. Parent comment alludes to docker-compose up.
        TekMol 4y ago
        You can run docker inside a VM? And it will be able to use the GPU and tunnel port 80 through the docker container, through the VM and to the web?
        fragmede 4y ago
        There's some trickery and details, but yes: https://github.com/NVIDIA/nvidia-docker https://github.com/NVIDIA/nvidia-docker
      - mnahkies 4y ago
        You can also get windows instances with GPU attached and RDP in, though personally I prefer Linux https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/accelerated-computing-instances.html#gpu-instances https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/accel...
      - fragmede 4y ago
        Here's an install script if you need handholding :) https://github.com/JoshuaKimsey/Linux-StableDiffusion-Script/blob/main/linux-sd.sh https://github.com/JoshuaKimsey/Linux-StableDiffusion-Script...
    - danuker 4y ago
      Lambda Labs is probably cheaper: https://lambdalabs.com/service/gpu-cloud https://lambdalabs.com/service/gpu-cloud
- stbtrax 4y ago
- wyldfire 4y ago
  I like this one but had some trouble with using img2img. Maybe my image was too small (it was smaller than 512x512). Failed with the same signature as an issue that was closed with a fix.
  - ghilston 4y ago
    I am mobile, but there's an issue reported on github about img2img
- jyrkesh 4y ago
  I've also been using this one (wasn't sure at first, they just migrated from the /hlky/ namespace on GitHub), but I have no idea at first glance what the differences are. I will say that this one has had REALLY active development as new features have been coming out, and is pretty polished at this point (albeit I'm using it more as a toy than anything, but it's awesome to have a quick way to use the new features that have been shipping out).
- KronisLV 4y ago
  Seems like it fails if you have an AMD GPU instead of an Nvidia one (at least that's my guess, based on the error contents): ERROR: for stable-diffusion Cannot start service stable-diffusion: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown
  - mach1ne 4y ago
    Indeed, Stable Diffusion does not currently run on AMD graphics cards.
    - btdmaster 4y ago
      Some people have been able to run on recent AMD cards with rocm: https://github.com/CompVis/stable-diffusion/issues/48 https://github.com/CompVis/stable-diffusion/issues/48
    - CapsAdmin 4y ago
      I'm able to run it on my rx6900 xt on Linux. I tried using a docker container and it took 3 min to generate a prompt. However it seems that 2:45 min is somehow spent on tje GPU and finally the remaining 15 seconds the GPU gets utilized. I haven't had the time to look into this yet, but it does seem to work.
    - fragmede 4y ago
      It does, on Linux, Windows, and MacOS.
      - needle0 4y ago
        I've gotten it running with a Radeon RX 6800 on Ubuntu Linux 22.04 (with overwriting PyTorch with a ROCm-supporting version), and on Windows 10 (in a very barebones way using ONNX), but are there better, more full-featured ways to get it running on Windows? Would love to know.
sharps_xp 4y ago
if someone can dockerize this, please reply with a link!
- stephanheijl 4y ago
  This is the dockerized version of this repo: https://github.com/AbdBarho/stable-diffusion-webui-docker https://github.com/AbdBarho/stable-diffusion-webui-docker
- geerlingguy 4y ago
  The main one I've been testing (https://github.com/sd-webui/stable-diffusion-webui https://github.com/sd-webui/stable-diffusion-webui) has a Docker Compose file. Grab that, install Docker, install Nvidia's Docker integration, copy the example Docker-env file, and docker-compose up is all you need. Edit: here's a gist with exact steps I used: https://gist.github.com/geerlingguy/384ed4aba35e3118f2a0f358fc974162 https://gist.github.com/geerlingguy/384ed4aba35e3118f2a0f358...
user-one1 4y ago
It can be run directly into google colab: https://colab.research.google.com/drive/1Iy-xW9t1-OQWhb0hNxueGij8phCyluOh https://colab.research.google.com/drive/1Iy-xW9t1-OQWhb0hNxu...
- TekMol 4y ago
  When I run it, I get "Your session crashed for an unknown reason."
  - fragmede 4y ago
    Just click runtime -> run all again. There’s a weirdness where Python’s loader gets confused and the most effective fix is to crash the interpreter
smrtinsert 4y ago
I have a 6gb 1660ti, barely holding on. Is a new 12gb card good enough for now, or should I go even higher to be safe for a few years of sd innovation?
- wyldfire 4y ago
  It sounds like there's forks that are able to work with <=8GB cards. And I'm not sure but I think the weights are using f32, so switching to half might make it yet easier still to get this to work w/less memory. But yeah the next generation of models would probably capitalize on more memory somehow.
  - filiphorvat 4y ago
    People have reported that this repo even works with 2gb cards if you run it with --lowvram and --opt-split-attention.
    - password4321 4y ago
      Yes, the amount of VRAM doesn't seem to be as much of a limitation anymore. However, processing power is still important.
- fassssst 4y ago
  The GeForce 4000 series is about to release and should make Stable Diffusion wayyyyy faster based on related H100 benchmarks posted today.
- drexlspivey 4y ago
  How is M1/M2 support for SD? Is there a significant performance drop? Presumably you would be able to buy a 32GB M2 and be future proof because of the shared memory between CPU/GPU.
  - jjcon 4y ago
    In my setup at least it runs essentially in CPU mode since there is no CUDA acceleration available and metal support is really messy right now. So while quite slow I don't run into memory issues at least. It runs much faster on my desktop GPU but that has more constraints (until I upgrade my personal 1080 to a 3090 one of these days).
hwers 4y ago
People recently figured out how to export stable diffusion to onnx so it’ll be exciting to see some actual web UIs for it soon (via quantized models and tfjs/onnxruntime for web)
- password4321 4y ago
  Very cool! Can you link to where this is taking place? A commenter mentioned today it might be possible to pre-download the model and load it into the browser from the local filesystem rather than include such a gigantic blob as an accompanying dependency, fighting different caching RFC's, security/usage restrictions, and anything else that might inadvertently trigger a re-download. https://news.ycombinator.com/item?id=32777909#32779093 https://news.ycombinator.com/item?id=32777909#32779093
  - minimaxir 4y ago
    Support for ONNX export was just added to diffusers, but no runtime logic for scheduling yet. https://github.com/huggingface/diffusers https://github.com/huggingface/diffusers
ducktective 4y ago
It seems Midjourney generates better results than SD or Dall-E. What's with the "hyper resolution", "4K, detailed" adjectives which are thrown left and right, while we are at it?
- Ckalegi 4y ago
  The metadata and file names of the images in the source data set are also inputs for the model training. These keywords are common tags across images that have these characteristics, so in the same way it knows what a unicorn looks like, it also knows what a 4k unicorn looks like compared to a hyper rez unicorn.
- schleck8 4y ago
  Those are prompt engineering keywords. SD is way more reliant on tinkering with the prompt than midjourney https://moritz.pm/posts/parameters https://moritz.pm/posts/parameters
  - throwaway1851 4y ago
    MidJourney needs a lot of prompt engineering too. And Dall-E also. If you look at the prompt as an opportunity to describe what you want to see, the results are often disappointing. It works better to think backwards about how the model was trained, and what sorts of web caption words it likely saw in training examples that used the sorts of features you’re hoping it will generate. This is more of a process of learning to ask the model to produce things it’s able to produce, using its special image language.
- cma 4y ago
  Midjourney uses SD under the hood (you can see in their license), but they augnment the model in various ways.
  - qudat 4y ago
    The results in midjourney are significantly better than SD. I find it much easier to get to a good result in MJ and I've been trying to understand why. Anymore insight you could share?
jrm4 4y ago
So, and this is an ELI5 kind of question I suppose. There must be something going on like "processing a kazillion images" and I'm trying to wrap my head around how (or what part of) that work is "offloaded" to your home computer/graphics card? I just can't seem to make sense of how you can do it at home if you're not somehow in direct contact with "all the data?" e.g. must you be connected to the internet, or "stable-diffusions servers" for this to work?
- codefined 4y ago
  All those 'kazillion' images are processed into a single 'model'. Similar to how our brain cannot remember 100% of all our experiences, this model will not store precise copies of all images it is trained off of. However, it will understand concepts, such as what a unicorn looks like. For StableDiffusion, the current model is ~4GB, which is downloaded the first time you run the model. These 4GB encode all the information that the model requires to derive your images.
- ducktective 4y ago
  As someone with ~0 knowledge in this field, I think this has to do with a concept called "transfer learning" in which you once train with that kazillion of images, then use that same "coefficients" for further run of the NN.
  - palunon 4y ago
    Nah, transfer learning is when you take a trained model, and train it a little more to better fit your (potentially very different) problem domain. Such as training a cat/dog/etc recognition model on MRI scans. The goal is usually to have the more fundamental parts of your model already working and you thus need way less domain specific data. Here, you're not training anything, you're running the models (both the CLIP language model and the unet) in feedforward. That's just deploying your model, not transfer learning.
- sC3DX 4y ago
  What you interact with as the user is the model and its weights. The model (presumably some kind of convolutional neural network) has many layers, every layer has some set of nodes, and every node has a weight, which is just some coefficient. The weights are 'learned' during the model training where the model takes in the data you mention and evaluates the output. This typically happens on a super beefy computer and can take a long time for a model like this. As images are evaluated the output gets better the weights get adjusted accordingly. Now we as the user just need the model and the weights!
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
amelius 4y ago
Regarding the opening image: if it can't correctly put the marks on dice, how can it put eyes, nose and mouth correctly on a human face?
- bastawhiz 4y ago
  Presumably the number of faces in the training set far exceeds the number of dice by more than a few orders of magnitude.
  - amelius 4y ago
    In one of the other posts I noticed this option: > GFPGAN Face Correction: Automatically correct distorted faces with a built-in GFPGAN option, fixes them in less than half a second So apparently there is still an issue with faces.
    - SturgeonsLaw 4y ago
      Yep, it gets faces mostly right, but as they say, the devil's in the details. Eyes in particular don't seem to have clearly delineated concentric circles for irises and pupils, instead they are often rendered as a "swirl". Example image directly from Stable Diffusion: https://i.imgur.com/XSk8fIv.png https://i.imgur.com/XSk8fIv.png And here is that image run through GFPGAN: https://i.imgur.com/I53AGmh.png https://i.imgur.com/I53AGmh.png Interesting to note how specialised GFPGAN is, as some of the other details (flowers, hair) seem to be worse in the processed image. I plan to finish this image by manually blending the best of both pictures.
      - userbinator 4y ago
        Ironically, that almost looks like the sort of fantasy image you'd find as GPU box art many years ago.
- eminence32 4y ago
  In my experience, Stable Diffusion is also pretty bad about human faces and hands too
fibonacc 4y ago
a sigh of relief as I thought that this would generate instead of pictures, React UI code based on plain text description. First they came for illustrators, then they came for UI designers.
- MattRix 4y ago
  heh yeah it won’t be that long until we have A Stable Diffusion for Web UI
- fragmede 4y ago
  You've seen https://twitter.com/sharifshameem/status/1282676454690451457 https://twitter.com/sharifshameem/status/1282676454690451457 yeah?
  - fibonacc 4y ago
    yes what I mean was mixing stable diffusion into the UI styling without the fine atomic details. Like say I want a dashboard with color themes from cyberpunk complete with graphics/art/logo. It would be a "good enough" category if something can literally be told to produce a complete frontend.
offsky 4y ago
I’m waiting for someone to wrap this up into a desktop app that I can install and run on my Mac.
- geuis 4y ago
  I've been looking into this for the last 2 days. Unless you're running an M1 Mac or newer, you're SOL. Stable Diffusion is built on PyTorch. PyTorch mainly has been designed to work with Nvidia cards. However PyTorch added support for something called RocM like a year ago that adds compatibility with newer AMD cards. Unfortunately RocM doesn't support slightly older AMD cards in conjunction with intel processors. So my 32gb pretty powerful 2020 16in MacBook Pro isn't capable of running Stable Diffusion. Any native app will likely have to rely on a remote cloud gpu. And boy, those are fucking expensive. Been researching what I need to stand up a service the last few days and it isn't cost friendly.
  - minimaxir 4y ago
    There is work on a CoreML version which may play nicer with older Macs w/sufficiently beefy dGPUs. https://github.com/huggingface/diffusers/issues/443 https://github.com/huggingface/diffusers/issues/443
    - osener 4y ago
      Will the CoreML version run faster than https://replicate.com/blog/run-stable-diffusion-on-m1-mac https://replicate.com/blog/run-stable-diffusion-on-m1-mac on an M1 Mac?
  - fragmede 4y ago
    Apple's MPS drivers supports AMD GPUs on MacOS
    - geuis 4y ago
      Just heard about MPS in another thread.
  - 4y ago
cloudking 4y ago
How does copyright work with output images? If someone runs the model on their own hardware, do they "own" the images generated? If 2 people generate the same image using the same prompt/seed, who "owns" the image?
- skipnup 4y ago
  Is it any different than Photoshop content aware fill? Or using a camera? Nobody would ever think about Adobe or Nikon having copyright claims over your pictures. For me it's just a tool, the artistic part is providing a good description/base image, refining and choosing the best output. Anyway, I'm not a lawyer and we probably live in different countries, so it'll be interesting to wait for the first lawsuit.
  - peoplefromibiza 4y ago
    > Nobody would ever think about Adobe or Nikon having copyright claims over your pictures But it is illegal to share pictures of Eiffel Tower, for example. People do it, but they shouldn't. If I put a picture of Eiffel Tower at night in a book or any other kind of commercial product, I have to pay to use it. Doesn't matter that it's there for my eyes to see it. The question is: are the images generated by an hyper accelerated learning machine using copyrighted material without the author's consent legal? I think they shouldn't be and the data included in the training should be free or licensed.
    - rkeene2 4y ago
      What if Stable Diffusion generates those images ? :-)
    - cypress66 4y ago
      That's actually just French copyright Law, has never been challenged I believe, and applies to the night lights only.
- cthalupa 4y ago
  https://www.smithsonianmag.com/smart-news/us-copyright-office-rules-ai-art-cant-be-copyrighted-180979808/ https://www.smithsonianmag.com/smart-news/us-copyright-offic... In the US, AI generated art cannot be copyrighted. Edit: Some additional details. The US also denied a copyright for one where the creator listed themselves, with the AI just being a co-creator. https://www.reddit.com/r/COPYRIGHT/comments/vshypc/the_us_copyright_office_on_june_29_2022_rejected/ https://www.reddit.com/r/COPYRIGHT/comments/vshypc/the_us_co... (Original article is paywalled, reddit post contains the relevant bits) In particular: >“Even though you argue that there is some human creative input present in the work that is distinct from RAGHAV’s contribution, this human authorship cannot be distinguished or separated from the final work produced by the computer program,” the office stated. The US does seem to be a bit of an outlier here. The above work was granted copyright in Canada and India. In the EU, AI generated artwork is likely copyrightable: https://link.springer.com/article/10.1007/s40319-021-01115-0 https://link.springer.com/article/10.1007/s40319-021-01115-0 The same for the UK: https://www.kilburnstrode.com/knowledge/ai/ai-musings/response-to-the-second-ai-consultation-status-quo https://www.kilburnstrode.com/knowledge/ai/ai-musings/respon... Edit2: I'm not a lawyer, this isn't legal advice, go contact one if you actually need legal advice here.
notfed 4y ago
Looks great but I use Linux and the README is fairly Windows-centric without warning. It'd be nice if there were clearly deliniated sections for Windows vs *nix. There's a (very ironically named) "Manual installation" section which might seem to be the answer for Linux, but then it's not immediately obvious which preceding sections are Linux or Windows without doing critical thinking.
cthalupa 4y ago
I've been having a lot of fun with Stable Diffusion and Midjourney. One thing that is very powerful with Stable Diffusion is using text inversion ( https://textual-inversion.github.io/ https://textual-inversion.github.io/ ) - you can add additional input samples to further extend the possibilities beyond what is included in the original model.
- adammarples 4y ago
  Can I run (train?) this textual inversion using the same consumer GPUs that work with stable-diffusion? Or does it require a much beefier machine
  - cthalupa 4y ago
    You can, though you might run into memory limitations running it on a GPU. There can be tuning done to lower the VRAM utilization, but I have been lucky enough to not need this - I do some CG work and ran into VRAM limitations there, so I'm on a 3090 with 24GB. You can always run it on a CPU and utilize your RAM instead if needed, though the training might extend to 24+ hours that way. Edit: Here's an example of someone successfully using textual inversion - https://www.reddit.com/r/StableDiffusion/comments/wz88lg/i_got_stable_diffusion_to_generate_competentish/ https://www.reddit.com/r/StableDiffusion/comments/wz88lg/i_g...
    - adammarples 4y ago
      Thanks
Metricon 4y ago
Another free offline and easy to use model with a GUI can be downloaded from here: https://grisk.itch.io/stable-diffusion-gui https://grisk.itch.io/stable-diffusion-gui For some genuinely incredible results try this pattern for instruction: Portrait of {Name of some type of identity such as "Faerie Princess" or "Dragon Queen"} {Name of a celebrity such as "Scarlett Johansson"}, beautiful face, symmetrical face, tone mapped, intricate, elegant, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and Greg Rutkowski and Alphonse Mucha and Boris Vallejo and Johannes Voss and Aleksi Briclot and Michael Komarck. Run several iterations of the same query as some results will have anomalies.
fomine3 4y ago
nit: seekbar is terrible UI to select exact large value, like resolution. I don't know why is it accepted.
underlines 4y ago
Nice to know tricks for /sd-webui/ - activate advanced: create prompt matrix and use @a painting of a (forest|desert|swamp|island|plains) painted by (claude monet|greg rutkowski|thomas kinkade) - add different relative weights for words in a prompt: watercolor :0.5 painting :0.2 by picasso :0.3 - Generate much larger images with your limited vram by using optimized versions of attention.py and model.py https://github.com/sd-webui/stable-diffusion-webui/discussions/795 https://github.com/sd-webui/stable-diffusion-webui/discussio... - Generate "Loab the AI haunting woman" if you can (Try using textual inversion with negatively weighted prompts) https://www.cnet.com/science/what-is-loab-the-haunting-ai-art-woman-explained/ https://www.cnet.com/science/what-is-loab-the-haunting-ai-ar...
- underlines 4y ago
  - add GFPGAN to fix distorted faces https://github.com/sd-webui/stable-diffusion-webui/wiki/Installation#gfpgan https://github.com/sd-webui/stable-diffusion-webui/wiki/Inst... - add RealESRGAN for better upscaling https://github.com/sd-webui/stable-diffusion-webui/wiki/Installation#realesrgan https://github.com/sd-webui/stable-diffusion-webui/wiki/Inst... - add LDSR for crazy good upscaling (for 10x the processing time) https://github.com/sd-webui/stable-diffusion-webui/wiki/Installation#ldsr https://github.com/sd-webui/stable-diffusion-webui/wiki/Inst...

284 points by feross 4y ago

This is the one I've been using https://github.com/sd-webui/stable-diffusion-webui https://github.com/sd-webui/stable-diffusion-webui . docker-compose up , works great.

TekMol 4y ago
Is there a way to run this in the cloud? On Google Colab or elsewhere?
- fragmede 4y ago
  Colab: https://colab.research.google.com/github/WASasquatch/StableDiffusionUI-TomRiddle/blob/main/StableDiffusionUI_Tom_Riddle_Edition.ipynb https://colab.research.google.com/github/WASasquatch/StableD... To run it elsewhere in the cloud, grab a GPU (spot) instance and SSH in.
  - TekMol 4y ago
    You mean a VM on a machine with a GPU? Or does it have to be a bare metal machine? What is a good provider of suitable VMs/machines? And what do you do after you SSHed in? The installation instructions seem to be for windows users (click here, then click there ...) is there a linux script that does the installation automatically?
    - fragmede 4y ago
      Just a VM with a GPU, doesn't need to be bare metal. AWS/GCP/Azure has em, but for GPU cloud instances, boutique vendors like CoreWeave, runpod, lambdalabs.com, vast.ai, paperspace may be more competitive. Parent comment alludes to docker-compose up.
      - TekMol 4y ago
        You can run docker inside a VM? And it will be able to use the GPU and tunnel port 80 through the docker container, through the VM and to the web?
        fragmede 4y ago
        There's some trickery and details, but yes: https://github.com/NVIDIA/nvidia-docker https://github.com/NVIDIA/nvidia-docker
    - mnahkies 4y ago
      You can also get windows instances with GPU attached and RDP in, though personally I prefer Linux https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/accelerated-computing-instances.html#gpu-instances https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/accel...
    - fragmede 4y ago
      Here's an install script if you need handholding :) https://github.com/JoshuaKimsey/Linux-StableDiffusion-Script/blob/main/linux-sd.sh https://github.com/JoshuaKimsey/Linux-StableDiffusion-Script...
  - danuker 4y ago
    Lambda Labs is probably cheaper: https://lambdalabs.com/service/gpu-cloud https://lambdalabs.com/service/gpu-cloud
stbtrax 4y ago
wyldfire 4y ago
I like this one but had some trouble with using img2img. Maybe my image was too small (it was smaller than 512x512). Failed with the same signature as an issue that was closed with a fix.
- ghilston 4y ago
  I am mobile, but there's an issue reported on github about img2img
jyrkesh 4y ago
I've also been using this one (wasn't sure at first, they just migrated from the /hlky/ namespace on GitHub), but I have no idea at first glance what the differences are. I will say that this one has had REALLY active development as new features have been coming out, and is pretty polished at this point (albeit I'm using it more as a toy than anything, but it's awesome to have a quick way to use the new features that have been shipping out).
KronisLV 4y ago
Seems like it fails if you have an AMD GPU instead of an Nvidia one (at least that's my guess, based on the error contents): ERROR: for stable-diffusion Cannot start service stable-diffusion: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown
- mach1ne 4y ago
  Indeed, Stable Diffusion does not currently run on AMD graphics cards.
  - btdmaster 4y ago
    Some people have been able to run on recent AMD cards with rocm: https://github.com/CompVis/stable-diffusion/issues/48 https://github.com/CompVis/stable-diffusion/issues/48
  - CapsAdmin 4y ago
    I'm able to run it on my rx6900 xt on Linux. I tried using a docker container and it took 3 min to generate a prompt. However it seems that 2:45 min is somehow spent on tje GPU and finally the remaining 15 seconds the GPU gets utilized. I haven't had the time to look into this yet, but it does seem to work.
  - fragmede 4y ago
    It does, on Linux, Windows, and MacOS.
    - needle0 4y ago
      I've gotten it running with a Radeon RX 6800 on Ubuntu Linux 22.04 (with overwriting PyTorch with a ROCm-supporting version), and on Windows 10 (in a very barebones way using ONNX), but are there better, more full-featured ways to get it running on Windows? Would love to know.

sharps_xp 4y ago

if someone can dockerize this, please reply with a link!

stephanheijl 4y ago
This is the dockerized version of this repo: https://github.com/AbdBarho/stable-diffusion-webui-docker https://github.com/AbdBarho/stable-diffusion-webui-docker
geerlingguy 4y ago
The main one I've been testing (https://github.com/sd-webui/stable-diffusion-webui https://github.com/sd-webui/stable-diffusion-webui) has a Docker Compose file. Grab that, install Docker, install Nvidia's Docker integration, copy the example Docker-env file, and docker-compose up is all you need. Edit: here's a gist with exact steps I used: https://gist.github.com/geerlingguy/384ed4aba35e3118f2a0f358fc974162 https://gist.github.com/geerlingguy/384ed4aba35e3118f2a0f358...

user-one1 4y ago

It can be run directly into google colab: https://colab.research.google.com/drive/1Iy-xW9t1-OQWhb0hNxueGij8phCyluOh https://colab.research.google.com/drive/1Iy-xW9t1-OQWhb0hNxu...

TekMol 4y ago
When I run it, I get "Your session crashed for an unknown reason."
- fragmede 4y ago
  Just click runtime -> run all again. There’s a weirdness where Python’s loader gets confused and the most effective fix is to crash the interpreter

smrtinsert 4y ago

I have a 6gb 1660ti, barely holding on. Is a new 12gb card good enough for now, or should I go even higher to be safe for a few years of sd innovation?

wyldfire 4y ago
It sounds like there's forks that are able to work with <=8GB cards. And I'm not sure but I think the weights are using f32, so switching to half might make it yet easier still to get this to work w/less memory. But yeah the next generation of models would probably capitalize on more memory somehow.
- filiphorvat 4y ago
  People have reported that this repo even works with 2gb cards if you run it with --lowvram and --opt-split-attention.
  - password4321 4y ago
    Yes, the amount of VRAM doesn't seem to be as much of a limitation anymore. However, processing power is still important.
fassssst 4y ago
The GeForce 4000 series is about to release and should make Stable Diffusion wayyyyy faster based on related H100 benchmarks posted today.
drexlspivey 4y ago
How is M1/M2 support for SD? Is there a significant performance drop? Presumably you would be able to buy a 32GB M2 and be future proof because of the shared memory between CPU/GPU.
- jjcon 4y ago
  In my setup at least it runs essentially in CPU mode since there is no CUDA acceleration available and metal support is really messy right now. So while quite slow I don't run into memory issues at least. It runs much faster on my desktop GPU but that has more constraints (until I upgrade my personal 1080 to a 3090 one of these days).

hwers 4y ago

People recently figured out how to export stable diffusion to onnx so it’ll be exciting to see some actual web UIs for it soon (via quantized models and tfjs/onnxruntime for web)

password4321 4y ago
Very cool! Can you link to where this is taking place? A commenter mentioned today it might be possible to pre-download the model and load it into the browser from the local filesystem rather than include such a gigantic blob as an accompanying dependency, fighting different caching RFC's, security/usage restrictions, and anything else that might inadvertently trigger a re-download. https://news.ycombinator.com/item?id=32777909#32779093 https://news.ycombinator.com/item?id=32777909#32779093
- minimaxir 4y ago
  Support for ONNX export was just added to diffusers, but no runtime logic for scheduling yet. https://github.com/huggingface/diffusers https://github.com/huggingface/diffusers

ducktective 4y ago

It seems Midjourney generates better results than SD or Dall-E. What's with the "hyper resolution", "4K, detailed" adjectives which are thrown left and right, while we are at it?

Ckalegi 4y ago
The metadata and file names of the images in the source data set are also inputs for the model training. These keywords are common tags across images that have these characteristics, so in the same way it knows what a unicorn looks like, it also knows what a 4k unicorn looks like compared to a hyper rez unicorn.
schleck8 4y ago
Those are prompt engineering keywords. SD is way more reliant on tinkering with the prompt than midjourney https://moritz.pm/posts/parameters https://moritz.pm/posts/parameters
- throwaway1851 4y ago
  MidJourney needs a lot of prompt engineering too. And Dall-E also. If you look at the prompt as an opportunity to describe what you want to see, the results are often disappointing. It works better to think backwards about how the model was trained, and what sorts of web caption words it likely saw in training examples that used the sorts of features you’re hoping it will generate. This is more of a process of learning to ask the model to produce things it’s able to produce, using its special image language.
cma 4y ago
Midjourney uses SD under the hood (you can see in their license), but they augnment the model in various ways.
- qudat 4y ago
  The results in midjourney are significantly better than SD. I find it much easier to get to a good result in MJ and I've been trying to understand why. Anymore insight you could share?

jrm4 4y ago

So, and this is an ELI5 kind of question I suppose. There must be something going on like "processing a kazillion images" and I'm trying to wrap my head around how (or what part of) that work is "offloaded" to your home computer/graphics card? I just can't seem to make sense of how you can do it at home if you're not somehow in direct contact with "all the data?" e.g. must you be connected to the internet, or "stable-diffusions servers" for this to work?

codefined 4y ago
All those 'kazillion' images are processed into a single 'model'. Similar to how our brain cannot remember 100% of all our experiences, this model will not store precise copies of all images it is trained off of. However, it will understand concepts, such as what a unicorn looks like. For StableDiffusion, the current model is ~4GB, which is downloaded the first time you run the model. These 4GB encode all the information that the model requires to derive your images.
ducktective 4y ago
As someone with ~0 knowledge in this field, I think this has to do with a concept called "transfer learning" in which you once train with that kazillion of images, then use that same "coefficients" for further run of the NN.
- palunon 4y ago
  Nah, transfer learning is when you take a trained model, and train it a little more to better fit your (potentially very different) problem domain. Such as training a cat/dog/etc recognition model on MRI scans. The goal is usually to have the more fundamental parts of your model already working and you thus need way less domain specific data. Here, you're not training anything, you're running the models (both the CLIP language model and the unet) in feedforward. That's just deploying your model, not transfer learning.
sC3DX 4y ago
What you interact with as the user is the model and its weights. The model (presumably some kind of convolutional neural network) has many layers, every layer has some set of nodes, and every node has a weight, which is just some coefficient. The weights are 'learned' during the model training where the model takes in the data you mention and evaluates the output. This typically happens on a super beefy computer and can take a long time for a model like this. As images are evaluated the output gets better the weights get adjusted accordingly. Now we as the user just need the model and the weights!

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

amelius 4y ago

Regarding the opening image: if it can't correctly put the marks on dice, how can it put eyes, nose and mouth correctly on a human face?

bastawhiz 4y ago
Presumably the number of faces in the training set far exceeds the number of dice by more than a few orders of magnitude.
- amelius 4y ago
  In one of the other posts I noticed this option: > GFPGAN Face Correction: Automatically correct distorted faces with a built-in GFPGAN option, fixes them in less than half a second So apparently there is still an issue with faces.
  - SturgeonsLaw 4y ago
    Yep, it gets faces mostly right, but as they say, the devil's in the details. Eyes in particular don't seem to have clearly delineated concentric circles for irises and pupils, instead they are often rendered as a "swirl". Example image directly from Stable Diffusion: https://i.imgur.com/XSk8fIv.png https://i.imgur.com/XSk8fIv.png And here is that image run through GFPGAN: https://i.imgur.com/I53AGmh.png https://i.imgur.com/I53AGmh.png Interesting to note how specialised GFPGAN is, as some of the other details (flowers, hair) seem to be worse in the processed image. I plan to finish this image by manually blending the best of both pictures.
    - userbinator 4y ago
      Ironically, that almost looks like the sort of fantasy image you'd find as GPU box art many years ago.
eminence32 4y ago
In my experience, Stable Diffusion is also pretty bad about human faces and hands too

fibonacc 4y ago

a sigh of relief as I thought that this would generate instead of pictures, React UI code based on plain text description. First they came for illustrators, then they came for UI designers.

MattRix 4y ago
heh yeah it won’t be that long until we have A Stable Diffusion for Web UI
fragmede 4y ago
You've seen https://twitter.com/sharifshameem/status/1282676454690451457 https://twitter.com/sharifshameem/status/1282676454690451457 yeah?
- fibonacc 4y ago
  yes what I mean was mixing stable diffusion into the UI styling without the fine atomic details. Like say I want a dashboard with color themes from cyberpunk complete with graphics/art/logo. It would be a "good enough" category if something can literally be told to produce a complete frontend.

offsky 4y ago

I’m waiting for someone to wrap this up into a desktop app that I can install and run on my Mac.

geuis 4y ago
I've been looking into this for the last 2 days. Unless you're running an M1 Mac or newer, you're SOL. Stable Diffusion is built on PyTorch. PyTorch mainly has been designed to work with Nvidia cards. However PyTorch added support for something called RocM like a year ago that adds compatibility with newer AMD cards. Unfortunately RocM doesn't support slightly older AMD cards in conjunction with intel processors. So my 32gb pretty powerful 2020 16in MacBook Pro isn't capable of running Stable Diffusion. Any native app will likely have to rely on a remote cloud gpu. And boy, those are fucking expensive. Been researching what I need to stand up a service the last few days and it isn't cost friendly.
- minimaxir 4y ago
  There is work on a CoreML version which may play nicer with older Macs w/sufficiently beefy dGPUs. https://github.com/huggingface/diffusers/issues/443 https://github.com/huggingface/diffusers/issues/443
  - osener 4y ago
    Will the CoreML version run faster than https://replicate.com/blog/run-stable-diffusion-on-m1-mac https://replicate.com/blog/run-stable-diffusion-on-m1-mac on an M1 Mac?
- fragmede 4y ago
  Apple's MPS drivers supports AMD GPUs on MacOS
  - geuis 4y ago
    Just heard about MPS in another thread.
- 4y ago

cloudking 4y ago

How does copyright work with output images? If someone runs the model on their own hardware, do they "own" the images generated? If 2 people generate the same image using the same prompt/seed, who "owns" the image?

skipnup 4y ago
Is it any different than Photoshop content aware fill? Or using a camera? Nobody would ever think about Adobe or Nikon having copyright claims over your pictures. For me it's just a tool, the artistic part is providing a good description/base image, refining and choosing the best output. Anyway, I'm not a lawyer and we probably live in different countries, so it'll be interesting to wait for the first lawsuit.
- peoplefromibiza 4y ago
  > Nobody would ever think about Adobe or Nikon having copyright claims over your pictures But it is illegal to share pictures of Eiffel Tower, for example. People do it, but they shouldn't. If I put a picture of Eiffel Tower at night in a book or any other kind of commercial product, I have to pay to use it. Doesn't matter that it's there for my eyes to see it. The question is: are the images generated by an hyper accelerated learning machine using copyrighted material without the author's consent legal? I think they shouldn't be and the data included in the training should be free or licensed.
  - rkeene2 4y ago
    What if Stable Diffusion generates those images ? :-)
  - cypress66 4y ago
    That's actually just French copyright Law, has never been challenged I believe, and applies to the night lights only.
cthalupa 4y ago
https://www.smithsonianmag.com/smart-news/us-copyright-office-rules-ai-art-cant-be-copyrighted-180979808/ https://www.smithsonianmag.com/smart-news/us-copyright-offic... In the US, AI generated art cannot be copyrighted. Edit: Some additional details. The US also denied a copyright for one where the creator listed themselves, with the AI just being a co-creator. https://www.reddit.com/r/COPYRIGHT/comments/vshypc/the_us_copyright_office_on_june_29_2022_rejected/ https://www.reddit.com/r/COPYRIGHT/comments/vshypc/the_us_co... (Original article is paywalled, reddit post contains the relevant bits) In particular: >“Even though you argue that there is some human creative input present in the work that is distinct from RAGHAV’s contribution, this human authorship cannot be distinguished or separated from the final work produced by the computer program,” the office stated. The US does seem to be a bit of an outlier here. The above work was granted copyright in Canada and India. In the EU, AI generated artwork is likely copyrightable: https://link.springer.com/article/10.1007/s40319-021-01115-0 https://link.springer.com/article/10.1007/s40319-021-01115-0 The same for the UK: https://www.kilburnstrode.com/knowledge/ai/ai-musings/response-to-the-second-ai-consultation-status-quo https://www.kilburnstrode.com/knowledge/ai/ai-musings/respon... Edit2: I'm not a lawyer, this isn't legal advice, go contact one if you actually need legal advice here.

notfed 4y ago

Looks great but I use Linux and the README is fairly Windows-centric without warning. It'd be nice if there were clearly deliniated sections for Windows vs *nix. There's a (very ironically named) "Manual installation" section which might seem to be the answer for Linux, but then it's not immediately obvious which preceding sections are Linux or Windows without doing critical thinking.

cthalupa 4y ago

I've been having a lot of fun with Stable Diffusion and Midjourney. One thing that is very powerful with Stable Diffusion is using text inversion ( https://textual-inversion.github.io/ https://textual-inversion.github.io/ ) - you can add additional input samples to further extend the possibilities beyond what is included in the original model.

adammarples 4y ago
Can I run (train?) this textual inversion using the same consumer GPUs that work with stable-diffusion? Or does it require a much beefier machine
- cthalupa 4y ago
  You can, though you might run into memory limitations running it on a GPU. There can be tuning done to lower the VRAM utilization, but I have been lucky enough to not need this - I do some CG work and ran into VRAM limitations there, so I'm on a 3090 with 24GB. You can always run it on a CPU and utilize your RAM instead if needed, though the training might extend to 24+ hours that way. Edit: Here's an example of someone successfully using textual inversion - https://www.reddit.com/r/StableDiffusion/comments/wz88lg/i_got_stable_diffusion_to_generate_competentish/ https://www.reddit.com/r/StableDiffusion/comments/wz88lg/i_g...
  - adammarples 4y ago
    Thanks

Metricon 4y ago

Another free offline and easy to use model with a GUI can be downloaded from here: https://grisk.itch.io/stable-diffusion-gui https://grisk.itch.io/stable-diffusion-gui For some genuinely incredible results try this pattern for instruction: Portrait of {Name of some type of identity such as "Faerie Princess" or "Dragon Queen"} {Name of a celebrity such as "Scarlett Johansson"}, beautiful face, symmetrical face, tone mapped, intricate, elegant, highly detailed, digital painting, artstation, concept art, smooth, sharp focus, illustration, art by artgerm and Greg Rutkowski and Alphonse Mucha and Boris Vallejo and Johannes Voss and Aleksi Briclot and Michael Komarck. Run several iterations of the same query as some results will have anomalies.

fomine3 4y ago

nit: seekbar is terrible UI to select exact large value, like resolution. I don't know why is it accepted.

underlines 4y ago

Nice to know tricks for /sd-webui/ - activate advanced: create prompt matrix and use @a painting of a (forest|desert|swamp|island|plains) painted by (claude monet|greg rutkowski|thomas kinkade) - add different relative weights for words in a prompt: watercolor :0.5 painting :0.2 by picasso :0.3 - Generate much larger images with your limited vram by using optimized versions of attention.py and model.py https://github.com/sd-webui/stable-diffusion-webui/discussions/795 https://github.com/sd-webui/stable-diffusion-webui/discussio... - Generate "Loab the AI haunting woman" if you can (Try using textual inversion with negatively weighted prompts) https://www.cnet.com/science/what-is-loab-the-haunting-ai-art-woman-explained/ https://www.cnet.com/science/what-is-loab-the-haunting-ai-ar...

underlines 4y ago
- add GFPGAN to fix distorted faces https://github.com/sd-webui/stable-diffusion-webui/wiki/Installation#gfpgan https://github.com/sd-webui/stable-diffusion-webui/wiki/Inst... - add RealESRGAN for better upscaling https://github.com/sd-webui/stable-diffusion-webui/wiki/Installation#realesrgan https://github.com/sd-webui/stable-diffusion-webui/wiki/Inst... - add LDSR for crazy good upscaling (for 10x the processing time) https://github.com/sd-webui/stable-diffusion-webui/wiki/Installation#ldsr https://github.com/sd-webui/stable-diffusion-webui/wiki/Inst...