39 ms·

Woaaa love these inputs. Thank you! Wasn't aware of the JAX backend will check it out. Right now we're on SD 1.5. We tried SDXL but found the quality improvemen

Woaaa love these inputs. Thank you! Wasn't aware of the JAX backend will check it out. Right now we're on SD 1.5. We tried SDXL but found the quality improvement to be marginal. Yes to area prompting/regional control to help people create more complex scenes. I need some design thinking first since it's easy to over build and spit out something super complicated. Immediate next step is to def add controlnet.

brucethemoose2 3y ago
> JAX Yeah, check out their post: https://huggingface.co/blog/sdxl_jax https://huggingface.co/blog/sdxl_jax I dunno how expensive TPU instances are these days, but the performance is insane! > We tried SDXL but found the quality improvement to be marginal. Yeah, the vanilla HF diffusers pipe is unimpressive to me. Try playing with this though, turn on FreeU and specify an anime style: https://github.com/MoonRide303/Fooocus-MRE https://github.com/MoonRide303/Fooocus-MRE I have never gotten such high quality results from simple prompts, even in cloud models like Midjourney/GPT4. The question is how to port even part of that magic over to the diffusers pipeline...
brucethemoose2 3y ago
Also, VoltaML has a good reference GPU AITemplate SD 1.5 implementation: https://github.com/VoltaML/voltaML-fast-stable-diffusion/tree/experimental https://github.com/VoltaML/voltaML-fast-stable-diffusion/tre... The speed jump is massive on my desktop GPU, probably even more dramatic on cloud hardware, and it may support some things (weight swapping/lora swapping/resolution changing/controlnet) better than JAX.
- Fengjiao 3y ago
  My issue previously with these prebuilt backends is that you can't tweak it like sdwebui does, but to make our thing work it took a thousand tweaks. Can look into this first to see how customizable it is.
  - brucethemoose2 3y ago
    VoltaML is a relatively vanilla diffusers-based backend, so its not a hairy monster to hack like you may have seen with SAI-based UIs (like Comfy, Fooocus and Automatic) The AITTemplate code is a lightly modified version of Facebook's example dynamic AIT script, to get rid of small issues like VRAM spikes: https://github.com/facebookincubator/AITemplate/tree/main/examples/05_stable_diffusion https://github.com/facebookincubator/AITemplate/tree/main/ex... InvokeAI is also diffusers based, but they seem to mess with the pipeline a bit more. Anyway, all that may be better as a reference for interesting features rather than a backend to try and adopt.

brucethemoose2 3y ago

> JAX Yeah, check out their post: https://huggingface.co/blog/sdxl_jax https://huggingface.co/blog/sdxl_jax I dunno how expensive TPU instances are these days, but the performance is insane! > We tried SDXL but found the quality improvement to be marginal. Yeah, the vanilla HF diffusers pipe is unimpressive to me. Try playing with this though, turn on FreeU and specify an anime style: https://github.com/MoonRide303/Fooocus-MRE https://github.com/MoonRide303/Fooocus-MRE I have never gotten such high quality results from simple prompts, even in cloud models like Midjourney/GPT4. The question is how to port even part of that magic over to the diffusers pipeline...

brucethemoose2 3y ago

Also, VoltaML has a good reference GPU AITemplate SD 1.5 implementation: https://github.com/VoltaML/voltaML-fast-stable-diffusion/tree/experimental https://github.com/VoltaML/voltaML-fast-stable-diffusion/tre... The speed jump is massive on my desktop GPU, probably even more dramatic on cloud hardware, and it may support some things (weight swapping/lora swapping/resolution changing/controlnet) better than JAX.

Fengjiao 3y ago
My issue previously with these prebuilt backends is that you can't tweak it like sdwebui does, but to make our thing work it took a thousand tweaks. Can look into this first to see how customizable it is.
- brucethemoose2 3y ago
  VoltaML is a relatively vanilla diffusers-based backend, so its not a hairy monster to hack like you may have seen with SAI-based UIs (like Comfy, Fooocus and Automatic) The AITTemplate code is a lightly modified version of Facebook's example dynamic AIT script, to get rid of small issues like VRAM spikes: https://github.com/facebookincubator/AITemplate/tree/main/examples/05_stable_diffusion https://github.com/facebookincubator/AITemplate/tree/main/ex... InvokeAI is also diffusers based, but they seem to mess with the pipeline a bit more. Anyway, all that may be better as a reference for interesting features rather than a backend to try and adopt.