3 ms·
Is there somewhere I can see your Stable Diffusion + torch.compile code? I am interesting in how you integrated.
by voz_ 4y ago
Is there somewhere I can see your Stable Diffusion + torch.compile code? I am interesting in how you integrated.
- brucethemoose2 4y agoIn `diffusers` implementations (like InvokeAI) its pretty easy: https://github.com/huggingface/diffusers/blob/42beaf1d23b5ccb5db18f3f1e7555918ce40434c/docs/source/en/optimization/torch2.0.mdx#using-accelerated-transformers-and-torchcompile https://github.com/huggingface/diffusers/blob/42beaf1d23b5cc... But I also compile the VAE and some other modules, I will reply again later when I can look at my local code. Some modules (like face restoration or the scheduler) still dont like torch.compile. For the Automatic1111 repo (and presumably other original Stability AI implementations), I just add `m.model = torch.compile(m.model)` here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/sd_hijack.py#L184 https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob... I tried changing the options in the config dict one by one, but TBH nothing seems to make a significant difference behind the default settings in benchmarks. I haven't messed with compiling LORA training yet, as I dont train much and it is sufficiently fast, but I'm sure it could be done.
- brucethemoose2 4y agoHere is the InvokeAI code, minus the codeformer/gfpgan changes that dont work yet: https://gist.github.com/brucethemoose/ea64f498b0aa51adcc88f543bf80ed97 https://gist.github.com/brucethemoose/ea64f498b0aa51adcc88f5... I intend to start some issues for this on the repo soon(TM).