4 ms·
That's what this article is about, they're doing the integration work. An integration with a very popular tool is already available for use. I've personally t
by vhold 3y ago
That's what this article is about, they're doing the integration work.
An integration with a very popular tool is already available for use. I've personally tested it and did get a 2x speed up. I didn't have to sign-up for anything.
https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT
- brucethemoose2 3y agoUgh... I know everyone uses automatic, but I don't really like automatic. Is janky as heck, especially with SDXL, especially if you want to dig into the code and make some manual hacks. Still, you are right, a native a1111 extensions is quite impressive and exactly the kind of thing Nvidia should pursue.
- qiine 3y agoWhat do you use instead? confyUI ?
- nickthegreek 3y agoComfyUI or fooocus handle SDXL way better than A1111 with 10gb VRAM or under. While I really like A1111, it just isnt usable with XL for me.
- brucethemoose2 3y agoI run them on a 3090, and fooocus is still way better.
- brucethemoose2 3y agoFooocus MRE for SDXL. The quality is just so much better than a1111 or vlad's fork. And its stable, with a good UI: https://github.com/MoonRide303/Fooocus-MRE/commits/moonride-main https://github.com/MoonRide303/Fooocus-MRE/commits/moonride-... I use VoltaML for SD 1.5: https://github.com/VoltaML/voltaML-fast-stable-diffusion/tree/experimental https://github.com/VoltaML/voltaML-fast-stable-diffusion/tre... Next on my todo list is trying ComfyUI with AIT: https://github.com/FizzleDorf/ComfyUI-AIT https://github.com/FizzleDorf/ComfyUI-AIT
- qiine 3y agoso many projects... its overwhelming
- brucethemoose2 3y agoFYI I just learned that Fooocus-MRE has been depreciated, but many of the same options are now in the debug menu upstream. ...Yes, it is impossible to keep up, lol
- diggan 3y agodiffusers also has example usage for TensorRT in the Stable Diffusion context: https://github.com/huggingface/diffusers/blob/85dccab7fd280cf085cc8007decc458d5fe0f032/examples/community/stable_diffusion_tensorrt_txt2img.py https://github.com/huggingface/diffusers/blob/85dccab7fd280c... Seems those particular examples were submitted by NVIDIA employee too: https://github.com/huggingface/diffusers/commit/9d44e2fb6600e80f410b2c05139c001fb0fa9794 https://github.com/huggingface/diffusers/commit/9d44e2fb6600... Shouldn't be too hard to adopt to other similar pipelines if you've used diffusers before.
- brucethemoose2 3y agoThat's a good example of a "hit and run" demo. None of the community Diffusers pipelines I know of have successfully integrated this. VoltaML made a valiant effort and gave up in favor of Facebooks AITemplate (for now).
- diggan 3y agoWorked for me and was easy to integrate, but I use diffusers directly. Not sure what the others are doing, but if you can manage to implement your own pipeline with diffusers, you should be able to integrate the TensorRT pipeline with relative ease. https://github.com/VoltaML/voltaML https://github.com/VoltaML/voltaML seems to have successfully integrated it as far as can tell, it says TensorRT is one of the compatible backends.
- brucethemoose2 3y agoVoltaML abandoned it (but may revisit it) and replaced it with AIT. The problem with a vanilla diffusers pipe is that you miss out on all the cool community augmentations that really make sd shine. Crazy img2img stuff aside, some random examples include: - Fooocus's implementation preserves 'momentum' when switching from the sdxl base model to the refiner. - freeu augmentation - tomesd - fooocus image style presets, with known styles from the dataset strait from SAI and others. - auto prompt expansion - performance hacks not in vanilla diffusers