3 ms·
You can’t split image models over 2 GPUs like you can LLMs.
by Auracle 1y ago
You can’t split image models over 2 GPUs like you can LLMs.
- BoredPositron 1y agoThey also released an inference server for their models. Wan and qwen-image can be split without problems. https://github.com/modelscope/DiffSynth-Engine https://github.com/modelscope/DiffSynth-Engine
- Auracle 1y agoUnless I missed something just from skimming their tutorial it looks like they can do parallelism to speed things up with some models, not actually split the model (apart from the usual chunk offloading techniques).