2 ms·
Should use no more than 6GiB for FP16 models at each stage. The current implementation is not RAM optimized.
by liuliu 3y ago
Should use no more than 6GiB for FP16 models at each stage. The current implementation is not RAM optimized.
- sorenjan 3y agoThe large C model uses 3.6 billion parameters which is 6.7 GiB if each parameter is 16 bits.
- liuliu 3y agoThe large C model have fair bit of parameters tied to text-conditioning, not to the main denoising process. Similar to how we split the network for SDXL Base, I am pretty confident we can split non-trivial amount of parameters to text-conditioning hence during denoising process, loading less than 3.6B parameters.
- brucethemoose2 3y agoWhat's more, they can presumably be swapped in and out like the SDXL base + refiner, right?