3 ms·
Since it is now split into two models to do the generation, you could load one and do the first stage of a bunch of images, then load the second and complete th
by cma 3y ago
Since it is now split into two models to do the generation, you could load one and do the first stage of a bunch of images, then load the second and complete them, with half the vram usage.
- ShamelessC 3y agoI believe the HF pipeline can do this already and I assume each stage uses more than 4 GB vram. There are other tricks they open source community will come up with though.
- brucethemoose2 3y ago8 bit (and 4 bit?) quantization is low hanging fruit, assuming its not already running in 8 bit. An 8GB requirement kinda sounds like they have already quantized the model though.