4 ms·
SD 1.5 is 983m parameters, SDXL is 3.5b, for reference. Very interesting. I've been streching my 12GB 3060 as far as I can; it's exciting that smaller hardware
by cheald 3y ago
SD 1.5 is 983m parameters, SDXL is 3.5b, for reference.
Very interesting. I've been streching my 12GB 3060 as far as I can; it's exciting that smaller hardware is still usable even with modern improvements.
- memossy 3y ago800m is good for mobile, 8b for graphics cards. Bigger than that is also possible, not saturated yet but need more GPUs.
- vorticalbox 3y agoyou ca also quantisation which lowers memory requirements at a small lose of performance.
- anon373839 3y agoDo you know how the memory demands compare to LLMs at the same number of parameters? For example, Mistral 7B quantized to 4 bits works very well on an 8GB card, though there isn’t room for long context.
- liuliu 3y agoI am going to look at quantization for 8b. But also, these are transformers, so variety of merging / Frankenstein-tune is possible. For example, you can use 8b model to populate the KV cache (which computes once, so can load from slower devices, such as RAM / SSD) and use 800M model for diffusion by replicating weights to match layers of the 8b model.
- ttul 3y agoStability has to make money somehow. By releasing an 8B parameter model, they’re encouraging people to use their paid API for inference. It’s not a terrible business decision. And hobbyists can play with the smaller models, which with some refining will probably be just fine for most non-professional use cases.
- jandrese 3y agoI would LOL if they released the "safe" model for free but made you pay for the one with boobs.
- ttul 3y agoOh they’ll never let you pay for porn generation. But they will happily entertain having you pay for quality commercial images that are basically a replacement for the entire graphic design industry.
- ohthehugemanate 3y agoIt's not an easy fap, but I guess I'm watching people get f*cked either way.
- teaearlgraycold 3y agoDon't people quantize SD down to 8 bits? I understand plenty of people don't have 8GB of VRAM (and I suppose you need some extra for supplemental data, so maybe 10GB?). But that's still well within the realm of consumer hardware capabilities.
- ttul 3y agoI’m the wrong person to ask, but it seems Stability intends to offer models from 800M to 8B parameters in size, which offers something for everyone.