2 ms·
Diffusion is relatively compute intensive compared to transformers llms, and (in current implementation) doesn't quantize as well. A 70B parameter model would
by brucethemoose2 3y ago
Diffusion is relatively compute intensive compared to transformers llms, and (in current implementation) doesn't quantize as well.
A 70B parameter model would be very slow and vram hungry, hence very expensive to run.
Also, image generation is more reliant on tooling surrounding the models than pure text prompting. I dont think even a 300B model would get things quite right through text prompting alone.
- amilios 3y agoHmm this is a good point, diffusion requires several (many?) inference passes as you refine the noise into an image, right? Makes sense that this is more expensive to scale up. Thanks for the explanation!
- brucethemoose2 3y agoTechnically the llms require a pass for each token, but the passes are cheaper and benefit more from batching.