4 ms·
The compute cost to train Stable Diffusion was $600k
- nalzok 4y ago> at market price $600k I don’t think they are really spending $600k. Chances are that they own a GPU cluster so they don’t need to pay the cloud premium.
- fragmede 4y agoSpot instance pricing for p4d.large is about a third the on-demand cost, though it's not clear what they're using to calculate market rate. An A100 card is roughly $10k retail, so it's not like "owning a GPU cloud" is that cheap a proposition either.
- Slix 4y agoCould training be crowd sourced among consumer GPUs like Folding at Home?
- angry-tempest 4y agoProbably not for a few years, you need a (maybe few) A100(s) to be able to backprop a model that big with float32.
- verdverm 4y agoiirc, they tweeted about using around 3800 in parallel
- jacooper 4y agoIs that relatively cheap or expensive? I have no context for this, do other model dveelopers publish how much it costed to train them?
- hnfong 4y agoGiven the impact the model might have on the world, and that $600k is roughly equal to the annual salary of 1~3 top notch engineers... it's probably intended to mean relatively cheap. For similar context, millions of VC money have been thrown around for much less impressive results. #firstworldproblems
- mpaepper 4y agoIf you want to learn how and why stable diffusion works: https://www.paepper.com/blog/posts/how-and-why-stable-diffusion-works-for-text-to-image-generation/ https://www.paepper.com/blog/posts/how-and-why-stable-diffus...