4 ms·
This is pretty big news when you look at the numbers floating around time to train AIs like Stable Diffusion - which is by far the most disruptive change that's
by XorNot 4y ago
This is pretty big news when you look at the numbers floating around time to train AIs like Stable Diffusion - which is by far the most disruptive change that's happened recently. 150,000 hours just went down to 100,000 hours if we take the conservative estimate.
Not fast enough (not at the price point) but getting there: of course since it's an entirely parallel problem, the real metric is cost-per-model since in reality you'd just buy time from the cloud.
I figure we're going to start seeing some big changes once cost-per-model puts it within reach of the hobbyist. If I today could buy a custom model for say, $1000 - that's accessible to the hobbyist experimentalist.
- justatdotin 4y ago> by far the most disruptive change that has been disclosed :/
- colordrops 4y agoIt's disruptive because it has been disclosed.
- dannyw 4y agoEven $10k is enough for a hobbyist group or crowd funding.
- ipsum2 4y agoStable diffusion is on the order of millions of dollars.
- tehsauce 4y agoIt was about $200k in compute.
- moonshotideas 4y agoIs there a breakdown of the costs available to view?
- homarp 4y agohttps://twitter.com/emostaque/status/1563870674111832066 https://twitter.com/emostaque/status/1563870674111832066 says "We actually used 256 A100s for this per the model card, 150k hours in total so at market price $600k"
- godmode2019 4y agoBut they still have those A100s? I think you have to compare it to how much ETH you could have mined as the opportunity cost of training a model vs other uses of a GPU
- l33tman 4y agoI think it was donated by AWS or something. This is not a scenario where it makes sense to buy and maintain your own server park of GPUs, when there is a race to the bottom already with cloud-based GPU rentals.
- homarp 4y agoAlso compare the impact of releasing Stable Diffusion vs 'yet another coin'/web3 announcement.
- homarp 4y agohttps://www.forbes.com/sites/kenrickcai/2022/09/07/stability-ai-funding-round-1-billion-valuation-stable-diffusion-text-to-image/ https://www.forbes.com/sites/kenrickcai/2022/09/07/stability...
- Ajedi32 4y agoCrowd funding models seems especially interesting, since in theory you could fully automate the process. Submit the dataset and code you want to train with, then once it's funded the service automatically runs the training and releases the resulting model to backers.
- macrolime 4y agoSuch a platform should include a cloud to run the model, where backers would share the revenue earned from people using the model through an API.
- 01100011 4y agoObviously perf and efficiency would be terrible, but could you distribute the training of a diffusion model across volunteers' computers? I would think that, for many fangroups, you'd have a lot of folks who would like to customize a diffusion model for their favorite characters and/or art.
- sanxiyn 4y agoLikely yes, it already has been done for DALL-E. Have a look at Training Transformers Together for what is involved: https://arxiv.org/abs/2207.03481 https://arxiv.org/abs/2207.03481.