4 ms·

Stable Diffusion[0] is an extremely expensive SOTA text-to-image diffusion model developed by a private nonprofit and trained on a massive "Internet tarball" th

by mkaic 4y ago

Stable Diffusion[0] is an extremely expensive SOTA text-to-image diffusion model developed by a private nonprofit and trained on a massive "Internet tarball" that's about to have its weights shared open-source (the code and dataset are already open source). Not trying to invalidate your argument, just presenting a pleasant counterexample. I don't think I quite agree with your opinion that AI will remain undemocratized. [0]https://github.com/CompVis/stable-diffusion https://github.com/CompVis/stable-diffusion

benreesman 4y ago
I really enjoyed this interview [1] with Emad Mostaque who I gather is probably the key funding and organization player in that stuff. It remains to be seen exactly how "open" it winds up playing out over time, but they're talking a compelling game and the EleutherAI people seem to be pretty heavily involved, which you probably wouldn't do if you weren't serious. Yandex has also put up a ~100B language model [2]. My old colleagues at Meta have also started nibbling around the edges of opening some of this stuff up [3]. The Meta folks still aren't just handing out the big ones, but they're definitely moving the ball forward. In particular their release of the training logs is a really positive development IMHO as it opens the curtains a bit on the reality of training these things: it's a difficult, error/failure-prone process, there's a lot of trial-and-error, restarting from checkpoints, etc. Anything that puts downward pressure on the magical thinking is A-OK in my book. The reality of this stuff is exciting/impressive enough: there's no need to embellish or exaggerate. [1] https://www.youtube.com/watch?v=YQ2QtKcK2dA https://www.youtube.com/watch?v=YQ2QtKcK2dA [2] https://github.com/yandex/YaLM-100B https://github.com/yandex/YaLM-100B [3] https://github.com/facebookresearch/metaseq https://github.com/facebookresearch/metaseq

by mkaic 4y ago

benreesman 4y ago

I really enjoyed this interview [1] with Emad Mostaque who I gather is probably the key funding and organization player in that stuff. It remains to be seen exactly how "open" it winds up playing out over time, but they're talking a compelling game and the EleutherAI people seem to be pretty heavily involved, which you probably wouldn't do if you weren't serious. Yandex has also put up a ~100B language model [2]. My old colleagues at Meta have also started nibbling around the edges of opening some of this stuff up [3]. The Meta folks still aren't just handing out the big ones, but they're definitely moving the ball forward. In particular their release of the training logs is a really positive development IMHO as it opens the curtains a bit on the reality of training these things: it's a difficult, error/failure-prone process, there's a lot of trial-and-error, restarting from checkpoints, etc. Anything that puts downward pressure on the magical thinking is A-OK in my book. The reality of this stuff is exciting/impressive enough: there's no need to embellish or exaggerate. [1] https://www.youtube.com/watch?v=YQ2QtKcK2dA https://www.youtube.com/watch?v=YQ2QtKcK2dA [2] https://github.com/yandex/YaLM-100B https://github.com/yandex/YaLM-100B [3] https://github.com/facebookresearch/metaseq https://github.com/facebookresearch/metaseq