4 ms·

The pure volume of projects spurred on by the release of Stable Diffusion source code is staggering

ShamelessC 4y ago
The original diffusion code for these projects comes from lots of research. In terms of code- OpenAI released guided-diffusion over ImageNet classes awhile ago, then GLIDE, their DALLE2 predecessor. CompVis incorporated that into their Latent Diffusion variant about half a year ago. A few weeks before Stable Diffusion released they also published the similar “retrieval augmented diffusion” codebase. And _then_ stable diffusion came out. The effectiveness of diffusion models for text to image is an idea that has been floating around a little longer than you might think.
- Vt71fcAqt7 4y ago
  I think I agree here. Stable diffusion is "just" an optimized version of DALLE2, which itself directly builds on the previous literature. Also, the 3d model generator posted to HN yesterday builds on NeRF. And they are all using CLIP I believe. Stable diffusion gave us speed and knowlege about how much data of what kind and quality needs to be in the training data to still generate good results.
  - ShamelessC 4y ago
    It's certainly not my intent to undermine the efforts of Robin Rombach, Andreas Blattman, Katherine Crowson (and many others). Katherine's work on clip-guided-diffusion over the `guided-diffusion` ImageNet checkpoints was effectively the first time the public got to see what text-to-image via diffusion instead of purely transformer-based solutions (like in DALLE1/dalle-mini) would look like. And it happened well before GLIDE was published (and gets a mention/citation). The CompVis team (Blattman, Rombach, etc.) has been able to not just compete, but surpass (in some ways - it's nuanced) the work of the big American research labs (OpenAI in particular) with solid novel research. Their research on `VQGAN` outperformed the Autoencoder from the DALLE-1 paper, and they've been competing directly in the vision space ever since. Incredibly talented people.
antegamisou 4y ago
Yet the SNR is getting abysmal day by day.

The original diffusion code for these projects comes from lots of research. In terms of code- OpenAI released guided-diffusion over ImageNet classes awhile ago, then GLIDE, their DALLE2 predecessor. CompVis incorporated that into their Latent Diffusion variant about half a year ago. A few weeks before Stable Diffusion released they also published the similar “retrieval augmented diffusion” codebase. And _then_ stable diffusion came out. The effectiveness of diffusion models for text to image is an idea that has been floating around a little longer than you might think.

Vt71fcAqt7 4y ago
I think I agree here. Stable diffusion is "just" an optimized version of DALLE2, which itself directly builds on the previous literature. Also, the 3d model generator posted to HN yesterday builds on NeRF. And they are all using CLIP I believe. Stable diffusion gave us speed and knowlege about how much data of what kind and quality needs to be in the training data to still generate good results.
- ShamelessC 4y ago
  It's certainly not my intent to undermine the efforts of Robin Rombach, Andreas Blattman, Katherine Crowson (and many others). Katherine's work on clip-guided-diffusion over the `guided-diffusion` ImageNet checkpoints was effectively the first time the public got to see what text-to-image via diffusion instead of purely transformer-based solutions (like in DALLE1/dalle-mini) would look like. And it happened well before GLIDE was published (and gets a mention/citation). The CompVis team (Blattman, Rombach, etc.) has been able to not just compete, but surpass (in some ways - it's nuanced) the work of the big American research labs (OpenAI in particular) with solid novel research. Their research on `VQGAN` outperformed the Autoencoder from the DALLE-1 paper, and they've been competing directly in the vision space ever since. Incredibly talented people.

antegamisou 4y ago

Yet the SNR is getting abysmal day by day.