3 ms·

All the novelty appears to boil down to "this is a new sampler for diffusion-style models". Apparently, they are inspired by a different physical process. But

All the novelty appears to boil down to "this is a new sampler for diffusion-style models". Apparently, they are inspired by a different physical process. But that doesn't change the fact that they start with noise and then iteratively solve a differential equation to get to its end point. 2nd-order samplers like Heun already massively increased sampling speed over the original diffusion models, but sadly the article doesn't compare to that but merely to the original 1st-order samplers. So it might be that this one doesn't even create a speedup in practice. Does anyone else have more info on how this is different from "traditional" numerical differential-equation-based de-noising?

thatcherc 4y ago
The big this seems to be that their noising/denoising process is invertible (to some extent) so it can be used to generate uncertainties, which is something the article says diffusion models can't do. The intuition is that the "flow" process used here is reversible (imagine going backwards in time and watching a cloud of charged particles coalescing into a bunch), which gives some nice properties that random blurring used in more common diffusion models does not. I think it's the reversible flow part that's important for the results, not the connection to the physical electrostatics system. It just happens to be that electrostatics have this nice flow behavior too and are a pretty approachable analogy for what the model's doing.
- fxtentacle 4y ago
  I also noticed that they present it as if other diffusion models were not invertible, but they are. Also, they appear to be using DDPM++, the exact same neural network architecture as Stable Diffusion? Article says: "to train the neural network, which in this case is a U-Net (DDPM++ backbone)." Also it's kinda weird that they show videos generating images from noise (like Imagen / Stable Diffusion) but don't cite any recent diffusion paper. EDIT: In fact they don't cite ANY 2022 paper. So my guess would be that they submitted this last year and now it's been made public because now the NeurIPS 2022 conference is taking place. But most likely, the actual research here predates the Stable Diffusion release.
  - SleekEagle 4y ago
    There is a stochastic element to diffusion models which is not present in PFGMs. Further, recall that Diffusion Models and PFGMs alike are "metamodels" - mathematical frameworks in which any function approximator can, in theory, be placed. They are not an architecture. In this case, the authors use the DDPM++ backbone which is the specific function approximator that they implement in their PFGM framework.

thatcherc 4y ago

The big this seems to be that their noising/denoising process is invertible (to some extent) so it can be used to generate uncertainties, which is something the article says diffusion models can't do. The intuition is that the "flow" process used here is reversible (imagine going backwards in time and watching a cloud of charged particles coalescing into a bunch), which gives some nice properties that random blurring used in more common diffusion models does not. I think it's the reversible flow part that's important for the results, not the connection to the physical electrostatics system. It just happens to be that electrostatics have this nice flow behavior too and are a pretty approachable analogy for what the model's doing.

fxtentacle 4y ago
I also noticed that they present it as if other diffusion models were not invertible, but they are. Also, they appear to be using DDPM++, the exact same neural network architecture as Stable Diffusion? Article says: "to train the neural network, which in this case is a U-Net (DDPM++ backbone)." Also it's kinda weird that they show videos generating images from noise (like Imagen / Stable Diffusion) but don't cite any recent diffusion paper. EDIT: In fact they don't cite ANY 2022 paper. So my guess would be that they submitted this last year and now it's been made public because now the NeurIPS 2022 conference is taking place. But most likely, the actual research here predates the Stable Diffusion release.
- SleekEagle 4y ago
  There is a stochastic element to diffusion models which is not present in PFGMs. Further, recall that Diffusion Models and PFGMs alike are "metamodels" - mathematical frameworks in which any function approximator can, in theory, be placed. They are not an architecture. In this case, the authors use the DDPM++ backbone which is the specific function approximator that they implement in their PFGM framework.