3 ms·
I'm a controls guy, not a quant guy but I've found the SDE perspective and this blog post to be incredible in helping me understand how a diffusion model works.
by patrick451 2y ago
I'm a controls guy, not a quant guy but I've found the SDE perspective and this blog post to be incredible in helping me understand how a diffusion model works.
- currymj 2y agoit makes intuitive sense, but at the same time, you can get good performance replacing the Gaussian noise with deterministic transformations. so in the end diffusion can’t explain that much about how diffusion models work. disappointingly, so many modern ML things are like this.
- sl8r 2y agoHey! I’m the post author. Highly recommend Sander Dieleman’s blog for alternative interpretations https://sander.ai/2023/07/20/perspectives.html https://sander.ai/2023/07/20/perspectives.html I personally find the SDEs the most intuitive, and the deterministic ODE / consistency models / rectified flow stuff as ideas that are easier to understand after the SDEs. But not everyone agrees!
- currymj 2y agoThanks for sharing this! I tend to agree that it’s easiest to understand this way. I just find it a frustrating fact about modern machine learning that in fact, the nice SDE interpretation is somehow not the “truth”, it’s just a tool for understanding.