4 ms·
I've noticed that when finance guys learn that there's a really useful AI thing called diffusion, they get all excited and start writing stochastic differential
by getnormality 2y ago
I've noticed that when finance guys learn that there's a really useful AI thing called diffusion, they get all excited and start writing stochastic differential equations and drawing 1-D Brownian motion plots all over the place. It's not yet clear to me whether this helps anyone understand AI diffusion models, but they seem to enjoy it a lot.
- markisus 2y agoIf by helping anyone you mean helping the math challenged (which is technically almost everyone) then I would be inclined to agree. But to the quant crowd I’m guessing that couching diffusion in the language of stochastic calculus is helpful.
- patrick451 2y agoI'm a controls guy, not a quant guy but I've found the SDE perspective and this blog post to be incredible in helping me understand how a diffusion model works.
- currymj 2y agoit makes intuitive sense, but at the same time, you can get good performance replacing the Gaussian noise with deterministic transformations. so in the end diffusion can’t explain that much about how diffusion models work. disappointingly, so many modern ML things are like this.
- sl8r 2y agoHey! I’m the post author. Highly recommend Sander Dieleman’s blog for alternative interpretations https://sander.ai/2023/07/20/perspectives.html https://sander.ai/2023/07/20/perspectives.html I personally find the SDEs the most intuitive, and the deterministic ODE / consistency models / rectified flow stuff as ideas that are easier to understand after the SDEs. But not everyone agrees!
- currymj 2y agoThanks for sharing this! I tend to agree that it’s easiest to understand this way. I just find it a frustrating fact about modern machine learning that in fact, the nice SDE interpretation is somehow not the “truth”, it’s just a tool for understanding.