4 ms·
Yeah, but with autoregressive models, the state grows, whereas with diffusion models, it remains fixed. As a result, a diffusion model can't access its past tho
by kgeist 2y ago
Yeah, but with autoregressive models, the state grows, whereas with diffusion models, it remains fixed. As a result, a diffusion model can't access its past thoughts (e.g., thoughts that rejected certain dead ends) and may start oscillating between the same subpar results if you keep denoising multiple times.
- atq2119 2y agoImage diffusion doesn't suffer from this, does it? So why would text diffusion?
- kgeist 2y agoImage diffusion has reasoning?
- spwa4 2y agoYou try to define that it does. Here's my attempt: reasoning is nothing but a set of rules to follow. Complex reasoning is a set of rules that includes instructions on how to extend the set of rules in some way. Image diffusion does both (if you include things like training and finetuning)