3 ms·

Can anyone share insight into how this is different from consistency models? The insight seems quite similar?

• bearobear 2y ago
  Consistency models is a special case of IMM where you do moment matching with 1 sample from each distribution (i.e., you cannot match distributions properly). See Fig 5 for an ablation study, of course, adding more samples when you are doing moment matching makes it more stable during training :)
  • throwaway2562 2y ago
    I’m trying to understand what the ‘spectral’ interpretation of IMM is: but perhaps I shouldn’t https://sander.ai/2024/09/02/spectral-autoregression.html https://sander.ai/2024/09/02/spectral-autoregression.html
  • bbminner 2y ago
    Makes sense. How can you even approximately estimate higher order differences in conditional moments in such a high dim space? Seems statistically impossible to get a reasonable estimate for a gradient. Moment matching in sample space has always been very hard.