3 ms·
Not my area, enjoyed the read. It reminded me of how you can decode a scaled-down version of a JPEG image by simply ignoring the higher-order DCT coefficients.
by magicalhippo 2y ago
Not my area, enjoyed the read. It reminded me of how you can decode a scaled-down version of a JPEG image by simply ignoring the higher-order DCT coefficients.
As such it seems the statement is that stable diffusion is like an autoregressive model which predicts the next set of higher-order FT coefficients from the lower-order ones.
Seems like this is something one could do with a "regular" autoregressive model, has this been tried? Seems obvious so I assume so, but curious how it compares.
- benanne 2y agoThanks for reading! Absolutely, I included a few references that explore that approach at the bottom of section 4 (last two paragraphs).
- magicalhippo 2y agoExcellent, thanks, will check them out. Had just finished watching the Physics of Language Models[1] talk, where they show how GPT2 models could learn non-trivial context-free grammars, as well as effectively do dynamic programming to an extent, so though it would be interesting to see how they performed in the spectral fine-graining task. [1]: https://physics.allen-zhu.com/home https://physics.allen-zhu.com/home
- magicalhippo 2y ago> I included a few references that explore that approach at the bottom of section 4 Man, reading on mobile phone just ain't the same. Somehow managed to not catch then end of that section. The first reference, "Generating Images with Sparse Representations", is very close to what I had in mind.