2 ms·
This is amazing and I think will probably end up being a pretty important development. I was just reading this great breakdown of how diffusion Gemma works: ht
by kamranjon 2mo ago
This is amazing and I think will probably end up being a pretty important development.
I was just reading this great breakdown of how diffusion Gemma works: https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-diffusiongemma https://newsletter.maartengrootendorst.com/p/a-visual-guide-...
In reference to the difficulties with applying this to autoregressive LLMs - I wonder if these type of hybrids might be a good candidate for this approach.
- justinnk 2mo agoI agree, this seems like an important discovery. And it hits a sweetspot of being effective and very simple to implement. Have a look at the pseudocode comparison at the bottom of the accompanying github page: https://explorative-modeling.github.io/ https://explorative-modeling.github.io/ Also, a very well-structured and easy to follow blog post by the author. I very much enjoyed reading it!
- deleted 2mo ago[deleted]
- deleted 2mo ago[deleted]
- woadwarrior01 2mo agoAt first glance, this work might also be relevant for quantizing diffusion transformers (something I'm working on at the moment).