3 ms·
Mm. I understand the argument (tho this subletly is not present in the article, maybe you should add it if that's actually what people do). That said I am not
by davidguetta 2y ago
Mm. I understand the argument (tho this subletly is not present in the article, maybe you should add it if that's actually what people do).
That said I am not 100% convinced : first the underlying model is still the same and because you are effectively learning a eps(x, t, y). secondly the equivalence suggests (if you actually use the difference as gradient) you should end up in the same global min/max as if you were doing this directly no ?
- reasonableklout 2y agoThanks for the feedback! Yes, when conditioned on a non-null label, the underlying model for classifier-free guidance can be seen as the same as a class-conditional model (in fact, the original CFG paper[0] discusses training a separate model - they didn't do that because it's extra work for not much benefit). And for a certain weighting of the guidance vector for each step, you can obtain the same sampling step you would get with a class-conditional model. But my understanding is that the point of guidance is that you can control this weighting, because you are able to estimate the unconditional score too. [0] https://arxiv.org/pdf/2207.12598 https://arxiv.org/pdf/2207.12598