3 ms·
So they minimize the forward KL divergence from the generative model to the data distribution, and then maximize the entropy of the generative model? Seems lik
by eli_gottlieb 3y ago
So they minimize the forward KL divergence from the generative model to the data distribution, and then maximize the entropy of the generative model? Seems like it would be simpler to just minimize the reverse/exclusive KL divergence, giving you a cross-entropy term to minimize and then an entropy of the generative model to maximize.
I guess the obstacle to doing that is not being able to evaluate the true data log-density, only sample from it, so the cross-entropy term can't be estimated?