28 ms·
GAN training is still spooky mysterious and can easily fail in nonintuitive ways. Sometimes GANs converge or not depending on the random number seed, even with
by gcr 10y ago
GAN training is still spooky mysterious and can easily fail in nonintuitive ways.
Sometimes GANs converge or not depending on the random number seed, even with the same hyperparameters.
- gwern 10y agoI'm not sure about that. The new GAN models over the past 2-3 months, like LS-GAN or WGAN, all seem to train much more stably. I've beaten up on WGAN with all sorts of strange tweaks and hyperparameter settings and while it may not work well, it's never catastrophically diverged on me the way DCGAN would at the drop of a hat.
- AlexCoventry 10y agoHave you found any good ways to speed it up? The five-fold training on the Critic is very expensive.
- gwern 10y agoNo, not yet. I suspect that increasing the discriminator-only learning rate might help but haven't tried.
- igul222 10y agoTry removing BN from the critic :)