3 ms·
> The most famous problem of GANs is instability during training and mode collapse - which is like a student learning especially for an exam (and not in general
by amenod 9y ago
> The most famous problem of GANs is instability during training and mode collapse - which is like a student learning especially for an exam (and not in general) thus optimising for the test instead of the real thing.
I must confess I haven't worker with GANs yet, but isn't that the whole point of GANs? Student is optimising for the test while the teacher is learning how to make tests as similar to reality as possible?
If I understand correctly, the main challenge is finding a way to allow teacher and student (well, generator and adversary) to learn at a similar rate, so that one doesn't stop learning because its competitor is too advanced. Is that correct?
- vinn124 9y ago> but isn't that the whole point of GANs? not quite, but youre on the right path. think about it this way: you (the generative model) are trying to predict a unit gaussian, which is just a fancy way to say bell curve. you get +1 if you predict a number in this distribution (eg 0.1 or -0.5, which is within one standard deviation of the mean of 0); you get -1 if you predict a number thats "far" from this distribution (something like 40 - which has an infinitesimally low probability of being drawn from a unit gaussian). mode collapse, then, is when you predict 0 all the time. yes, you are technically correct but youve failed to learn the true distribution. obviously ive simplified this quite a bit and have anthropomorphize the model, but i hope you get the gist. otherwise, the [original paper](https://arxiv.org/abs/1406.2661 https://arxiv.org/abs/1406.2661) is refreshingly easy to read.
- amenod 9y agoThanks!