3 ms·
Why are you trying to backpropagate over data augmentations? I've never done that (or heard about it being done). Usually I just do the augmentations on the
by spott 6y ago
Why are you trying to backpropagate over data augmentations? I've never done that (or heard about it being done). Usually I just do the augmentations on the input samples and then feed the augmented samples to the network.
Differentiable augmentations aren't necessary unless the augmentations are midstream (so you have to propagate parameters above the augmentations, which is weird) or have parameters (at which point you aren't learning how to work on different views of the same sample, you are learning how to modify a sample to be more learnable, which is a different problem that you are trying to solve).
Don't get me wrong, augmenting samples to reduce device bias is a hard problem, but you might be making it harder than it needs to be.
- gwern 6y agoThe data augmentations we are interested in are in fact 'midstream', as they augment the examples before passing into the D or the classification loss but you must backprop from that back through the augmentation into the original model, because you don't want the augmentations to 'leak': the G is not supposed to generate augmented samples, the augmentation is there to regularize the D and reduce its ability to memorize real datapoints. It would probably be better to consider them as a kind of consistency or metric loss along the lines of SimCLR (which has helped inspire these very new GAN data augmentation techniques). It's a bit weird, which is perhaps why despite its simplicity (indicated by no less than 4 simultaneous inventions of it in the past few months), it hasn't been done before. You really should read the linked Github thread if you are interested.