3 ms·
I know this is a joke, but the theory of generalization in NNs is rapidly advancing and it's not quite that simplistic: https://arxiv.org/abs/2003.02139 https:/
by deepnotderp 7y ago
I know this is a joke, but the theory of generalization in NNs is rapidly advancing and it's not quite that simplistic: https://arxiv.org/abs/2003.02139 https://arxiv.org/abs/2003.02139
- chewxy 7y agoYa. And the choice of optimizer (in this case adam) also imposes upon it some regularization scheme. I just thought I'd highlight a bit of funniness.
- profunctor 7y agoHow does Adam provide regularisation? I’d never heard of this before and I don’t recall it from when I read the paper.