4 ms·
This is from 2019 so I suspect there’s a more recent paper now, but this is interesting: https://openai.com/blog/deep-double-descent/ https://openai.com/blog/de
by DougBTX 4y ago
This is from 2019 so I suspect there’s a more recent paper now, but this is interesting: https://openai.com/blog/deep-double-descent/ https://openai.com/blog/deep-double-descent/
- disgruntledphd2 4y agoAgreed, but it doesn't really explain it in any real sense.
- mr_toad 4y ago“However, in the over-parameterized regime, there are many models that fit the train set and there exist such good models.” One explanation is that a very large model effectively fits several models and penalises over-fit models itself.
- skinner_ 4y agoWork is ongoing, this is a whole field of research now, a good keyword to search for is "implicit regularization".
- marcosdumay 4y agoI really like how AI research is all about empirical analysis of mathematics.