3 ms·
haven't modern model architectures gotten better at avoiding this kind of overfitting? like obviously data quality is still very important, but my understanding
by sterlind 3y ago
haven't modern model architectures gotten better at avoiding this kind of overfitting? like obviously data quality is still very important, but my understanding is that dropout mitigates this by randomly cutting out these unwanted feature channels. the models learn to distinguish all differences, rather than just one, or fixed combinations of several.
- dragonwriter 3y ago> haven't modern model architectures gotten better at avoiding this kind of overfitting? Overfitting is, AIUI, a training method and data issue, not a model issue alone. I doubt any model is resistant to overfitting if you give it data where the answer is reliably encoded some aspect it can use but outside of what you want it to look at. Now, you can notice suspicious results and investigate (or you can just publish a 100% success rate and call it a day.)
- daveguy 3y agoIt really doesn't have to do with most ML architectures. It has to do with experiment design. If some data used in testing is part of the training process there will be over fitting. That's why a final test set is required for unbiased evaluation.