3 ms·
>> My question is are people using overfitting as an excuse of a what is instead a badly made NN. Overfitting is a major issue in machine learning and it's an
by YeGoblynQueenne 9y ago
>> My question is are people using overfitting as an excuse of a what is instead a
badly made NN.
Overfitting is a major issue in machine learning and it's an inherent
characteristic of learning from examples and not the result of a mistake, or of
poor practice. There are special techniques developed explicitly to reduce
overfitting- early stopping (what red75prime above, describes), regularisation,
bagging (in decision trees) etc. A lot of work also goes into ensuring measures of
learning performance don't mistake overfitting for successful learning (e.g. k-fold cross
validation).
I'm sorry that I don't have time to track down a good source for a discussion of the
bias-variance tradeoff and overfitting. You can start at the wikipedia page
[https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff] and follow the
links. In short- a model that learns to reproduce its example data with very
high fidelity, risks generalising poorly, whereas a model that generalises well
may have high training error. Linear classifiers in particular are
high-bias, whereas nonlinear learners, like multi-layered neural networks or
decision trees, are high-variance.
The problem is real, it's a big bugbear and you won't find any specialist who
dismisses it, or who considers it "not a real issue".