4 ms·
>Neural nets are infamously incapable of generalising beyond their training dataset In what sense? DNNs can certainly generalize to examples outside the traini
by kahnjw 7y ago
>Neural nets are infamously incapable of generalising beyond their training dataset
In what sense? DNNs can certainly generalize to examples outside the training set. I agree that you get no guarantees on performance for samples drawn from a distribution that is different than the training/test set. Not having guaranteed expected performance isn't the same as "incapable" of generalizing to a new distribution.
- YeGoblynQueenne 7y agoSorry for the late reply. To clarify, when I say "training dataset" I mean the entire dataset used for training. Not the training partition in a cross-validation training/test split. Neural nets can interpolate between the data points in their training dataset, but cannot extrapolate to cover data points outside this dataset. This is not a matter of architecture. DNNs are no exception. Neural nets are just very bad at generalising. This lack of generalisation ability is why neural nets need to be trained with huge amounts of data, the more the better. Because they can't generalise, the only way to get them to recognise more instances of a class is to show them more examples of that class. Here's a longer discussion of this: https://blog.keras.io/the-limitations-of-deep-learning.html https://blog.keras.io/the-limitations-of-deep-learning.html