3 ms·
What is meant above by "generalisation" is generalisation on out-of-sample data, i.e. not the sample on which you train and test your model (whether you have ac
by YeGoblynQueenne 8y ago
What is meant above by "generalisation" is generalisation on out-of-sample data, i.e. not the sample on which you train and test your model (whether you have access to the test set or not). If out-of-sample data has a different distribution than the sampled dataset, then you're SOL, regularisation or not.
As a for instance, here's an interesting paper I found out about from HN that describes how image classifiers trained on one standard dataset (ImageNet etc) do much worse on other standard datasets and how it's even possible to identify the dataset a classifier was trained on:
Unbiased look at dataset bias
https://people.csail.mit.edu/torralba/publications/datasets_cvpr11.pdf https://people.csail.mit.edu/torralba/publications/datasets_...
- tel 8y agoNot strictly true, regularization in various forms can help have expert knowledge apply to create better extrapolation. Parametric models could be thought of as a kind of regularization even.