4 ms·
I would really like to hear a definition of what generalization means, because I don't think we have one. Unless we're talking generalization to arbitrary dist
by Cacti 9y ago
I would really like to hear a definition of what generalization means, because I don't think we have one.
Unless we're talking generalization to arbitrary distributions, which is of course unsolvable.
- eanzenberg 9y agoUsually in ML context generalization means anytime a model makes a prediction on unseen (not in the training set) inputs. Usually you do CV to see how well your model generalized because you limit the training data seen, predict unseen inputs that you know the answer to, and see how far off the prediction is.
- jd007 9y agoI tend to think of "generalization", in the general sense (pun not intended), to be information compression. We are simply trying to answer the question: what is the shortest description (i.e. most informationally compressed/dense version) that fits what we see in this infinite (at least to us mortals) universe of ours? Mathematically the length of such a description can be thought of as the Kolmogorov complexity. Edit: I should add, the information compression performed when generalizing can (and often is) lossy.