4 ms·
well thanks for the info - but then I shift my critique to that I find it unnecessary to distort ML and biological concepts just to simplify the subject, when a
by joyofdata 12y ago
well thanks for the info - but then I shift my critique to that I find it unnecessary to distort ML and biological concepts just to simplify the subject, when an accurate depiction wouldn't be much more difficult. Especially to not differentiate properly between memorization and generalization/learning is odd b/c this is one of the most prominent mistakes - it is specifically not the goal to minimize the in-sample-error! that would lead to very bad results most of the time
- p1esk 12y agoActually, Ilya explains his statement regarding minimizing training errors in his comment exchange with Bengio: "Although I didn't define it in the article, generalization (to me) means that the gap between the training and the test error is small. So for example, a very bad model that has similar training and test errors does not overfit, and hence generalizes, according to the way I use these concepts. It follows that generalization is easy to achieve whenever the capacity of the model (as measured by the number of parameters or its VC-dimension) is limited --- we merely need to use more training cases than the model has parameters / VC dimension. Thus, the difficult part is to get a low training error."