> I realize this is the typical mathematician argument. Interestingly, a mathematician's argument on this exact method was actually given in 1977, in "An Asymp

> I realize this is the typical mathematician argument. Interestingly, a mathematician's argument on this exact method was actually given in 1977, in "An Asymptotic Equivalence of Choice of Model by Cross-Validation and Akaike's Criterion". [1] What Stone showed was that (modulo technicalities) performing this procedure - leave one out cross validation, the procedure you claim "cannot work correctly" - was asymptotically equivalent to minimising the AIC [2] of the model. So we have an asymptotic guarantee (for whatever that's worth) on the behaviour of this procedure. [1]: http://www.jstor.org/stable/2984877 http://www.jstor.org/stable/2984877 [2]: http://en.wikipedia.org/wiki/Akaike_information_criterion http://en.wikipedia.org/wiki/Akaike_information_criterion