Yes I realise the importance of a testing set too. I see that as just another level validation. Hierarchies of left out data. If you understand in your heart wh

Yes I realise the importance of a testing set too. I see that as just another level validation. Hierarchies of left out data. If you understand in your heart what cross validation is, then the step to using a testing set is a trivial extension. I brought up cross validation in particular, because I see PhDs eating books on advanced techniques like structure learning. But they have no practical experience. And they ignore the basics for so long, as they don't realise validation is the most important bit. If you have not drawn the curve of training set error, validation error against training time, you have no done machine learning yet... This article doesn't mention it either :(