> Testing on your training data doesn't in-and-of-itself lead to overfitting, but it will hide overfitting if it does exist (and is a terrible practice for that

> Testing on your training data doesn't in-and-of-itself lead to overfitting, but it will hide overfitting if it does exist (and is a terrible practice for that reason). I read that as pointing out the danger of using training data for hyper-parameter search validation. In practice if you're aggressively tuning your model and feature choices using your training data for validation you're almost certain to have overfitting. I think it actually makes sense to have 3 data sets if you can afford it. > Not sure why this should be the case. Many ensemble models are very memory hungry & slow, Random Forests being a good example. They are flexible and have minimal assumptions, sure, but that doesn't mean you shouldn't try any other modeling technique, especially if you have domain knowledge. I read this as "Try random forests first. If you get good enough results, you're done." He mentions 9 models later and says "In my opinion, and strictly my opinion, the above models will out-perform any others and we don’t need to evaluate any other models." > Not true at all. You won't be able to meaningfully compare coefficient magnitudes, but you can certainly apply linear models. The documentation for StandardScaler says "For instance many elements used in the objective function of a learning algorithm (such as the RBF kernel of Support Vector Machines or the L1 and L2 regularizers of linear models) assume that all features are centered around 0 and have variance in the same order. If a feature has a variance that is orders of magnitude larger that others, it might dominate the objective function and make the estimator unable to learn from other features correctly as expected." > Since normalization is done feature-by-feature, there is no reason why this should be true. StandardScaler re-centers around the mean. That can turn a SparseMatrix into a NotSoSparseMatrix. In a framework that supports only in memory matrices that's probably not what you want, even if it might work. He mentions how to deal with it. Can't comment on PCA. Overall, this seems like practical advice from someone with a lot of experience on Kaggle. Kaggle isn't real life so I'm sure there are better things to do for real models. It's still probably reasonable to take his advice in context instead of assuming it's wrong.