>> a) I think one of the biggest challenges in a Kaggle competition is getting away from overfitting to the leaderboard. This actually depends on the data. The

>> a) I think one of the biggest challenges in a Kaggle competition is getting away from overfitting to the leaderboard. This actually depends on the data. The commenter above won the Social Circles competition. That competition had a very small number of instances - it looks like it was 60 in the training set and 50 in the test set. It had one of the larger shakeups in Kaggle history.