Even with those other methods in place, the leaderboard would still favor methods that overfit. That is why the final score is determined on a dataset (the val

Even with those other methods in place, the leaderboard would still favor methods that overfit. That is why the final score is determined on a dataset (the validation set) that is not used until the model is locked down. The public leaderboard is based on the test set. A more in depth explanation on the use of three datasets for model building can be found here: http://stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set http://stats.stackexchange.com/questions/19048/what-is-the-d...