2 ms·
So, why have 2 leaderboards? Or the private one has limited visibility by contestant? (there seems to be simple ways for kaggle to solve this: put a delay on t
by raverbashing 11y ago
So, why have 2 leaderboards? Or the private one has limited visibility by contestant?
(there seems to be simple ways for kaggle to solve this: put a delay on the score generation, limit number of submissions, limit precision of scores on the public leaderboard)
- Zarel 11y agoI'm guessing that the private one isn't visible to contestants or the public until the contest is over (hence "private"). The published dataset gives contestants an idea of how good they are, but prevents gaming the system by overfitting, because the contest is actually scored with the second unknown dataset.
- louden 11y agoEven with those other methods in place, the leaderboard would still favor methods that overfit. That is why the final score is determined on a dataset (the validation set) that is not used until the model is locked down. The public leaderboard is based on the test set. A more in depth explanation on the use of three datasets for model building can be found here: http://stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set http://stats.stackexchange.com/questions/19048/what-is-the-d...
- compbio 11y agoThe private one is revealed after the competition is over. Only the private leaderboard counts. To get an idea of how well your model performs, you get scores for, let's say 30% of the holdout data, on the public leaderboard. You can also use local cross-validation to gauge model performance. There is a limit on the number of submissions. Usually 5 or less a day. Also precision of scores is visibly limited, but for final ranking all decimals count.