I'll let the other answers stand for explaining Kaggle and their game, and try to answer "overfitting" with an ELI5 example. Here is a hypothetical problem: P

I'll let the other answers stand for explaining Kaggle and their game, and try to answer "overfitting" with an ELI5 example. Here is a hypothetical problem: Please write a calculator program, that takes input from stdin, containing one arithmetic expression per line (integers and + - etc), and simplifies the expression, returning one integer per line. Here's a sample input/output pair! Input 2+2 3+1*2 Output 4 5 And you're all clever, and you write the following program: # python print "4\n5" Congratulations, you pass my sample test case, but you have overfit your model to the sample data, and it will not generalize to other datasets for the same task. If this were a Kaggle contest, and there were a leaderboard on how well people are doing on the sample data, you'd have a perfect score on the leaderboard. But then when it's time to see who wins the contest, and I try your program against some other inputs, you'd get 0%. Ha ha, funny joke you played!