3 ms·

People are saying that you do not just submit your trained model to Kaggle. You also submit the code that was used to train the model from the training set, whi

People are saying that you do not just submit your trained model to Kaggle. You also submit the code that was used to train the model from the training set, which is used in the winning models to train them from scratch on the training set. Of course, that wouldn't have prevented this type of cheating of course, but it does mean that you can't submit a model that was trained on your own private data set.

harry8 7y ago
You can overfit a model to your hold out set quite easily with repeated trials. It's a trap you have to avoid in normal circumstances! (Feynman: "You are the easiest person to fool"). Even if you have to submit code to generate your model parameters from "the training set" (which hasn't been explained at all well by "People" if that is indeed the case) you could do that overfitting deliberately here with the illegal unseen data as your hold out set. Aside from the advantage of a bigger training set. Aside from the advantage in model selection, which is not done with code from a training set. Aside from the advantage to your feature engineering also not done in code. Aside from the advantage to your regularization choices, bias parameters etc etc. So yes you absolutely /can/ submit a model trained on your own private data set even if what you submit is a model code that will be re-trained. Even if "the training set" is different to the provided - you still have that scraped data so you can slice it up with the provided training set so that any selected training set does well against the rest. Now the overfit you've just carefully engineered should win against the honest models unless you suck, right? It's kind of risible that they had to go further and hard code certain results, don't you think? Perhaps if they still couldn't win with scraped, illegal additional data then everyone else had illegal data too? Perhaps Kaggle is not a good indicator of how good ML techniques are in practise? Perhaps Kaggle systematically overstates ML effectiveness due to this kind of uncaught cheating in many of their competitions? I bet kaggle won't look too hard at that.

harry8 7y ago

You can overfit a model to your hold out set quite easily with repeated trials. It's a trap you have to avoid in normal circumstances! (Feynman: "You are the easiest person to fool"). Even if you have to submit code to generate your model parameters from "the training set" (which hasn't been explained at all well by "People" if that is indeed the case) you could do that overfitting deliberately here with the illegal unseen data as your hold out set. Aside from the advantage of a bigger training set. Aside from the advantage in model selection, which is not done with code from a training set. Aside from the advantage to your feature engineering also not done in code. Aside from the advantage to your regularization choices, bias parameters etc etc. So yes you absolutely /can/ submit a model trained on your own private data set even if what you submit is a model code that will be re-trained. Even if "the training set" is different to the provided - you still have that scraped data so you can slice it up with the provided training set so that any selected training set does well against the rest. Now the overfit you've just carefully engineered should win against the honest models unless you suck, right? It's kind of risible that they had to go further and hard code certain results, don't you think? Perhaps if they still couldn't win with scraped, illegal additional data then everyone else had illegal data too? Perhaps Kaggle is not a good indicator of how good ML techniques are in practise? Perhaps Kaggle systematically overstates ML effectiveness due to this kind of uncaught cheating in many of their competitions? I bet kaggle won't look too hard at that.