3 ms·
Luckily you don't work in the field. You can get perfect accuracy on a target set while having approximately zero accuracy on any other set, if allowed to run
by hiddencost 11y ago
Luckily you don't work in the field. You can get perfect accuracy on a target set while having approximately zero accuracy on any other set, if allowed to run infinite tests. This isn't cheating because it helped them be better. It's cheating because it helped them get results that look very good but won't generalize well.
- squigs25 11y agoIn a contest like this, I would expect (and I could be wrong) that there is a final validation set not accessible until "game day". Thus, overfitting to a training data set of any kind would be detrimental and degrade model performance.