That makes sense. I was thinking of the multiple test set approach, but rather than forcing people to re-submit on a new test set near the end (if that's what y

That makes sense. I was thinking of the multiple test set approach, but rather than forcing people to re-submit on a new test set near the end (if that's what you meant), the organisers could just return the score based on a random fraction of the test set. 'Cheaters' would then overfit to just half of the actual test set and this would be apparent when the final scores (on the other half) are revealed. I suspect this is what Kaggle do, as they don't make you re-submit at the end (AFAIK).