Am I reading this wrong? They only had validation sets with no final test set which makes the results kinda worthless because we don't know how overfit they wer

Am I reading this wrong? They only had validation sets with no final test set which makes the results kinda worthless because we don't know how overfit they were to these validation sets (which can easily happen with any sort of parameter tuning). There is a reason why a proper study needs three splits train/validation (and possible multiple of these if you use k-fold) and a final to be used as sparingly as possible 'test' set. See the paper, "On estimating model accuracy with repeated cross-validation" Even the sklearn docs for cross validation show this split: https://scikit-learn.org/stable/_images/grid_search_cross_validation.png https://scikit-learn.org/stable/_images/grid_search_cross_va...