Any metric that is not the true metric that is optimized on will lead to worse results if overfit. You need to make sure there's at least a reasonable amount of

Any metric that is not the true metric that is optimized on will lead to worse results if overfit. You need to make sure there's at least a reasonable amount of variance.