I do accept it is evidence - but I don't think I would say a strong indicator, and it doesn't justify the strong conclusions made that the process is completely

I do accept it is evidence - but I don't think I would say a strong indicator, and it doesn't justify the strong conclusions made that the process is completely arbitrary. - It's imbalanced classification: there are going to be more rejections than acceptances; so you have less statistical power, to find if there is a real relationship, when you just analyse the acceptances. Even apart from that, class imbalance can unintuitively affect post-hoc inferences. - Scores are in context of the other candidates at a given level, and stage in the interview process, so are fundamentally already residual in some sense. - I'm willing to bet, from hiring I've seen, that scores are distributed somewhat about the 'hire/no-hire' bar, and that extremes are rare - the process is going to be tuned, implicitly in the organisation, to make that 'hire/no-hire' decision, rather than predict performance. > it is reasonable to assume, in the absence of other data, that it is uncorrelated for other samples as well I think that's a very strong statement: it's not a random sample, its explicitly concentrated at one end of the distribution. I do accept it is evidence, just not near enough to justify the conclusion the whole process is arbitrary.