3 ms·

> The fact [the scores of someone who gets hired] are not predictive of their eventual performance, does not mean that hiring process is completely random, or w

> The fact [the scores of someone who gets hired] are not predictive of their eventual performance, does not mean that hiring process is completely random, or worthless. It does not automatically mean this, but it is a strong indicator. If the output of the classifier is not correlated with a given output sample it is reasonable to assume, in the absence of other data, that it is uncorrelated for other samples as well.

feral 10y ago
I do accept it is evidence - but I don't think I would say a strong indicator, and it doesn't justify the strong conclusions made that the process is completely arbitrary. - It's imbalanced classification: there are going to be more rejections than acceptances; so you have less statistical power, to find if there is a real relationship, when you just analyse the acceptances. Even apart from that, class imbalance can unintuitively affect post-hoc inferences. - Scores are in context of the other candidates at a given level, and stage in the interview process, so are fundamentally already residual in some sense. - I'm willing to bet, from hiring I've seen, that scores are distributed somewhat about the 'hire/no-hire' bar, and that extremes are rare - the process is going to be tuned, implicitly in the organisation, to make that 'hire/no-hire' decision, rather than predict performance. > it is reasonable to assume, in the absence of other data, that it is uncorrelated for other samples as well I think that's a very strong statement: it's not a random sample, its explicitly concentrated at one end of the distribution. I do accept it is evidence, just not near enough to justify the conclusion the whole process is arbitrary.
statics 10y ago
No, it's not a strong indicator that the process is random or worthless. That makes the assumption that the people rejected by the process would have scored just as well. It could be very good at determining who has the technical skills to work there. It might be that their performance scoring of hires is measuring something else- like how much their manager likes them.
- vonmoltke 10y ago
  It could, but if technical skill is uncorrelated with performance scoring it will be really hard to determine how many people who make it through the process actually have the technical skill and how many don't.

feral 10y ago

I do accept it is evidence - but I don't think I would say a strong indicator, and it doesn't justify the strong conclusions made that the process is completely arbitrary. - It's imbalanced classification: there are going to be more rejections than acceptances; so you have less statistical power, to find if there is a real relationship, when you just analyse the acceptances. Even apart from that, class imbalance can unintuitively affect post-hoc inferences. - Scores are in context of the other candidates at a given level, and stage in the interview process, so are fundamentally already residual in some sense. - I'm willing to bet, from hiring I've seen, that scores are distributed somewhat about the 'hire/no-hire' bar, and that extremes are rare - the process is going to be tuned, implicitly in the organisation, to make that 'hire/no-hire' decision, rather than predict performance. > it is reasonable to assume, in the absence of other data, that it is uncorrelated for other samples as well I think that's a very strong statement: it's not a random sample, its explicitly concentrated at one end of the distribution. I do accept it is evidence, just not near enough to justify the conclusion the whole process is arbitrary.

statics 10y ago

No, it's not a strong indicator that the process is random or worthless. That makes the assumption that the people rejected by the process would have scored just as well. It could be very good at determining who has the technical skills to work there. It might be that their performance scoring of hires is measuring something else- like how much their manager likes them.

vonmoltke 10y ago
It could, but if technical skill is uncorrelated with performance scoring it will be really hard to determine how many people who make it through the process actually have the technical skill and how many don't.