4 ms·

This is how research works, you isolate variables. By OP’s logic science is never testing for anything “real.” I’m not surprised that HN rushes to discredit th

This is how research works, you isolate variables. By OP’s logic science is never testing for anything “real.” I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy, but this is kinda silly.

pedrosorio 5y ago
Isolating this variable allows one to study how good hiring managers are at estimating base probabilities for education/competence given social class (news alert: they are not independent). It says nothing about what actually matters: do hiring managers with the ability to properly interview "equally qualified" candidates from different social backgrounds do so effectively (i.e. offer them similar compensation), or do they still rely on the base rates (i.e. bias), and how big is that effect?
jefftk 5y ago
But this is also not how isolating variables works. If you give someone no information, and require them to make a decision based only on speech patterns, that does not actually tell you very much about how they would make a decision if they had lots of information including speech patterns. Compare: a) only speech b) lots of complex skills-based information that is (verified in a separate experiment) equivalent on balance, plus also speech It's quite possible that you could see a large difference in decisions in (a) without any detectable difference in (b). For example, imagine in this study hiring managers are (perhaps subconsciously) estimating education level from speech. If they already have lots of details about candidate's education, that "screens out" the rough proxy of their speech patterns.
splitstud 5y ago
Questioning the validity of the isolation, and conclusions drawn from the result is how science works.
KMag 5y ago
> I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy, but this is kinda silly. I think this is a straw man argument, unless you can point to more than one person here (now I see none) arguing hiring is a perfect meritocracy. I see lots of people questioning the methodology and conclusions drawn, but I don't see anyone claiming hiring is perfectly meritocratic. On the one hand, if we need to perfectly understand the problem before starting to solve it, we'll be forever stuck in analysis. On the other hand, if we are incorrect in our initial analysis, we risk wasting time and other resources on the wrong solutions, reducing resources available for other solutions further down the road. It's very important to distinguish constructive criticism from animosity.
bryanrasmussen 5y ago
>I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy I'm surprised you evidently haven't seen the thousands of threads where HN predominantly does nothing but complain about how hiring is not a perfect meritocracy.
michaelt 5y ago
Well, the experimenters could have combined their corpus of job candidates describing themselves with a bunch of recordings of actors giving the same descriptions in different accents. Then each hiring manager could have been presented with a different subset of that pool, with both resume and recording. A multivariate regression would then reveal the relative impacts of class, being an actor, and the different self-descriptions.

pedrosorio 5y ago

Isolating this variable allows one to study how good hiring managers are at estimating base probabilities for education/competence given social class (news alert: they are not independent). It says nothing about what actually matters: do hiring managers with the ability to properly interview "equally qualified" candidates from different social backgrounds do so effectively (i.e. offer them similar compensation), or do they still rely on the base rates (i.e. bias), and how big is that effect?

jefftk 5y ago

But this is also not how isolating variables works. If you give someone no information, and require them to make a decision based only on speech patterns, that does not actually tell you very much about how they would make a decision if they had lots of information including speech patterns. Compare: a) only speech b) lots of complex skills-based information that is (verified in a separate experiment) equivalent on balance, plus also speech It's quite possible that you could see a large difference in decisions in (a) without any detectable difference in (b). For example, imagine in this study hiring managers are (perhaps subconsciously) estimating education level from speech. If they already have lots of details about candidate's education, that "screens out" the rough proxy of their speech patterns.

splitstud 5y ago

Questioning the validity of the isolation, and conclusions drawn from the result is how science works.

KMag 5y ago

> I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy, but this is kinda silly. I think this is a straw man argument, unless you can point to more than one person here (now I see none) arguing hiring is a perfect meritocracy. I see lots of people questioning the methodology and conclusions drawn, but I don't see anyone claiming hiring is perfectly meritocratic. On the one hand, if we need to perfectly understand the problem before starting to solve it, we'll be forever stuck in analysis. On the other hand, if we are incorrect in our initial analysis, we risk wasting time and other resources on the wrong solutions, reducing resources available for other solutions further down the road. It's very important to distinguish constructive criticism from animosity.

bryanrasmussen 5y ago

>I’m not surprised that HN rushes to discredit the idea that hiring is not a perfect meritocracy I'm surprised you evidently haven't seen the thousands of threads where HN predominantly does nothing but complain about how hiring is not a perfect meritocracy.

michaelt 5y ago

Well, the experimenters could have combined their corpus of job candidates describing themselves with a bunch of recordings of actors giving the same descriptions in different accents. Then each hiring manager could have been presented with a different subset of that pool, with both resume and recording. A multivariate regression would then reveal the relative impacts of class, being an actor, and the different self-descriptions.