5 ms·

"We found zero relationship. It’s a complete random mess." Something overlooked is the millions (billions?) of non-productive labor hours that go into the fail

by VLM 12y ago

"We found zero relationship. It’s a complete random mess." Something overlooked is the millions (billions?) of non-productive labor hours that go into the failed traditional hiring process, when even the smartest guys in the room have to admit that upon statistical analysis its all a waste of time. That's an enormous staggering financial drain on the economy. Overall, across the entire economy, the overall minimum cost mode of hiring would appear to be a union work hall. So employer shows up and the dude who's been sitting there unemployed the longest is hired. This would require incompetent people to admit their own incompetence, so its never going to be implemented, we're all above average here and just because no one else can do traditional hiring correctly doesn't mean I won't be the first to ever get it right because I've been told since birth I'm a special snowflake and I have the participation trophies to prove it, etc. I'm not saying we have to unionize (whole nother topic) but stealing the union work hall "technology" would seem a very wise idea.

jia_min 12y ago
From the NYT's article quoting Laszlo Bock: "Years ago, we did a study to determine whether anyone at Google is particularly good at hiring." Google made the classic mistake of using the interviewer as the predictor of interest. Meanwhile, decades of research has shown it's not who does the asking but how and what you ask. For example, structured interviews are about 50% more predictive of job performance (r = .51) compared to unstructured interviews (i.e., the way almost everyone interviews; r = .38). So why did Google run this experiment when they had an army of Industrial/Organizational Psychologists who already knew the research? They wanted to collect their own data and test the theories on their own employees, which I encourage and applaud.
- wnissen 12y ago
  Do you have a cite for this? I thought Google looked at interviewers and found that no particular interviewer did any better than any other, except, literally, one guy in a narrow sub-specialty. That's quite a lot different than concluding a 50% success rate is no better than chance, which I agree would be problematic at best, flat wrong at worst if the expected occurrence was low.
  - jia_min 12y ago
    Here's the NYT article where Laszlo mentions the "random mess" they found when they used the interviewer as the predictor and then subsequently mentions how they use structured interviews now instead: http://www.nytimes.com/2013/06/20/business/in-head-hunting-big-data-may-not-be-such-a-big-deal.html http://www.nytimes.com/2013/06/20/business/in-head-hunting-b... The interview correlations I cited are from Schmidt and Hunter, 1998: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%... My point is, it doesn't matter who does the interviewing - looking for people who are "good interviewers" is a unicorn chase because the vast majority of people don't have a magical ability to guess who's going to be a good performer pre-hire. Unfortunately, Amazon is making the same mistake with their "bar raisers": http://firstround.com/article/Mechanize-Your-Hiring-Process-to-Make-Better-Decisions http://firstround.com/article/Mechanize-Your-Hiring-Process-... Geez, you engineers don't seem to like to learn from each other.
    - deciplex 12y ago
      But Google doesn't do structured interviews. Not always, at any rate.
      - jia_min 12y ago
        The company line is that they use structured interviews. What actually happens in practice is often a different story.
joncrocks 12y ago
What's important here is that what you have here is only one side of the coin. If you're looking at the performance of only those who make it successfully, what you don't know is how the people who didn't make it through the process would perform on a comparative basis. For example.... There's a pool of candidates, 90% weak, 10% strong (weak and strong if they were measured post-hiring). If your recruitment process took that ratio and ensures you hire 50% weak and 50% strong, that's obviously not completely wasted effort.

by VLM 12y ago

jia_min 12y ago

From the NYT's article quoting Laszlo Bock: "Years ago, we did a study to determine whether anyone at Google is particularly good at hiring." Google made the classic mistake of using the interviewer as the predictor of interest. Meanwhile, decades of research has shown it's not who does the asking but how and what you ask. For example, structured interviews are about 50% more predictive of job performance (r = .51) compared to unstructured interviews (i.e., the way almost everyone interviews; r = .38). So why did Google run this experiment when they had an army of Industrial/Organizational Psychologists who already knew the research? They wanted to collect their own data and test the theories on their own employees, which I encourage and applaud.

wnissen 12y ago
Do you have a cite for this? I thought Google looked at interviewers and found that no particular interviewer did any better than any other, except, literally, one guy in a narrow sub-specialty. That's quite a lot different than concluding a 50% success rate is no better than chance, which I agree would be problematic at best, flat wrong at worst if the expected occurrence was low.
- jia_min 12y ago
  Here's the NYT article where Laszlo mentions the "random mess" they found when they used the interviewer as the predictor and then subsequently mentions how they use structured interviews now instead: http://www.nytimes.com/2013/06/20/business/in-head-hunting-big-data-may-not-be-such-a-big-deal.html http://www.nytimes.com/2013/06/20/business/in-head-hunting-b... The interview correlations I cited are from Schmidt and Hunter, 1998: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%... My point is, it doesn't matter who does the interviewing - looking for people who are "good interviewers" is a unicorn chase because the vast majority of people don't have a magical ability to guess who's going to be a good performer pre-hire. Unfortunately, Amazon is making the same mistake with their "bar raisers": http://firstround.com/article/Mechanize-Your-Hiring-Process-to-Make-Better-Decisions http://firstround.com/article/Mechanize-Your-Hiring-Process-... Geez, you engineers don't seem to like to learn from each other.
  - deciplex 12y ago
    But Google doesn't do structured interviews. Not always, at any rate.
    - jia_min 12y ago
      The company line is that they use structured interviews. What actually happens in practice is often a different story.

joncrocks 12y ago

What's important here is that what you have here is only one side of the coin. If you're looking at the performance of only those who make it successfully, what you don't know is how the people who didn't make it through the process would perform on a comparative basis. For example.... There's a pool of candidates, 90% weak, 10% strong (weak and strong if they were measured post-hiring). If your recruitment process took that ratio and ensures you hire 50% weak and 50% strong, that's obviously not completely wasted effort.