6 ms·

Of course we should be more critical, the methodology here is far from perfect. But right now, people have much faith in interviewing, and this research suggest

Of course we should be more critical, the methodology here is far from perfect. But right now, people have much faith in interviewing, and this research suggests that this faith may not be justified. As you mentioned, this is not the only research that reaches this conclusion. There is also the work by Daniel Kahneman[1] which I find pretty rigorous and draw the same conclusions about interviewing. So obviously, the title of this article should be "Maybe interviewing is not that useful" instead of "The utter uselessness of job interviews", but besides this I find your comment unjustified. In fact, it's quite the opposite, I believe this type of work contributed to make us more critical by questioning some basic facts about interviewing, that i would have never questioned just a couple of years ago. > Over time I’ve become more skeptical about this kind of psychology research (as more studies fail to replicate) ok, this is interesting, where is it mentioned? [1] Think fast and think slow. There is this short article which mention some of the results and has been discussed on HN a couple of times already. http://www.nytimes.com/2011/10/23/magazine/dont-blink-the-hazards-of-confidence.html http://www.nytimes.com/2011/10/23/magazine/dont-blink-the-ha...

mooreds 10y ago
> ok, this is interesting, where is it mentioned https://mobile.nytimes.com/2015/08/28/science/many-social-science-findings-not-as-strong-as-claimed-study-says.html https://mobile.nytimes.com/2015/08/28/science/many-social-sc... among other places
andrewmutz 10y ago
It's easy to present data that shows flaws with today's interviewing. It's a lot harder to present a better way of predicting candidate performance in the workplace, along with substantial data that indicates it's better than today's methods. Corporations would love more effective ways to determine effectiveness/performance before hiring. Interviewing is terrible, but that doesn't mean there is a better option.
- emodendroket 10y ago
  That feels a bit like saying we should just stick to blood letting and leeches because we don't know any more effective way of treating disease. Just because we don't have a superior alternative doesn't actually mean the current method is effective.
  - RugnirViking 10y ago
    A) Leeches are often used in literal modern hospitals in the developed world as an effective treatment for certain ills. B) If we had no alternative, we should stick to it, but look at other things in the meantime. You seem to be advocating doing nothing at all, which is trivially easy to show works for no-one
    - emodendroket 10y ago
      Yes, leeches have extremely limited use. However, bloodletting and leeches killed far more people than they ever "treated."
- jaggederest 10y ago
  I strongly disagree. There are all sorts of situations where having a bad option is much worse than having no option, hiring and medicine clearly among them. > Corporations would love more effective ways to determine effectiveness/performance before hiring. This is irrelevant to evaluating the current methods. Even if there is no replacement, if they are useless, we should know it, bar none.
- jernfrost 10y ago
  Nonsense. First step is to acknowledge that interviewing doesn't work very well. You can't keep deluding yourself because you haven't found a better way. Accept reality. This is one of the biggest problems I see with business guys today. They want absolute certainty in a world which can't offer it. Start accepting that there is a lot of stuff we don't know and can't know at the moment and we simply have to work towards getting better. You can't get better if you don't acknowledge that there is a problem in the first place. These silly, must have +5 years of experience, and check all these boxes with technologies to get the job clearly shows the industry is completely lost at the moment and isn't willing to acknowledge it.
  - mahyarm 10y ago
    Of course people know interviews aren't the greatest, but it's the best tool they have right now. What people are pushing back against is criticism without solutions. There are a bunch of other testing methods that are illegal in the USA too, such as IQ tests.
    - LnxPrgr3 10y ago
      When users of code I've written complain that something sucks to use, I don't demand they come up with a better solution. Probably I don't even want to know what they think the answer is—unless they design user interfaces for a living, odds are their answer will be terrible and they won't even be happy with it if I build it. This is as it should be—I'm paid to solve problems. Why is this different? The more I read about interviewing, the more I realize too many people think they have this problem solved—their amateur psychology is impeccable and their technical screens test for exactly the right things, no more and no less. Did they do a bunch of controlled studies to convince themselves of this, or are they taking sounding good, or intuition about the statistical outcomes of different techniques, to be equivalent to truth? Maybe the first step is to collectively realize we have close to no clue what we're doing, and are being asked to solve a hard problem: individually, to talk to someone for an hour and make a hiring recommendation. In aggregate, to make the decision based on a handful of these one-hour conversations. Maybe the first step is to realize this is a problem worth trying to solve. Maybe the first step is vocal non-acceptance.
- Sevii 10y ago
  We have had a better way for decades, its banned. What do you think all these algorithm questions are supposed to do?
  - mafm 10y ago
    What is the better way that is banned?
    - frankc 10y ago
      I suspect he is referring to iq tests. They have a pretty high correlation to success and are banned.
      - yorwba 10y ago
        I wonder how much correlation with IQ is allowed before custom algorithms questions qualify as an illegal IQ test.
        Spooky23 10y ago
        Employees will test the limits with lawsuits. I know at least one large state lost a class action lawsuit due to racial bias on civil service exams. The problem with comparing IQ results at hire to employment success is that employment outcome is difficult to define over time. You're also unlikely to get statistically relevant data without focusing on large organizations with standardized HR processes. Most of the research is based on supervisory evaluations, which are not the most reliable indicators of anything for a variety of reasons. The other thing I find amusing is that business folk who talk about this miss the fact that there are large workforces in the US that either have or do use standardized testing like this to hire and promote. Those are government bureaucracies, which function relatively well, but are hardly a model that most folks advocating this would aspire towards.
        lr4444lr 10y ago
        I don't think the issue is correlation, but rather disparate impact[0]. Algorithmic tests have a pretty low bar to clear in IT jobs. Using them for secretaries or lab technicians is another story. https://en.wikipedia.org/wiki/Disparate_impact https://en.wikipedia.org/wiki/Disparate_impact
jaredklewis 10y ago
People have too much faith in the efficacy of interviewing, ergo we should give slipshod journalism and science lacking rigor a free pass? I'm all for upending the status quo with interviews, but let's not throw out science and reporting just to get there.
schmit 10y ago
My point is not that this work is flawed, or that there should not be an article reporting on this research or the topic of interviewing. Rather, I think it'd be better for a third party to write about the topic in a more objective manner, rather than a professor promoting his own research (and thus with skewed incentives). In particular, I was disappointed to find a (short) paragraph in the article that I find bogus. That does not mean the article shouldn't have been posted in the first place, but just that this paragraph should have been edited or removed. I think there is something wrong when I feel like I have to look up the actual research paper and check whether the claims made in an article are supported by data and methodology. I should not be a skeptic when reading New York times articles. To be fair, it is posted in the opinion section, but should we really just take this article as an opinion? That doesn't feel right to me either. Then onto your last question and Daniel Kahneman, we can talk about that for a long time, but let me keep it short. The best place I know (though technical) is the blog by Andrew Gelman ([1][2] turned up in a 5 second Google, but there is way more on his blog), and Daniel Kahneman himself has "admitted" flaws in his studies [3][4]. [1] http://andrewgelman.com/2014/09/03/disagree-alan-turing-daniel-kahneman-regarding-strength-statistical-evidence/ http://andrewgelman.com/2014/09/03/disagree-alan-turing-dani... [2] http://andrewgelman.com/2016/06/26/29449/ http://andrewgelman.com/2016/06/26/29449/ [3] http://retractionwatch.com/2017/02/20/placed-much-faith-underpowered-studies-nobel-prize-winner-admits-mistakes/ http://retractionwatch.com/2017/02/20/placed-much-faith-unde... [4] https://replicationindex.wordpress.com/2017/02/02/reconstruction-of-a-train-wreck-how-priming-research-went-of-the-rails/ https://replicationindex.wordpress.com/2017/02/02/reconstruc...
Consultant32452 10y ago
I worked in the IT arm of a household name US non-IT company (just to provide basic context). Eventually we determined that interviewing just wasn't that helpful. We started telling the recruiting company, "Send me the best you've got that's available by Tuesday." Now certainly the recruiter is going to do some selection biasing there, but we found we had just as much success as our previous interviewing process that had a phone interview, face to face interview, and a test.
jwatte 10y ago
The fact that psychology (and other social sciences and even medicine) have a replication crisis in spades is well known. Just Google it, if you haven't been following the general science news for the last few years. Even the "hard" sciences have trouble, because journals prefer publishing new and positive results, rather than replications or negatives.

mooreds 10y ago

> ok, this is interesting, where is it mentioned https://mobile.nytimes.com/2015/08/28/science/many-social-science-findings-not-as-strong-as-claimed-study-says.html https://mobile.nytimes.com/2015/08/28/science/many-social-sc... among other places

andrewmutz 10y ago

It's easy to present data that shows flaws with today's interviewing. It's a lot harder to present a better way of predicting candidate performance in the workplace, along with substantial data that indicates it's better than today's methods. Corporations would love more effective ways to determine effectiveness/performance before hiring. Interviewing is terrible, but that doesn't mean there is a better option.

emodendroket 10y ago
That feels a bit like saying we should just stick to blood letting and leeches because we don't know any more effective way of treating disease. Just because we don't have a superior alternative doesn't actually mean the current method is effective.
- RugnirViking 10y ago
  A) Leeches are often used in literal modern hospitals in the developed world as an effective treatment for certain ills. B) If we had no alternative, we should stick to it, but look at other things in the meantime. You seem to be advocating doing nothing at all, which is trivially easy to show works for no-one
  - emodendroket 10y ago
    Yes, leeches have extremely limited use. However, bloodletting and leeches killed far more people than they ever "treated."
jaggederest 10y ago
I strongly disagree. There are all sorts of situations where having a bad option is much worse than having no option, hiring and medicine clearly among them. > Corporations would love more effective ways to determine effectiveness/performance before hiring. This is irrelevant to evaluating the current methods. Even if there is no replacement, if they are useless, we should know it, bar none.
jernfrost 10y ago
Nonsense. First step is to acknowledge that interviewing doesn't work very well. You can't keep deluding yourself because you haven't found a better way. Accept reality. This is one of the biggest problems I see with business guys today. They want absolute certainty in a world which can't offer it. Start accepting that there is a lot of stuff we don't know and can't know at the moment and we simply have to work towards getting better. You can't get better if you don't acknowledge that there is a problem in the first place. These silly, must have +5 years of experience, and check all these boxes with technologies to get the job clearly shows the industry is completely lost at the moment and isn't willing to acknowledge it.
- mahyarm 10y ago
  Of course people know interviews aren't the greatest, but it's the best tool they have right now. What people are pushing back against is criticism without solutions. There are a bunch of other testing methods that are illegal in the USA too, such as IQ tests.
  - LnxPrgr3 10y ago
    When users of code I've written complain that something sucks to use, I don't demand they come up with a better solution. Probably I don't even want to know what they think the answer is—unless they design user interfaces for a living, odds are their answer will be terrible and they won't even be happy with it if I build it. This is as it should be—I'm paid to solve problems. Why is this different? The more I read about interviewing, the more I realize too many people think they have this problem solved—their amateur psychology is impeccable and their technical screens test for exactly the right things, no more and no less. Did they do a bunch of controlled studies to convince themselves of this, or are they taking sounding good, or intuition about the statistical outcomes of different techniques, to be equivalent to truth? Maybe the first step is to collectively realize we have close to no clue what we're doing, and are being asked to solve a hard problem: individually, to talk to someone for an hour and make a hiring recommendation. In aggregate, to make the decision based on a handful of these one-hour conversations. Maybe the first step is to realize this is a problem worth trying to solve. Maybe the first step is vocal non-acceptance.
Sevii 10y ago
We have had a better way for decades, its banned. What do you think all these algorithm questions are supposed to do?
- mafm 10y ago
  What is the better way that is banned?
  - frankc 10y ago
    I suspect he is referring to iq tests. They have a pretty high correlation to success and are banned.
    - yorwba 10y ago
      I wonder how much correlation with IQ is allowed before custom algorithms questions qualify as an illegal IQ test.
      - Spooky23 10y ago
        Employees will test the limits with lawsuits. I know at least one large state lost a class action lawsuit due to racial bias on civil service exams. The problem with comparing IQ results at hire to employment success is that employment outcome is difficult to define over time. You're also unlikely to get statistically relevant data without focusing on large organizations with standardized HR processes. Most of the research is based on supervisory evaluations, which are not the most reliable indicators of anything for a variety of reasons. The other thing I find amusing is that business folk who talk about this miss the fact that there are large workforces in the US that either have or do use standardized testing like this to hire and promote. Those are government bureaucracies, which function relatively well, but are hardly a model that most folks advocating this would aspire towards.
      - lr4444lr 10y ago
        I don't think the issue is correlation, but rather disparate impact[0]. Algorithmic tests have a pretty low bar to clear in IT jobs. Using them for secretaries or lab technicians is another story. https://en.wikipedia.org/wiki/Disparate_impact https://en.wikipedia.org/wiki/Disparate_impact

jaredklewis 10y ago

People have too much faith in the efficacy of interviewing, ergo we should give slipshod journalism and science lacking rigor a free pass? I'm all for upending the status quo with interviews, but let's not throw out science and reporting just to get there.

schmit 10y ago

My point is not that this work is flawed, or that there should not be an article reporting on this research or the topic of interviewing. Rather, I think it'd be better for a third party to write about the topic in a more objective manner, rather than a professor promoting his own research (and thus with skewed incentives). In particular, I was disappointed to find a (short) paragraph in the article that I find bogus. That does not mean the article shouldn't have been posted in the first place, but just that this paragraph should have been edited or removed. I think there is something wrong when I feel like I have to look up the actual research paper and check whether the claims made in an article are supported by data and methodology. I should not be a skeptic when reading New York times articles. To be fair, it is posted in the opinion section, but should we really just take this article as an opinion? That doesn't feel right to me either. Then onto your last question and Daniel Kahneman, we can talk about that for a long time, but let me keep it short. The best place I know (though technical) is the blog by Andrew Gelman ([1][2] turned up in a 5 second Google, but there is way more on his blog), and Daniel Kahneman himself has "admitted" flaws in his studies [3][4]. [1] http://andrewgelman.com/2014/09/03/disagree-alan-turing-daniel-kahneman-regarding-strength-statistical-evidence/ http://andrewgelman.com/2014/09/03/disagree-alan-turing-dani... [2] http://andrewgelman.com/2016/06/26/29449/ http://andrewgelman.com/2016/06/26/29449/ [3] http://retractionwatch.com/2017/02/20/placed-much-faith-underpowered-studies-nobel-prize-winner-admits-mistakes/ http://retractionwatch.com/2017/02/20/placed-much-faith-unde... [4] https://replicationindex.wordpress.com/2017/02/02/reconstruction-of-a-train-wreck-how-priming-research-went-of-the-rails/ https://replicationindex.wordpress.com/2017/02/02/reconstruc...

Consultant32452 10y ago

I worked in the IT arm of a household name US non-IT company (just to provide basic context). Eventually we determined that interviewing just wasn't that helpful. We started telling the recruiting company, "Send me the best you've got that's available by Tuesday." Now certainly the recruiter is going to do some selection biasing there, but we found we had just as much success as our previous interviewing process that had a phone interview, face to face interview, and a test.

jwatte 10y ago

The fact that psychology (and other social sciences and even medicine) have a replication crisis in spades is well known. Just Google it, if you haven't been following the general science news for the last few years. Even the "hard" sciences have trouble, because journals prefer publishing new and positive results, rather than replications or negatives.