5 ms·

> Laszlo Bock studied the predictive value of technical interviews when he was in charge of people operations at Google. He reported finding that they had zero

> Laszlo Bock studied the predictive value of technical interviews when he was in charge of people operations at Google. He reported finding that they had zero predictive power. Mind citing a source here? That would be a pretty big deal, both because it flies in the face of the entire industry, and because of the implications about Google if it determined the interviews were useless and then continued spending an incredible amount of money conducting them.

mikelevins 5y ago
I was relying on my own memory, so feel free to disregard my remarks. If you want to know what Bock has said about the subject, Google for Laszlo Bock along with some terms like "hiring" and "predictive". His book on the subject is here: https://www.goodreads.com/book/show/22875447-work-rules https://www.goodreads.com/book/show/22875447-work-rules It says that brain teasers and unstructured interviews don't work. Structured interviews do. So what's a structured interview? It's a standardized set of predefined factual questions determined by relevant domain experts to be directly relevant to the job. In effect, a custom standardized test. He says the problem with them is that interviewers don't like using them. Oh, and he says the problem with unstructured interviews--which interviewers prefer--is that the best predictor of the interview's result is an impression that the interviewer forms in the first ten seconds, and that impression has zero predictive power about the candidate's suitability for the job. Google a little more widely to find more general research in this area. For example: https://www.researchgate.net/publication/240237107_The_Role_of_General_Cognitive_Ability_and_Job_Performance_Why_There_Cannot_Be_a_Debate https://www.researchgate.net/publication/240237107_The_Role_... ...which says the best predictor overall is general cognitive ability (what we usually call IQ) and https://www.academia.edu/32446971/Big_Five_Personality_Dimensions_and_Job_Performance_in_Army_and_Civil_Occupations_A_European_Perspective https://www.academia.edu/32446971/Big_Five_Personality_Dimen... ...which says that two of the Big Five personality traits (namely, Conscientiousness and Emotional Stability) are good predictors of good job performance. IQ, by the way, is strongly correlated with another of the Big Five, generally called Openness. So if we believe all of this research, then what you sould be looking for is objectively-verifiable skills directly related to what you expect the job to actually entail, combined with general intelligence, conscientiousness, and emotional stability. By the way, I score low in emotional stability, so I have to rely on the other traits. If you find that my characterizations are grossly wrong, feel free to trash me over them and I will thank you for improving my understanding and humbly apologize for wasting your time.
itsdrewmiller 5y ago
Yeah pretty sure this is BS. Bock talked extensively about interviews in his book. I recall him saying that after 4 interviews there were clear diminishing returns. If the interviews had no value, the diminishing returns would have been total at one.
- mikelevins 5y ago
  A fair point. I reread, and his claims were more specific than I allowed: brain-teaser-style questions have no predictive value; neither do "unstructured interviews", because the interviewers generally form an impression in the first ten seconds, that impression has no predictive value, and the interviewers then generally spend the rest of the interview confirming that initial impression. However, structured interviews--interviews that consist of standardized domain-specific questions about relevant skills--do have predictive value. The problem with them is that interviewers don't like doing them. I conflated the specific kinds of interviewing he deprecated with "the technical interview", I assume because those are the kinds of technical interviews I've seen. In thirty years or so, I've never seen the kind of structured interview Bock praised. Still, conflating the different things was my mistake. Thanks for correcting me.

mikelevins 5y ago

I was relying on my own memory, so feel free to disregard my remarks. If you want to know what Bock has said about the subject, Google for Laszlo Bock along with some terms like "hiring" and "predictive". His book on the subject is here: https://www.goodreads.com/book/show/22875447-work-rules https://www.goodreads.com/book/show/22875447-work-rules It says that brain teasers and unstructured interviews don't work. Structured interviews do. So what's a structured interview? It's a standardized set of predefined factual questions determined by relevant domain experts to be directly relevant to the job. In effect, a custom standardized test. He says the problem with them is that interviewers don't like using them. Oh, and he says the problem with unstructured interviews--which interviewers prefer--is that the best predictor of the interview's result is an impression that the interviewer forms in the first ten seconds, and that impression has zero predictive power about the candidate's suitability for the job. Google a little more widely to find more general research in this area. For example: https://www.researchgate.net/publication/240237107_The_Role_of_General_Cognitive_Ability_and_Job_Performance_Why_There_Cannot_Be_a_Debate https://www.researchgate.net/publication/240237107_The_Role_... ...which says the best predictor overall is general cognitive ability (what we usually call IQ) and https://www.academia.edu/32446971/Big_Five_Personality_Dimensions_and_Job_Performance_in_Army_and_Civil_Occupations_A_European_Perspective https://www.academia.edu/32446971/Big_Five_Personality_Dimen... ...which says that two of the Big Five personality traits (namely, Conscientiousness and Emotional Stability) are good predictors of good job performance. IQ, by the way, is strongly correlated with another of the Big Five, generally called Openness. So if we believe all of this research, then what you sould be looking for is objectively-verifiable skills directly related to what you expect the job to actually entail, combined with general intelligence, conscientiousness, and emotional stability. By the way, I score low in emotional stability, so I have to rely on the other traits. If you find that my characterizations are grossly wrong, feel free to trash me over them and I will thank you for improving my understanding and humbly apologize for wasting your time.

itsdrewmiller 5y ago

Yeah pretty sure this is BS. Bock talked extensively about interviews in his book. I recall him saying that after 4 interviews there were clear diminishing returns. If the interviews had no value, the diminishing returns would have been total at one.

mikelevins 5y ago
A fair point. I reread, and his claims were more specific than I allowed: brain-teaser-style questions have no predictive value; neither do "unstructured interviews", because the interviewers generally form an impression in the first ten seconds, that impression has no predictive value, and the interviewers then generally spend the rest of the interview confirming that initial impression. However, structured interviews--interviews that consist of standardized domain-specific questions about relevant skills--do have predictive value. The problem with them is that interviewers don't like doing them. I conflated the specific kinds of interviewing he deprecated with "the technical interview", I assume because those are the kinds of technical interviews I've seen. In thirty years or so, I've never seen the kind of structured interview Bock praised. Still, conflating the different things was my mistake. Thanks for correcting me.