5 ms·

I doubt there’s been too much on software interviewing specifically but interviewing/candidate selection for jobs is a large part of personnel psychology. A rec

I doubt there’s been too much on software interviewing specifically but interviewing/candidate selection for jobs is a large part of personnel psychology. A recent meta-analysis of what they know is below. Basically general mental ability (g, IQ) is the single best predictor, work sample tests work well, structured interviews can increase the performance of IQ tests or work samples and unstructured interviews are a trash fire. > The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 100 Years > This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and the 18 other selection procedures. Overall, the 3 combinations with the highest multivariate validity and utility for job performance were GMA plus a work sample test (mean validity of .63), GMA plus an integrity test (mean validity of .65), and GMA plus a structured interview (mean validity of .63). A further advantage of the latter 2 combinations is that they can be used for both entry level selection and selection of experienced employees. The practical utility implications of these summary findings are substantial. The implications of these research findings for the development of theories of job performance are discussed. https://www.semanticscholar.org/paper/The-Validity-and-Utility-of-Selection-Methods-in-%3A-Schmidt/235f263383afe7cda38ef9a49ceb5dc1090f4769 https://www.semanticscholar.org/paper/The-Validity-and-Utili...

dcolkitt 8y ago
Can't agree with this enough. The most consistent finding in industrial psychology is that general cognitive ability is by far the best predictor of job performance in virtually every role ever tested. Even jobs that don't seem cognitively demanding, like janitors or infantry. Higher IQ candidates almost universally do better, even if they start with much less experience. The takeaway as it applies to modern software hiring, is that skillset is way overemphasized. It's quite common to see job ads that heavily focus on the company's specific tech stack. ("Must have experience with Rails, React, Travis, and AWS") It's much better to cast a much wider net, and try to find the smartest people anywhere. High IQ people can easily re-tool their specific skillset. What's interesting is this is much closer to how the most successful tech firms, like Google, tend to hire.
- kayoone 8y ago
  I can't agree with this. High IQ can't be everything. Wouldn't this value experience and knowledge (like everything you learned in university) at zero? Imo there are a lot of great/amazing and productive Software Engineers that wouldn't do well in a whiteboard algorithm interview. The people that tend to do well, are the ones that have a lot of practice with the kind of problems.
  - ghaff 8y ago
    I read the parent as talking about narrow skill sets that a smart and motivated individual can pick up pretty quickly. If you're looking for a senior developer you probably don't want to hire someone who has never programmed even if they're widely acknowledged as really smart and an expert in ball bearing design. And obviously applies in lots of areas. If I'm looking for someone to head up a digital marketing or marketing research initiative, someone whose only work experience is software development is probably not the best choice no matter how smart and motivated they are. OTOH, as you get to the level where people are more managers than practitioners, their specific skill sets presumably start to make less of a difference. Added: And, as peer noted, people do shift careers significantly all the time. I never really directly used my undergrad (or grad) engineering degrees all that much.
  - pps43 8y ago
    > Wouldn't this value experience and knowledge (like everything you learned in university) at zero? Having switched professions twice, I now use practically nothing of what I learned in university from about second year forward. Math and to some extent physics are still relevant, but that's pretty much it. It might be painful to admit, but the practical value of that rather specialized knowledge that took me several years of hard work to obtain is pretty much zero now.
    - klntsky 8y ago
      > It might be painful to admit, but the practical value of that rather specialized knowledge that took me several years of hard work to obtain is pretty much zero now. It's sad that your professors didn't tell you beforehand that you were going to learn how to learn during that time, not just study a particular technology stack.
lmkg 8y ago
DO NOTE that the in the United States, the use of IQ tests for hiring is potentially legally fraught. A SCOTUS ruling[1] has found that IQ tests are assumed to disfavor minority employees, and therefore using them as a major factor in hiring decisions may run afoul of the Civil Rights Act. There are ways around this, but generally speaking you have to prove the specific test you are administering either does not disfavor minorities, or else show that it is directly related to the specific position you're hiring for. [1] https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co. I am not a lawyer, do not take legal advice from randos on the interwebs. You should consult with a Real Actual Professional on how to deal with this risk. There have been both laws and court cases since the above that impact the ruling, which I am not qualified to analyze.
tlogan 8y ago
That is so obvious - just hire people smarter than you. But that is so hard: we have pride, we have "company culture", we are stubborn, we are talking about "A players", etc. etc. So yes - try to make interviews structured and organized so that there is no random noise.
jameslk 8y ago
IQ tests don't measure creativity. Can I assume these studies suggest creativity is not very relevant to job performance?

dcolkitt 8y ago

Can't agree with this enough. The most consistent finding in industrial psychology is that general cognitive ability is by far the best predictor of job performance in virtually every role ever tested. Even jobs that don't seem cognitively demanding, like janitors or infantry. Higher IQ candidates almost universally do better, even if they start with much less experience. The takeaway as it applies to modern software hiring, is that skillset is way overemphasized. It's quite common to see job ads that heavily focus on the company's specific tech stack. ("Must have experience with Rails, React, Travis, and AWS") It's much better to cast a much wider net, and try to find the smartest people anywhere. High IQ people can easily re-tool their specific skillset. What's interesting is this is much closer to how the most successful tech firms, like Google, tend to hire.

kayoone 8y ago
I can't agree with this. High IQ can't be everything. Wouldn't this value experience and knowledge (like everything you learned in university) at zero? Imo there are a lot of great/amazing and productive Software Engineers that wouldn't do well in a whiteboard algorithm interview. The people that tend to do well, are the ones that have a lot of practice with the kind of problems.
- ghaff 8y ago
  I read the parent as talking about narrow skill sets that a smart and motivated individual can pick up pretty quickly. If you're looking for a senior developer you probably don't want to hire someone who has never programmed even if they're widely acknowledged as really smart and an expert in ball bearing design. And obviously applies in lots of areas. If I'm looking for someone to head up a digital marketing or marketing research initiative, someone whose only work experience is software development is probably not the best choice no matter how smart and motivated they are. OTOH, as you get to the level where people are more managers than practitioners, their specific skill sets presumably start to make less of a difference. Added: And, as peer noted, people do shift careers significantly all the time. I never really directly used my undergrad (or grad) engineering degrees all that much.
- pps43 8y ago
  > Wouldn't this value experience and knowledge (like everything you learned in university) at zero? Having switched professions twice, I now use practically nothing of what I learned in university from about second year forward. Math and to some extent physics are still relevant, but that's pretty much it. It might be painful to admit, but the practical value of that rather specialized knowledge that took me several years of hard work to obtain is pretty much zero now.
  - klntsky 8y ago
    > It might be painful to admit, but the practical value of that rather specialized knowledge that took me several years of hard work to obtain is pretty much zero now. It's sad that your professors didn't tell you beforehand that you were going to learn how to learn during that time, not just study a particular technology stack.

lmkg 8y ago

DO NOTE that the in the United States, the use of IQ tests for hiring is potentially legally fraught. A SCOTUS ruling[1] has found that IQ tests are assumed to disfavor minority employees, and therefore using them as a major factor in hiring decisions may run afoul of the Civil Rights Act. There are ways around this, but generally speaking you have to prove the specific test you are administering either does not disfavor minorities, or else show that it is directly related to the specific position you're hiring for. [1] https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co. I am not a lawyer, do not take legal advice from randos on the interwebs. You should consult with a Real Actual Professional on how to deal with this risk. There have been both laws and court cases since the above that impact the ruling, which I am not qualified to analyze.

tlogan 8y ago

That is so obvious - just hire people smarter than you. But that is so hard: we have pride, we have "company culture", we are stubborn, we are talking about "A players", etc. etc. So yes - try to make interviews structured and organized so that there is no random noise.

jameslk 8y ago

IQ tests don't measure creativity. Can I assume these studies suggest creativity is not very relevant to job performance?