7 ms·

I have found myself wishing a lot over the last year or so that interviewing was less of an art and more of a science. The writer talks about being quizzed on

I have found myself wishing a lot over the last year or so that interviewing was less of an art and more of a science. The writer talks about being quizzed on a google product she had only superficially used. She recommends that people do their homework more, but honestly, even a dedicated googlophile would struggle to have tried every google product in depth. She was unlucky that they asked about that specific product. Honestly, there is a lot more luck in the interview process than makes me comfortable. The difference between getting the job vs losing it can be that the interviewer chose to ask you about smart pointers rather than heap allocation, and that's galling when the reality of the job is that you're going to spend half your time moving text boxes by two pixels and changing log formats to suit the new Unified Log Structure your sysadmins have dreamed up.

ArekDymalski 12y ago
> interviewing was less of an art and more of a science I'd extend your wish to "Whole recruitment process should be less of an art and more of a science". The problem is that even the properly conducted interview using established, good practices (like competence based/behavioral interview) can give you max. 50% accuracy in assessing the competencies. What means that additional methods are absolutely necessary. But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position. And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods.
- MikeTaylor 12y ago
  The point of fizz-buzz and similar exercises is not to find suitable candidates. It's to eliminate plainly unsuitable candidates, very early in the process, at minimal cost.
- blowski 12y ago
  Until messy human beings are replaced with perfectly rational robots, the recruitment process is always going to be an 'art'. Roles are often vaguely defined, because it depends on who is available. People lie, exaggerate and game the system in interviews. Neither side wants to reveal their weak points. Figuring out the underlying truth and matching the best available candidate to the best possible job description is not something algorithms are good at.
  - ArekDymalski 12y ago
    You are right. However, there's a whole spectrum between messy and irrational human being and perfect robot. My point was that people should take an effort to reduce/control their imperfection, biases etc.. The choice of "I'll be messy because I can't be as good as robot" is easy but flawed.
- shamney 12y ago
  are competency based interviews really a well established method of finding the best people?
  - ArekDymalski 12y ago
    By "established" I meant: defined and researched (the 50% accuracy I mentioned above comes from the literature). But they aren't "established" as popular/widely used. Partly because they require well defined competencies, but also because they aren't easy to conduct in comparison to typical interview (which has the accuracy around 20% AFAIR).
    - shamney 12y ago
      could you link to some of this literature? competency based interviews are standard for junior positions with large companies in the UK. however, this approach often turns interviews into a bullshitting competition.
      - sfilipov 12y ago
        > this approach often turns interviews into a bullshitting competition. That is also my experience. As a result I generally avoid applying to big companies where I expect a lot of non-technical interviews. If that's how everyone is hired, I don't want to end up in a MENSA club instead of a place where people have strong technical abilities.
      - ArekDymalski 12y ago
        Here comes the full text of an old research comparing traditional interviewing with other methods: https://www.uam.es/personal_pdi/psicologia/pei/diferencias/Hunter1984JobPerformance.pdf https://www.uam.es/personal_pdi/psicologia/pei/diferencias/H... (tl;dr traditional interviwing has validity .14) Here's the more recent, often cited meta-analysis: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%... (tl;dr structured interview has validity of .51) Naturally it has an extensive bibliography if you'd like to dive deeper in the topic. And even more references can be found here: http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20et%20al%201994%20JAP%20Validity%20of%20interviews.pdf http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20...
- raverbashing 12y ago
  "But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position. And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods. " Yes, I agree Nobody should write FizzBuzz on the board exactly because if you get a person that can't do that your initial filter/selection is wrong. Dead wrong.
  - AnimalMuppet 12y ago
    Let's say that my initial filter/selection is less than 100% bullet proof. Now what? Do I pretend that it's perfect? Or do I try to find a very inexpensive way to put a safety net behind my initial filter?
    - raverbashing 12y ago
      There are less stupid ways of testing it than FizzBuzz Before hiring an architect do you ask him to sketch a house? Before hiring a surgeon do you ask him to do a mini surgery? Then don't waste my time with fizzbuzz! Shoot a harder question (even a coding question) and this is saving on time and mutual trust.
- lolo_ 12y ago
  I've given a lot of interviews (>100) and while I totally accept it's a best guest and imperfect, I find that you get more out of fizzbuzz and the like than meets the eye. Firstly, I certainly agree unnecessary algo questions for a job that doesn't involve algorithmic problem solving on a day-to-day is silly and leads to false negatives. But the simpler problems give you more than just the basic 'can you code' test - it gives you an insight their into thought process. For example, seeing how somebody progresses from:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); // ah hm both % 3 and %5... } To:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0 && i % 5 == 0) console.log('fizzbuzz'); else if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); ... } And dealing with the fact they have a mistake that needs fixing is always interesting to observe, and gives you insight into how they approach problems. It seems so incredibly trivial and irrelevant at first hand but shows you a lot (assuming they've not seen it before.) I've been really surprised by this. I suspect the idea of there being a science of this is a bit of a mistake - there's plenty of stuff that involves so many factors that it's really hard to know the really ideal approach. A quick look at the actual state of software engineering studies (30 undergrad students != all programmers) or e.g. macroeconomics should give you pause. Certainly loads of scope for improvement though, I agree wholeheartedly!
  - frownie 12y ago
    I concur. When hiring with these kinds of small problems, you actually see a lot. First, in my case, it sorts programmers from not-programmers (it's in the 50% ballpark). Then with a problem a bit harder, it helps sort "programmer-in-the-guts" from "programmer-for-a-job". At the end, I've removed 90% of the candidates and I'm left with average to very good programmers. Moreover, leaving people to do these little tests in the team and proposing them to ask questions whenever they want, plus asking them to explain their solutions helps a lot in seeing the social skill I need (communication, openness...) Of course, it's not 100% safe, but I'd say it's a good heuristic :-)
    - lolo_ 12y ago
cpach 12y ago
'tptacek has written lots of insightful comments about a more scientific approach to hiring. One example: https://news.ycombinator.com/item?id=8770030 https://news.ycombinator.com/item?id=8770030
- smtddr 12y ago
  This is the first comment from tptacek that I disagree with; strongly. I know this HN audience loves its scientific approach to everything and everyone in life, but I actually don't. I'll probably never accept applying science to dealing with people simply because algorithms would never have enough inputs to describe a person accurately. I much prefer the "organic" discussion with people to get an idea of how they think and feel. If a company gave me a job offer without me getting a good idea of the thoughts & feelings of my coworkers, or me having doubts that they know mine, I probably wouldn't take the job. At my current gig I'm usually one of the culture-fit interviewers. FWIW, in my most recent performance-review I was described as having high "EQ". http://www.huffingtonpost.com/2013/12/05/are-you-emotionally-intel_n_4371920.html http://www.huffingtonpost.com/2013/12/05/are-you-emotionally... I've never heard that term before, but having read about it I think it fits me perfectly.
  - tptacek 12y ago
    What part of that comment do you disagree with? It recommends a lot of different things.
    - smtddr 12y ago
      In general, I just don't like trying to describe people with science. That's a personal philosophy that I don't think I can really convince anyone else of if they don't already believe it. But specifically: >> Do not allow team members to contribute feedback based on "culture fit". I'm aware of how culture fit can lead to unfair discrimination, but throwing it out entirely I think is like "throwing the baby out with the bath water." I think culture-fit is important to have the right "vibe" in the office. Unfortunately, I know that "vibe preservation" can lead to aforementioned discrimination very easily but I also believe the "vibe" does exist and damaging it results in problems. >> Try to eliminate phone interviews entirely. They don't add value, but do amplify noise. If phone-interviews were 100% gone, then we'd need 2 in-person interviews. The first interview asking pretty much the same questions that could have been done over the phone. I think phone-interviews help screen out obvious bad-fits early on without investing too much time. >> Never interview a candidate with more than one interviewer From my own experience I've seen 2 interviewers describe a candidate almost completely opposite from each other and I found it incredibly fascinating to see how different 2 people witness the exact same event at the same time. Everyone being aware of those differences and the rest of the team knowing how the interviewers think day-to-day, I think gives balance. >> Have lunch with candidates if you like, but don't make lunch a Q&A with the candidate. What would you talk about at lunch if not Q&A? Or are we assuming that you specifically mean prohibit question to the candidate but expecting questions _from_ the candidate. >> Script your interviews. While I do think it's important to have some kind of benchmark sequence of questions, I think more is learned about the candidate(and I learn more about my potential new coworkers) when it feels more like an organic conversation rather than those scripted conversations I endure when calling AT&T about my malfunctioning LAN line. Or maybe you don't mean scripted in that strict of a form. Incidentally, do you have some kind of script notifying you when you're mentioned on HN? Sometimes it feels that way. You kinda "non-organically" appear when someone mentions you. :-P
jghn 12y ago
A friend in I were just remarking last night how ridiculously stochastic the hiring process really is. I'm convinced that if we had perfect insight to all tech hiring in the universe that the comparison of hire/nohire and wouldbegood/wouldbebad is so close to being about 50% all around that this is all for naught.

ArekDymalski 12y ago

> interviewing was less of an art and more of a science I'd extend your wish to "Whole recruitment process should be less of an art and more of a science". The problem is that even the properly conducted interview using established, good practices (like competence based/behavioral interview) can give you max. 50% accuracy in assessing the competencies. What means that additional methods are absolutely necessary. But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position. And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods.

MikeTaylor 12y ago
The point of fizz-buzz and similar exercises is not to find suitable candidates. It's to eliminate plainly unsuitable candidates, very early in the process, at minimal cost.
blowski 12y ago
Until messy human beings are replaced with perfectly rational robots, the recruitment process is always going to be an 'art'. Roles are often vaguely defined, because it depends on who is available. People lie, exaggerate and game the system in interviews. Neither side wants to reveal their weak points. Figuring out the underlying truth and matching the best available candidate to the best possible job description is not something algorithms are good at.
- ArekDymalski 12y ago
  You are right. However, there's a whole spectrum between messy and irrational human being and perfect robot. My point was that people should take an effort to reduce/control their imperfection, biases etc.. The choice of "I'll be messy because I can't be as good as robot" is easy but flawed.
shamney 12y ago
are competency based interviews really a well established method of finding the best people?
- ArekDymalski 12y ago
  By "established" I meant: defined and researched (the 50% accuracy I mentioned above comes from the literature). But they aren't "established" as popular/widely used. Partly because they require well defined competencies, but also because they aren't easy to conduct in comparison to typical interview (which has the accuracy around 20% AFAIR).
  - shamney 12y ago
    could you link to some of this literature? competency based interviews are standard for junior positions with large companies in the UK. however, this approach often turns interviews into a bullshitting competition.
    - sfilipov 12y ago
      > this approach often turns interviews into a bullshitting competition. That is also my experience. As a result I generally avoid applying to big companies where I expect a lot of non-technical interviews. If that's how everyone is hired, I don't want to end up in a MENSA club instead of a place where people have strong technical abilities.
    - ArekDymalski 12y ago
      Here comes the full text of an old research comparing traditional interviewing with other methods: https://www.uam.es/personal_pdi/psicologia/pei/diferencias/Hunter1984JobPerformance.pdf https://www.uam.es/personal_pdi/psicologia/pei/diferencias/H... (tl;dr traditional interviwing has validity .14) Here's the more recent, often cited meta-analysis: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%... (tl;dr structured interview has validity of .51) Naturally it has an extensive bibliography if you'd like to dive deeper in the topic. And even more references can be found here: http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20et%20al%201994%20JAP%20Validity%20of%20interviews.pdf http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20...
raverbashing 12y ago
"But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position. And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods. " Yes, I agree Nobody should write FizzBuzz on the board exactly because if you get a person that can't do that your initial filter/selection is wrong. Dead wrong.
- AnimalMuppet 12y ago
  Let's say that my initial filter/selection is less than 100% bullet proof. Now what? Do I pretend that it's perfect? Or do I try to find a very inexpensive way to put a safety net behind my initial filter?
  - raverbashing 12y ago
    There are less stupid ways of testing it than FizzBuzz Before hiring an architect do you ask him to sketch a house? Before hiring a surgeon do you ask him to do a mini surgery? Then don't waste my time with fizzbuzz! Shoot a harder question (even a coding question) and this is saving on time and mutual trust.
lolo_ 12y ago
I've given a lot of interviews (>100) and while I totally accept it's a best guest and imperfect, I find that you get more out of fizzbuzz and the like than meets the eye. Firstly, I certainly agree unnecessary algo questions for a job that doesn't involve algorithmic problem solving on a day-to-day is silly and leads to false negatives. But the simpler problems give you more than just the basic 'can you code' test - it gives you an insight their into thought process. For example, seeing how somebody progresses from:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); // ah hm both % 3 and %5... } To:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0 && i % 5 == 0) console.log('fizzbuzz'); else if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); ... } And dealing with the fact they have a mistake that needs fixing is always interesting to observe, and gives you insight into how they approach problems. It seems so incredibly trivial and irrelevant at first hand but shows you a lot (assuming they've not seen it before.) I've been really surprised by this. I suspect the idea of there being a science of this is a bit of a mistake - there's plenty of stuff that involves so many factors that it's really hard to know the really ideal approach. A quick look at the actual state of software engineering studies (30 undergrad students != all programmers) or e.g. macroeconomics should give you pause. Certainly loads of scope for improvement though, I agree wholeheartedly!
- frownie 12y ago
  I concur. When hiring with these kinds of small problems, you actually see a lot. First, in my case, it sorts programmers from not-programmers (it's in the 50% ballpark). Then with a problem a bit harder, it helps sort "programmer-in-the-guts" from "programmer-for-a-job". At the end, I've removed 90% of the candidates and I'm left with average to very good programmers. Moreover, leaving people to do these little tests in the team and proposing them to ask questions whenever they want, plus asking them to explain their solutions helps a lot in seeing the social skill I need (communication, openness...) Of course, it's not 100% safe, but I'd say it's a good heuristic :-)
  - lolo_ 12y ago

cpach 12y ago

'tptacek has written lots of insightful comments about a more scientific approach to hiring. One example: https://news.ycombinator.com/item?id=8770030 https://news.ycombinator.com/item?id=8770030

smtddr 12y ago
This is the first comment from tptacek that I disagree with; strongly. I know this HN audience loves its scientific approach to everything and everyone in life, but I actually don't. I'll probably never accept applying science to dealing with people simply because algorithms would never have enough inputs to describe a person accurately. I much prefer the "organic" discussion with people to get an idea of how they think and feel. If a company gave me a job offer without me getting a good idea of the thoughts & feelings of my coworkers, or me having doubts that they know mine, I probably wouldn't take the job. At my current gig I'm usually one of the culture-fit interviewers. FWIW, in my most recent performance-review I was described as having high "EQ". http://www.huffingtonpost.com/2013/12/05/are-you-emotionally-intel_n_4371920.html http://www.huffingtonpost.com/2013/12/05/are-you-emotionally... I've never heard that term before, but having read about it I think it fits me perfectly.
- tptacek 12y ago
  What part of that comment do you disagree with? It recommends a lot of different things.
  - smtddr 12y ago
    In general, I just don't like trying to describe people with science. That's a personal philosophy that I don't think I can really convince anyone else of if they don't already believe it. But specifically: >> Do not allow team members to contribute feedback based on "culture fit". I'm aware of how culture fit can lead to unfair discrimination, but throwing it out entirely I think is like "throwing the baby out with the bath water." I think culture-fit is important to have the right "vibe" in the office. Unfortunately, I know that "vibe preservation" can lead to aforementioned discrimination very easily but I also believe the "vibe" does exist and damaging it results in problems. >> Try to eliminate phone interviews entirely. They don't add value, but do amplify noise. If phone-interviews were 100% gone, then we'd need 2 in-person interviews. The first interview asking pretty much the same questions that could have been done over the phone. I think phone-interviews help screen out obvious bad-fits early on without investing too much time. >> Never interview a candidate with more than one interviewer From my own experience I've seen 2 interviewers describe a candidate almost completely opposite from each other and I found it incredibly fascinating to see how different 2 people witness the exact same event at the same time. Everyone being aware of those differences and the rest of the team knowing how the interviewers think day-to-day, I think gives balance. >> Have lunch with candidates if you like, but don't make lunch a Q&A with the candidate. What would you talk about at lunch if not Q&A? Or are we assuming that you specifically mean prohibit question to the candidate but expecting questions _from_ the candidate. >> Script your interviews. While I do think it's important to have some kind of benchmark sequence of questions, I think more is learned about the candidate(and I learn more about my potential new coworkers) when it feels more like an organic conversation rather than those scripted conversations I endure when calling AT&T about my malfunctioning LAN line. Or maybe you don't mean scripted in that strict of a form. Incidentally, do you have some kind of script notifying you when you're mentioned on HN? Sometimes it feels that way. You kinda "non-organically" appear when someone mentions you. :-P

jghn 12y ago

A friend in I were just remarking last night how ridiculously stochastic the hiring process really is. I'm convinced that if we had perfect insight to all tech hiring in the universe that the comparison of hire/nohire and wouldbegood/wouldbebad is so close to being about 50% all around that this is all for naught.