5 ms·

> interviewing was less of an art and more of a science I'd extend your wish to "Whole recruitment process should be less of an art and more of a science".

> interviewing was less of an art and more of a science I'd extend your wish to "Whole recruitment process should be less of an art and more of a science". The problem is that even the properly conducted interview using established, good practices (like competence based/behavioral interview) can give you max. 50% accuracy in assessing the competencies. What means that additional methods are absolutely necessary. But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position. And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods.

MikeTaylor 12y ago
The point of fizz-buzz and similar exercises is not to find suitable candidates. It's to eliminate plainly unsuitable candidates, very early in the process, at minimal cost.
blowski 12y ago
Until messy human beings are replaced with perfectly rational robots, the recruitment process is always going to be an 'art'. Roles are often vaguely defined, because it depends on who is available. People lie, exaggerate and game the system in interviews. Neither side wants to reveal their weak points. Figuring out the underlying truth and matching the best available candidate to the best possible job description is not something algorithms are good at.
- ArekDymalski 12y ago
  You are right. However, there's a whole spectrum between messy and irrational human being and perfect robot. My point was that people should take an effort to reduce/control their imperfection, biases etc.. The choice of "I'll be messy because I can't be as good as robot" is easy but flawed.
shamney 12y ago
are competency based interviews really a well established method of finding the best people?
- ArekDymalski 12y ago
  By "established" I meant: defined and researched (the 50% accuracy I mentioned above comes from the literature). But they aren't "established" as popular/widely used. Partly because they require well defined competencies, but also because they aren't easy to conduct in comparison to typical interview (which has the accuracy around 20% AFAIR).
  - shamney 12y ago
    could you link to some of this literature? competency based interviews are standard for junior positions with large companies in the UK. however, this approach often turns interviews into a bullshitting competition.
    - sfilipov 12y ago
      > this approach often turns interviews into a bullshitting competition. That is also my experience. As a result I generally avoid applying to big companies where I expect a lot of non-technical interviews. If that's how everyone is hired, I don't want to end up in a MENSA club instead of a place where people have strong technical abilities.
    - ArekDymalski 12y ago
      Here comes the full text of an old research comparing traditional interviewing with other methods: https://www.uam.es/personal_pdi/psicologia/pei/diferencias/Hunter1984JobPerformance.pdf https://www.uam.es/personal_pdi/psicologia/pei/diferencias/H... (tl;dr traditional interviwing has validity .14) Here's the more recent, often cited meta-analysis: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%... (tl;dr structured interview has validity of .51) Naturally it has an extensive bibliography if you'd like to dive deeper in the topic. And even more references can be found here: http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20et%20al%201994%20JAP%20Validity%20of%20interviews.pdf http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20...
raverbashing 12y ago
"But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position. And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods. " Yes, I agree Nobody should write FizzBuzz on the board exactly because if you get a person that can't do that your initial filter/selection is wrong. Dead wrong.
- AnimalMuppet 12y ago
  Let's say that my initial filter/selection is less than 100% bullet proof. Now what? Do I pretend that it's perfect? Or do I try to find a very inexpensive way to put a safety net behind my initial filter?
  - raverbashing 12y ago
    There are less stupid ways of testing it than FizzBuzz Before hiring an architect do you ask him to sketch a house? Before hiring a surgeon do you ask him to do a mini surgery? Then don't waste my time with fizzbuzz! Shoot a harder question (even a coding question) and this is saving on time and mutual trust.
lolo_ 12y ago
I've given a lot of interviews (>100) and while I totally accept it's a best guest and imperfect, I find that you get more out of fizzbuzz and the like than meets the eye. Firstly, I certainly agree unnecessary algo questions for a job that doesn't involve algorithmic problem solving on a day-to-day is silly and leads to false negatives. But the simpler problems give you more than just the basic 'can you code' test - it gives you an insight their into thought process. For example, seeing how somebody progresses from:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); // ah hm both % 3 and %5... } To:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0 && i % 5 == 0) console.log('fizzbuzz'); else if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); ... } And dealing with the fact they have a mistake that needs fixing is always interesting to observe, and gives you insight into how they approach problems. It seems so incredibly trivial and irrelevant at first hand but shows you a lot (assuming they've not seen it before.) I've been really surprised by this. I suspect the idea of there being a science of this is a bit of a mistake - there's plenty of stuff that involves so many factors that it's really hard to know the really ideal approach. A quick look at the actual state of software engineering studies (30 undergrad students != all programmers) or e.g. macroeconomics should give you pause. Certainly loads of scope for improvement though, I agree wholeheartedly!
- frownie 12y ago
  I concur. When hiring with these kinds of small problems, you actually see a lot. First, in my case, it sorts programmers from not-programmers (it's in the 50% ballpark). Then with a problem a bit harder, it helps sort "programmer-in-the-guts" from "programmer-for-a-job". At the end, I've removed 90% of the candidates and I'm left with average to very good programmers. Moreover, leaving people to do these little tests in the team and proposing them to ask questions whenever they want, plus asking them to explain their solutions helps a lot in seeing the social skill I need (communication, openness...) Of course, it's not 100% safe, but I'd say it's a good heuristic :-)
  - lolo_ 12y ago

MikeTaylor 12y ago

The point of fizz-buzz and similar exercises is not to find suitable candidates. It's to eliminate plainly unsuitable candidates, very early in the process, at minimal cost.

blowski 12y ago

Until messy human beings are replaced with perfectly rational robots, the recruitment process is always going to be an 'art'. Roles are often vaguely defined, because it depends on who is available. People lie, exaggerate and game the system in interviews. Neither side wants to reveal their weak points. Figuring out the underlying truth and matching the best available candidate to the best possible job description is not something algorithms are good at.

ArekDymalski 12y ago
You are right. However, there's a whole spectrum between messy and irrational human being and perfect robot. My point was that people should take an effort to reduce/control their imperfection, biases etc.. The choice of "I'll be messy because I can't be as good as robot" is easy but flawed.

shamney 12y ago

are competency based interviews really a well established method of finding the best people?

ArekDymalski 12y ago
By "established" I meant: defined and researched (the 50% accuracy I mentioned above comes from the literature). But they aren't "established" as popular/widely used. Partly because they require well defined competencies, but also because they aren't easy to conduct in comparison to typical interview (which has the accuracy around 20% AFAIR).
- shamney 12y ago
  could you link to some of this literature? competency based interviews are standard for junior positions with large companies in the UK. however, this approach often turns interviews into a bullshitting competition.
  - sfilipov 12y ago
    > this approach often turns interviews into a bullshitting competition. That is also my experience. As a result I generally avoid applying to big companies where I expect a lot of non-technical interviews. If that's how everyone is hired, I don't want to end up in a MENSA club instead of a place where people have strong technical abilities.
  - ArekDymalski 12y ago
    Here comes the full text of an old research comparing traditional interviewing with other methods: https://www.uam.es/personal_pdi/psicologia/pei/diferencias/Hunter1984JobPerformance.pdf https://www.uam.es/personal_pdi/psicologia/pei/diferencias/H... (tl;dr traditional interviwing has validity .14) Here's the more recent, often cited meta-analysis: http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%20Validity%20and%20Utility%20Psychological%20Bulletin.pdf http://mavweb.mnsu.edu/howard/Schmidt%20and%20Hunter%201998%... (tl;dr structured interview has validity of .51) Naturally it has an extensive bibliography if you'd like to dive deeper in the topic. And even more references can be found here: http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20et%20al%201994%20JAP%20Validity%20of%20interviews.pdf http://www.people.vcu.edu/~mamcdani/Publications/McDaniel%20...

raverbashing 12y ago

"But even the most accurate methods will lead to wrong decisions when companies don't have precisely and adequately defined what competencies are actually required to perform well in a given position. And from my experience this is the core problem. People debate over questions/fizz-buzz/puzzles/whiteboard/whatever but usually do not invest enough time to determine what would they like to measure with these methods. " Yes, I agree Nobody should write FizzBuzz on the board exactly because if you get a person that can't do that your initial filter/selection is wrong. Dead wrong.

AnimalMuppet 12y ago
Let's say that my initial filter/selection is less than 100% bullet proof. Now what? Do I pretend that it's perfect? Or do I try to find a very inexpensive way to put a safety net behind my initial filter?
- raverbashing 12y ago
  There are less stupid ways of testing it than FizzBuzz Before hiring an architect do you ask him to sketch a house? Before hiring a surgeon do you ask him to do a mini surgery? Then don't waste my time with fizzbuzz! Shoot a harder question (even a coding question) and this is saving on time and mutual trust.

lolo_ 12y ago

I've given a lot of interviews (>100) and while I totally accept it's a best guest and imperfect, I find that you get more out of fizzbuzz and the like than meets the eye. Firstly, I certainly agree unnecessary algo questions for a job that doesn't involve algorithmic problem solving on a day-to-day is silly and leads to false negatives. But the simpler problems give you more than just the basic 'can you code' test - it gives you an insight their into thought process. For example, seeing how somebody progresses from:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); // ah hm both % 3 and %5... } To:- for(var i = 1; i <= 100; i++) { if (i % 3 == 0 && i % 5 == 0) console.log('fizzbuzz'); else if (i % 3 == 0) console.log('fizz'); else if (i % 5 == 0) console.log('buzz'); ... } And dealing with the fact they have a mistake that needs fixing is always interesting to observe, and gives you insight into how they approach problems. It seems so incredibly trivial and irrelevant at first hand but shows you a lot (assuming they've not seen it before.) I've been really surprised by this. I suspect the idea of there being a science of this is a bit of a mistake - there's plenty of stuff that involves so many factors that it's really hard to know the really ideal approach. A quick look at the actual state of software engineering studies (30 undergrad students != all programmers) or e.g. macroeconomics should give you pause. Certainly loads of scope for improvement though, I agree wholeheartedly!

frownie 12y ago
I concur. When hiring with these kinds of small problems, you actually see a lot. First, in my case, it sorts programmers from not-programmers (it's in the 50% ballpark). Then with a problem a bit harder, it helps sort "programmer-in-the-guts" from "programmer-for-a-job". At the end, I've removed 90% of the candidates and I'm left with average to very good programmers. Moreover, leaving people to do these little tests in the team and proposing them to ask questions whenever they want, plus asking them to explain their solutions helps a lot in seeing the social skill I need (communication, openness...) Of course, it's not 100% safe, but I'd say it's a good heuristic :-)
- lolo_ 12y ago