2 ms·

I guess if you're planning on wasting a candidate's entire day, you can afford wasting time on puzzles. (I have read the science that says we make decisions on

I guess if you're planning on wasting a candidate's entire day, you can afford wasting time on puzzles. (I have read the science that says we make decisions on the hiring issue far faster than in eight hours, which matches my experience, so why sit there and fiddle around all day when the decision has been made?) I find it easy to believe that many people make a decision in far less than eight hours of interviewing, however I do not believe that all interview processes lasting eight hours are a waste of time for the candidate and for the interviewers. To pick an example, what about a process involving twelve different interviewers, each of whom takes between a half hour and an hour, with the process spread over a certain amount of elapsed time? Can we really assert that we cannot make a better decision with this process than with having one interviewer spend thirty minutes with the candidate?

jerf 18y ago
I don't interview alone. We used to do interviews alone in sequence but realized we were just wasting time. You also skipped out on something I explicitly mentioned, which is that there are times I go over when it is called for. (Recall that the median interview result is "obviously no".) Also bear in mind when I talk about "wasting time", I'm also concerned about the candidates time. Spending 8 hours interviewing someone that you decided "no" on in the first 30 seconds is doing nobody favors. That people defend this practice boggles my mind. (I don't know if you're defending that or not, but I've seen it defended elsewhere.) (Also note I did not say you should turn them out in 30 seconds, that's not respectful either. They should have a chance to convince you you are wrong. It's rare, but it happens. But the third hour vs. the fourth hour is unlikely to produce any changes.) Time is money, and that includes the candidate's time. The idea of flying people out for a two-day interview cycle actually sort of angers me unless they're planning on compensating me for it, and I mean with more than a few free meals and a hotel stay.
- raganwald 18y ago
  You also skipped out on something I explicitly mentioned, which is that there are times I go over when it is called for. I didn't ignore that, I was thinking about it when I suggested people taking a variable amount of time to perform an interview. Spending 8 hours interviewing someone that you decided "no" on in the first 30 seconds is doing nobody favors. That people defend this practice boggles my mind. (I don't know if you're defending that or not, but I've seen it defended elsewhere.) I am absolutely not defending that practice. If, for example, you plan to have six people do consecutive interviews and you also give each person a veto, there is no reason to have person two through five do the interview if person one votes "NO HIRE." On the other hand, if your policy is to have a discussion after each interview where the interviewer raises concerns that subsequent interviewers may wish to address, then person one might terminate their interview but persons two through five might feel it's still worthwhile to continue. Or perhaps not. The idea of flying people out for a two-day interview cycle actually sort of angers me unless they're planning on compensating me for it, and I mean with more than a few free meals and a hotel stay I think you are mixing your strategy for getting hired with a discussion about the best strategy for hiring. I always try to remember that it is not my job to hire myself, therefore when putting together an interview process it may make sense to do things that would cause me personally to decline to pursue the opportunity. To give a very simple example, I personally do not like writing code in an interview. I strongly dislike trying to "Guess the coding style the interviewer is thinking of." I never know if what I write will be not clever enough or too clever, especially when given something ridiculously trivial to implement. I feel a lot of pressure. Nevertheless... I have had very good results asking candidates to write code in the interview process. Oh well!
  - jerf 18y ago
    The other thing I've embraced is that there isn't a right way to interview, or perhaps more accurately, there isn't anybody who can be trusted to identify it. This again factors into my willingness to not feel too bound up by convention. I certainly may have brought my own feelings into the multi-day marathon issue, but it's for the purpose of making sure I understand the other side of the table. I think not appreciating giving away two days of your time is of a different nature than not liking to write code in an interview. The latter seems pretty clearly like a "suck it up" problem, if you know what I mean. Personally, I consider the interview being unpleasant for the interviewee is a given, and all I can do is minimize that, I can never eliminate it, because the stress and uncertainly is sufficient on its own to make the process unpleasant. In interviews when I ask code problems, I try to do my best to work with the interviewee. They get to pick the language, and I'm generally not looking at any style issues at all (if by "style" we mean "things that have no impact on running code"). If anything I'm a little too accommodating when I also ask them to choose their own data structures for the problems in question. This throws a surprisingly large number of people. (It's the price of allowing people to pick their implementation language; I'm not going to go write a separate test problem for each of the ten languages I might reasonably expect to get an answer in, especially when I don't even know them all. I actually feel I've sort of blundered into a good test here; a surprising number of people know "arrays" and nothing else and choose very bad representations for things of their own free will.)
jwilliams 18y ago
All the research I've seen seems to draw the conclusion that the only strong predictor of performance in a job -- is past performance. Puzzles and psychometric testing are useful, but have a lot of limitations. However, you may have a job that involves a high level of responsibility (e.g. people's lives) and you want to assess their response to stress/crisis. Again, it's not perfect -- you don't truely know how someone will respond in a unique situation -- but in those circumstances you might want all the information possible. References, by and large, are useless. Cultural fit is incredibly important, but there are very few interviewers or interview techniques that do this adequately. There is so much bias in this it's hard to assess very well. So what are the keys for me? Validating what they have on their CV. If they have experience with a particularly technology, I ask about it. I propose problems that they are likely to encounter every day in their job, and see how they go about fixing it. From what I've seen, abstract problems do not exercise this sufficiently. What is it from there? Well, if what they have been doing is a fit for the job, then that's great. If there is a gap, there is an assessment on what it will take to bridge. How a person has conducted their career is a good indicator on how achievable this is.
- raganwald 18y ago
  All the research I've seen seems to draw the conclusion that the only strong predictor of performance in a job -- is past performance. You are replying to a comment suggesting that there may be value in a process using multiple interviewers over a total time of eight hours. It could well be that one, some, many, or even all of the multiple people involved use your suggested process and that that it takes a total of eight hours to get a good read on past performance. So... is your comment orthogonal to mine?
  - jwilliams 18y ago
    So... is your comment orthogonal to mine? Ahha - yeah - I think I started to reply and wandered off topic :) I think it was more in reply to the OP's original comment about puzzles being a waste of time. I think 8 hours is fine - if it's progressing in the right areas. I tend not to like panel interviews, but a broad range of interviewers isn't a bad idea - the best interviews I've had usually include walking the floor a bit, seeing the environment, chatting to potential teammates - although I'd worry that it would be hard to get a consensus if too many people are involved.

jerf 18y ago

I don't interview alone. We used to do interviews alone in sequence but realized we were just wasting time. You also skipped out on something I explicitly mentioned, which is that there are times I go over when it is called for. (Recall that the median interview result is "obviously no".) Also bear in mind when I talk about "wasting time", I'm also concerned about the candidates time. Spending 8 hours interviewing someone that you decided "no" on in the first 30 seconds is doing nobody favors. That people defend this practice boggles my mind. (I don't know if you're defending that or not, but I've seen it defended elsewhere.) (Also note I did not say you should turn them out in 30 seconds, that's not respectful either. They should have a chance to convince you you are wrong. It's rare, but it happens. But the third hour vs. the fourth hour is unlikely to produce any changes.) Time is money, and that includes the candidate's time. The idea of flying people out for a two-day interview cycle actually sort of angers me unless they're planning on compensating me for it, and I mean with more than a few free meals and a hotel stay.

raganwald 18y ago
You also skipped out on something I explicitly mentioned, which is that there are times I go over when it is called for. I didn't ignore that, I was thinking about it when I suggested people taking a variable amount of time to perform an interview. Spending 8 hours interviewing someone that you decided "no" on in the first 30 seconds is doing nobody favors. That people defend this practice boggles my mind. (I don't know if you're defending that or not, but I've seen it defended elsewhere.) I am absolutely not defending that practice. If, for example, you plan to have six people do consecutive interviews and you also give each person a veto, there is no reason to have person two through five do the interview if person one votes "NO HIRE." On the other hand, if your policy is to have a discussion after each interview where the interviewer raises concerns that subsequent interviewers may wish to address, then person one might terminate their interview but persons two through five might feel it's still worthwhile to continue. Or perhaps not. The idea of flying people out for a two-day interview cycle actually sort of angers me unless they're planning on compensating me for it, and I mean with more than a few free meals and a hotel stay I think you are mixing your strategy for getting hired with a discussion about the best strategy for hiring. I always try to remember that it is not my job to hire myself, therefore when putting together an interview process it may make sense to do things that would cause me personally to decline to pursue the opportunity. To give a very simple example, I personally do not like writing code in an interview. I strongly dislike trying to "Guess the coding style the interviewer is thinking of." I never know if what I write will be not clever enough or too clever, especially when given something ridiculously trivial to implement. I feel a lot of pressure. Nevertheless... I have had very good results asking candidates to write code in the interview process. Oh well!
- jerf 18y ago
  The other thing I've embraced is that there isn't a right way to interview, or perhaps more accurately, there isn't anybody who can be trusted to identify it. This again factors into my willingness to not feel too bound up by convention. I certainly may have brought my own feelings into the multi-day marathon issue, but it's for the purpose of making sure I understand the other side of the table. I think not appreciating giving away two days of your time is of a different nature than not liking to write code in an interview. The latter seems pretty clearly like a "suck it up" problem, if you know what I mean. Personally, I consider the interview being unpleasant for the interviewee is a given, and all I can do is minimize that, I can never eliminate it, because the stress and uncertainly is sufficient on its own to make the process unpleasant. In interviews when I ask code problems, I try to do my best to work with the interviewee. They get to pick the language, and I'm generally not looking at any style issues at all (if by "style" we mean "things that have no impact on running code"). If anything I'm a little too accommodating when I also ask them to choose their own data structures for the problems in question. This throws a surprisingly large number of people. (It's the price of allowing people to pick their implementation language; I'm not going to go write a separate test problem for each of the ten languages I might reasonably expect to get an answer in, especially when I don't even know them all. I actually feel I've sort of blundered into a good test here; a surprising number of people know "arrays" and nothing else and choose very bad representations for things of their own free will.)

jwilliams 18y ago

All the research I've seen seems to draw the conclusion that the only strong predictor of performance in a job -- is past performance. Puzzles and psychometric testing are useful, but have a lot of limitations. However, you may have a job that involves a high level of responsibility (e.g. people's lives) and you want to assess their response to stress/crisis. Again, it's not perfect -- you don't truely know how someone will respond in a unique situation -- but in those circumstances you might want all the information possible. References, by and large, are useless. Cultural fit is incredibly important, but there are very few interviewers or interview techniques that do this adequately. There is so much bias in this it's hard to assess very well. So what are the keys for me? Validating what they have on their CV. If they have experience with a particularly technology, I ask about it. I propose problems that they are likely to encounter every day in their job, and see how they go about fixing it. From what I've seen, abstract problems do not exercise this sufficiently. What is it from there? Well, if what they have been doing is a fit for the job, then that's great. If there is a gap, there is an assessment on what it will take to bridge. How a person has conducted their career is a good indicator on how achievable this is.

raganwald 18y ago
All the research I've seen seems to draw the conclusion that the only strong predictor of performance in a job -- is past performance. You are replying to a comment suggesting that there may be value in a process using multiple interviewers over a total time of eight hours. It could well be that one, some, many, or even all of the multiple people involved use your suggested process and that that it takes a total of eight hours to get a good read on past performance. So... is your comment orthogonal to mine?
- jwilliams 18y ago
  So... is your comment orthogonal to mine? Ahha - yeah - I think I started to reply and wandered off topic :) I think it was more in reply to the OP's original comment about puzzles being a waste of time. I think 8 hours is fine - if it's progressing in the right areas. I tend not to like panel interviews, but a broad range of interviewers isn't a bad idea - the best interviews I've had usually include walking the floor a bit, seeing the environment, chatting to potential teammates - although I'd worry that it would be hard to get a consensus if too many people are involved.