3 ms·

Disagree completely. There are simply too many things that you learn in the interview that are too difficult to learn any other way. Having someone code is NOT

by edw519 11y ago

Disagree completely. There are simply too many things that you learn in the interview that are too difficult to learn any other way. Having someone code is NOT about seeing what kind of code they produce (well maybe a little); it's about getting to know as much as you can about them in as little time as possible. I want to put the candidate under some amount of stress to see how they respond. After all, responding to stress (deadlines, technical issues, people issues, etc.) is a very important part of a programmers job. Just a few of the things I usually learn with a short coding test in an interview right away: - Did he understand the problem? - How did he approach the problem? - Does he appear as if he has any idea what he’s doing? - Can he explain what he has done? - Can he defend what he has done? - Does he understand the concepts of order, cleanness, iteration, branching, recursion, etc., etc., etc... - Based on this small sample, do I think he can do the work we need done? - Do I like him? Will he fit in and be a good team member? Of course, I share all of this with the candidate. There's no secret agenda. We just want to make a fair assessment of fit. This is good for everyone. All a take-home project does is separate those who can build superior code from those who can build good code under ideal conditions. This is nice to know, but not a very important metric in the real world. Ideal conditions rarely happen on real projects. Why simulate them in the screening process?

tptacek 11y ago
The problem is that most interviewers think they can assess these things, but (a) really can't, and (b) are having their readings confounded by the nervousness and discomfort of candidates. Also: we banned "Do I like him? Will be fit in" at Matasano several years ago, and were far better off for doing so. Answer honestly: for every bullet in this list, did your team have a standardized rubric by which you could mechanically evaluate candidates? Matasano had a much smaller list of considerations for candidates, but developing working rubrics for assessing them took years. If you really have a working rubric for this ambitious list, you could be making a lot of money with it. Note also the pronoun you've chosen to use through your comment. I know it isn't deliberate, but that choice also has an impact.
- HeyLaughingBoy 11y ago
  This is an excellent point and it's one of the ways we tried to reduce the interview subjectivity at my last employer. Don't get me wrong: in the end, an interview is highly subjective, but when five people are evaluating someone, you want to have some way of calibrating their expectations so they are at least close to being on the same page. It helped to even out the process. If Tim says the candidate was a 3 out of 5 on multithreading concepts, I know what questions he could have been asked (there may be 6 to choose from on the list) and what quality of answers he gave (typical candidate responses are shown, in increasing order of "quality"). Did it help get us better developers? I don't think so. In the end we concluded that our biggest problem wasn't screening good from bad candidates -- we usually made pretty good hires. The problem was getting good candidates to apply in the first place.
Jemaclus 11y ago
I don't disagree with you, but my problem with short coding tests is that they're often very, very contrived. The kind of contrived that it is extremely likely that it will be something I have zero experience with in my day-to-day work. It's extremely difficult to find a coding challenge that hits all the points you mentioned. At my current company, we have several problems that we choose between, depending on the skill level of the candidate. But even after interviewing over a hundred candidates, I still feel like none of us get half the information we think we're getting. I don't have any answers, but I don't think your plan of attack is necessarily any better than the author's.
spinlock 11y ago
> All a take-home project does is separate those who can build superior code from those who can build good code under ideal conditions. It also separates those with no job who spent 20 hours on the project from those with jobs who put in the requested 4 hours.
- seansmccullough 11y ago
  And using the assumption that the quality of employed engineers is higher than the quality of unemployed engineers, you are lowering the quality of your perspective employment pool.

by edw519 11y ago

tptacek 11y ago

The problem is that most interviewers think they can assess these things, but (a) really can't, and (b) are having their readings confounded by the nervousness and discomfort of candidates. Also: we banned "Do I like him? Will be fit in" at Matasano several years ago, and were far better off for doing so. Answer honestly: for every bullet in this list, did your team have a standardized rubric by which you could mechanically evaluate candidates? Matasano had a much smaller list of considerations for candidates, but developing working rubrics for assessing them took years. If you really have a working rubric for this ambitious list, you could be making a lot of money with it. Note also the pronoun you've chosen to use through your comment. I know it isn't deliberate, but that choice also has an impact.

HeyLaughingBoy 11y ago
This is an excellent point and it's one of the ways we tried to reduce the interview subjectivity at my last employer. Don't get me wrong: in the end, an interview is highly subjective, but when five people are evaluating someone, you want to have some way of calibrating their expectations so they are at least close to being on the same page. It helped to even out the process. If Tim says the candidate was a 3 out of 5 on multithreading concepts, I know what questions he could have been asked (there may be 6 to choose from on the list) and what quality of answers he gave (typical candidate responses are shown, in increasing order of "quality"). Did it help get us better developers? I don't think so. In the end we concluded that our biggest problem wasn't screening good from bad candidates -- we usually made pretty good hires. The problem was getting good candidates to apply in the first place.

Jemaclus 11y ago

I don't disagree with you, but my problem with short coding tests is that they're often very, very contrived. The kind of contrived that it is extremely likely that it will be something I have zero experience with in my day-to-day work. It's extremely difficult to find a coding challenge that hits all the points you mentioned. At my current company, we have several problems that we choose between, depending on the skill level of the candidate. But even after interviewing over a hundred candidates, I still feel like none of us get half the information we think we're getting. I don't have any answers, but I don't think your plan of attack is necessarily any better than the author's.

spinlock 11y ago

> All a take-home project does is separate those who can build superior code from those who can build good code under ideal conditions. It also separates those with no job who spent 20 hours on the project from those with jobs who put in the requested 4 hours.

seansmccullough 11y ago
And using the assumption that the quality of employed engineers is higher than the quality of unemployed engineers, you are lowering the quality of your perspective employment pool.