4 ms·

Cool idea, and nice write-up, but now I want to find out how the guys hired with the help of this method fare in their jobs!

by gopher1 13y ago

wbillingsley 13y ago
This seems to be an ill thought out technique. The exercise has been designed so that all the candidate has is a false task and an awkward channel for communication. Coding at interview is essentially always a false task, as the context and realistic goals are stripped away and the problem scaled back to something that can fit into a short timeframe. Typically, enough interviewers are bad enough at setting problems that there's a "market for lemons" effect -- the candidate does not know ahead of time whether you are a good or poor interviewer. As their goal is to get the job (not to only get the job if this particular interviewer happens to be a good interviewer -- see note) that means a significant portion of their energies goes into second-guessing who they are dealing with. And stage fright -- I've watched candidates make a complete hash of programming tasks at interview, be hired nonetheless (because we saw something else in them), and turn out actually to be competent programmers too. Strip the communication back to just slowly typed text, with a thirty minute timeframe so every communication is exceptionally expensive, and now we have an almost entirely artificial puzzle. Suppose one starts by reading and understanding the code; another starts by running the tests. One decides, given the thirty minute timeframe he's going to have to batch his testing at the end because the number of smells identified sounds like it's the test measure; another decides to run the tests at every tiny edit because it sounds as if the interviewer wants him to show his test-driven credentials; another decides that interviews like to "see how the candidate thinks" and spends most of his time asking text questions of the interviewer. From this we could deduce, what? To be brutal, I wonder if frankly we might as well be deducing that the first one is a Sagittarius and this week's horoscope doesn't bode well for them. Because for all we know, the differences in behaviour are as likely to be about different guesses about the interview (it is a cut down task, so what has the interviewer decided to exclude and what have they decided to include, and does this interviewer even have any idea about that?) as about how the candidates actually go about programming on tasks in context. But perhaps that's just my scientific curmudgeonliness about the "cult of hiring" -- the irrational belief that programmers interviewing candidates have, usually without evidence, that they are "able to ascertain how a candidate thinks" in a short space of time on an artificial task. While it might be mice to hear how the candidates hired performed, at n=2, we still would not be able to make any reliable inferences. note -- While you might think the candidate would only want the job if it is a good interviewer, that is not true. First, because poor interviewers can nonetheless be good colleagues. But more importantly because the candidate loses nothing in being offered a job he/she rejects, but does lose something in not being offered a job they might accept.
- collyw 13y ago
  Agreed. Sometimes I write nice code, I come back a few months later, understand how it works and it is easy to change / add features to it. Other times I write nasty hacks, because it is required quickly, and they just need something that works. Depends on time constraints and other factors. This interview technique obviously assumes that time is plentiful, and refactoring is the most important thing about the code. Usually it is not (at least for most of the places I have worked over the last ten years). I personally find I get a good idea about someones ability by talking to them. I have a good idea about my colleagues ability, despite not having read much of their code. If they seem s to have a passion for programming, understand a variety of concepts, know what libraries are useful for what tasks, then I will have an idea that they are good. Or are they still logging onto MySQL from the command line, because anything else is too complex to set up. Print statements for debugging, because an IDE is to complex to set up with graphical debugging. Still using the same tools that we got taught in University ten years ago, because they have not bothered to keep up? Yes, I work with these people as well. If I was hiring, these ones would not get chosen.

Cool idea, and nice write-up, but now I want to find out how the guys hired with the help of this method fare in their jobs!

by gopher1 13y ago

Cool idea, and nice write-up, but now I want to find out how the guys hired with the help of this method fare in their jobs!

wbillingsley 13y ago

This seems to be an ill thought out technique. The exercise has been designed so that all the candidate has is a false task and an awkward channel for communication. Coding at interview is essentially always a false task, as the context and realistic goals are stripped away and the problem scaled back to something that can fit into a short timeframe. Typically, enough interviewers are bad enough at setting problems that there's a "market for lemons" effect -- the candidate does not know ahead of time whether you are a good or poor interviewer. As their goal is to get the job (not to only get the job if this particular interviewer happens to be a good interviewer -- see note) that means a significant portion of their energies goes into second-guessing who they are dealing with. And stage fright -- I've watched candidates make a complete hash of programming tasks at interview, be hired nonetheless (because we saw something else in them), and turn out actually to be competent programmers too. Strip the communication back to just slowly typed text, with a thirty minute timeframe so every communication is exceptionally expensive, and now we have an almost entirely artificial puzzle. Suppose one starts by reading and understanding the code; another starts by running the tests. One decides, given the thirty minute timeframe he's going to have to batch his testing at the end because the number of smells identified sounds like it's the test measure; another decides to run the tests at every tiny edit because it sounds as if the interviewer wants him to show his test-driven credentials; another decides that interviews like to "see how the candidate thinks" and spends most of his time asking text questions of the interviewer. From this we could deduce, what? To be brutal, I wonder if frankly we might as well be deducing that the first one is a Sagittarius and this week's horoscope doesn't bode well for them. Because for all we know, the differences in behaviour are as likely to be about different guesses about the interview (it is a cut down task, so what has the interviewer decided to exclude and what have they decided to include, and does this interviewer even have any idea about that?) as about how the candidates actually go about programming on tasks in context. But perhaps that's just my scientific curmudgeonliness about the "cult of hiring" -- the irrational belief that programmers interviewing candidates have, usually without evidence, that they are "able to ascertain how a candidate thinks" in a short space of time on an artificial task. While it might be mice to hear how the candidates hired performed, at n=2, we still would not be able to make any reliable inferences. note -- While you might think the candidate would only want the job if it is a good interviewer, that is not true. First, because poor interviewers can nonetheless be good colleagues. But more importantly because the candidate loses nothing in being offered a job he/she rejects, but does lose something in not being offered a job they might accept.

collyw 13y ago
Agreed. Sometimes I write nice code, I come back a few months later, understand how it works and it is easy to change / add features to it. Other times I write nasty hacks, because it is required quickly, and they just need something that works. Depends on time constraints and other factors. This interview technique obviously assumes that time is plentiful, and refactoring is the most important thing about the code. Usually it is not (at least for most of the places I have worked over the last ten years). I personally find I get a good idea about someones ability by talking to them. I have a good idea about my colleagues ability, despite not having read much of their code. If they seem s to have a passion for programming, understand a variety of concepts, know what libraries are useful for what tasks, then I will have an idea that they are good. Or are they still logging onto MySQL from the command line, because anything else is too complex to set up. Print statements for debugging, because an IDE is to complex to set up with graphical debugging. Still using the same tools that we got taught in University ten years ago, because they have not bothered to keep up? Yes, I work with these people as well. If I was hiring, these ones would not get chosen.