3 ms·

I've tried it three ways: having interviewees write code on paper, on a whiteboard, and on a computer. Of the three, the computer fares worst. Many developers

I've tried it three ways: having interviewees write code on paper, on a whiteboard, and on a computer. Of the three, the computer fares worst. Many developers work in highly customized dev environments with special keyboards or monitor layouts. Using an unfamiliar setup, cold, in a stressful situation can make them sieze up or perform poorly. Paper is no good because you can't watch their problem-solving process without hovering over their shoulder, and they get no feedback from you if they're going into the weeds. The whiteboard seems to work best, because you get a good give-and-take effect between the candidate and interviewer.

edw519 17y ago
Agreed. When having interviewees code, I really don't care about the result. I care about the process. I have always learned much more about the candidate from the discussion about the exercise than from the actual result.
fuelfive 17y ago
For frogmetrics interviews, we did screen sharing sessions. You get to watch them code in their native environment, and see how they tackle the kinds of problems you'd be paying them to solve. In fact, the project we gave everybody was the next big feature we needed to implement, and the guy who won the job just brought his already-completed code along with him.
- alain94040 17y ago
  Does it sound right to you that your next big feature can be implemented during an interview, before the candidate is even hired? I know, you didn't mean that it was all completed, but still... how much free work can you get from people before it's called abuse? If you had said "the guy who won the job just brought the best ideas with him", I'd be all happy.
  - drusenko 17y ago
    They didn't say that the code from candidates that weren't accepted was used, just the code from the one that was. I suppose you prefer that interview coding tasks be arbitrary and wasted effort rather than put to good use?
  - fuelfive 17y ago
    Some clarifications: 1. We only used code from the guy who won, and he never saw the other submissions. 2. The interview project was time boxed to 24 hours. The guy who won did it in about 9 hours. The actual feature is big, and has lots of edge cases: our devices need to be able to update themselves when new versions become available, roll back if there's an error, archive past versions, and never, ever end up in a state where they're bricked. The applicants did a small toy implementation of this functionality. The guy who won wrote good enough initial code to be used as a foundation for the rest.
Erf 17y ago
The setup I've found works best is to have two computers with browsers pointed to an Etherpad document, to allow pseudocode, and use problem domains that don't require the use of references. That allows face-to-face communication, and avoids the uncomfortable violation of personal space induced by over-the-shoulder glancing. Incidentally, this technique scales very effectively for phone interviews.
_dan 17y ago
I've found that trying to get candidates to write code on the spot is often more trouble than it's worth. It can take ages to cover any significant ground, and nerves are always going to affect people in different ways. If I find a promising-looking candidate, I send them a few really simple tasks to code up (nothing at all complicated, or particularly "academic", maybe an hour's work at the very most). I set no time limit, and even don't demand all the tasks are completed. This serves as a very effective filter for quickly discarding the completely-useless (which easily comprises 70% of the applications we get), and for the rest gives me a pretty good insight into their abilities. I've tried sending these tasks both before and after interviews, and I'm definitely in favour of sending them first and interviewing second. Apart from the obvious stupid-filtering benefits, I've found it helps immensely in keeping interviews relaxed. Having seen some code, I have a vague baseline for their abilities, so I can start with something I know they're familiar with and slowly work towards more complicated and interesting topics (without having to drop any questions that are met with a blank stare). Although getting example code after an interview is an interesting experience. My predictions about a candidate's performance based on an interview have proven, with alarming regularity, to be way off the mark.

edw519 17y ago

Agreed. When having interviewees code, I really don't care about the result. I care about the process. I have always learned much more about the candidate from the discussion about the exercise than from the actual result.

fuelfive 17y ago

For frogmetrics interviews, we did screen sharing sessions. You get to watch them code in their native environment, and see how they tackle the kinds of problems you'd be paying them to solve. In fact, the project we gave everybody was the next big feature we needed to implement, and the guy who won the job just brought his already-completed code along with him.

alain94040 17y ago
Does it sound right to you that your next big feature can be implemented during an interview, before the candidate is even hired? I know, you didn't mean that it was all completed, but still... how much free work can you get from people before it's called abuse? If you had said "the guy who won the job just brought the best ideas with him", I'd be all happy.
- drusenko 17y ago
  They didn't say that the code from candidates that weren't accepted was used, just the code from the one that was. I suppose you prefer that interview coding tasks be arbitrary and wasted effort rather than put to good use?
- fuelfive 17y ago
  Some clarifications: 1. We only used code from the guy who won, and he never saw the other submissions. 2. The interview project was time boxed to 24 hours. The guy who won did it in about 9 hours. The actual feature is big, and has lots of edge cases: our devices need to be able to update themselves when new versions become available, roll back if there's an error, archive past versions, and never, ever end up in a state where they're bricked. The applicants did a small toy implementation of this functionality. The guy who won wrote good enough initial code to be used as a foundation for the rest.

Erf 17y ago

The setup I've found works best is to have two computers with browsers pointed to an Etherpad document, to allow pseudocode, and use problem domains that don't require the use of references. That allows face-to-face communication, and avoids the uncomfortable violation of personal space induced by over-the-shoulder glancing. Incidentally, this technique scales very effectively for phone interviews.

_dan 17y ago

I've found that trying to get candidates to write code on the spot is often more trouble than it's worth. It can take ages to cover any significant ground, and nerves are always going to affect people in different ways. If I find a promising-looking candidate, I send them a few really simple tasks to code up (nothing at all complicated, or particularly "academic", maybe an hour's work at the very most). I set no time limit, and even don't demand all the tasks are completed. This serves as a very effective filter for quickly discarding the completely-useless (which easily comprises 70% of the applications we get), and for the rest gives me a pretty good insight into their abilities. I've tried sending these tasks both before and after interviews, and I'm definitely in favour of sending them first and interviewing second. Apart from the obvious stupid-filtering benefits, I've found it helps immensely in keeping interviews relaxed. Having seen some code, I have a vague baseline for their abilities, so I can start with something I know they're familiar with and slowly work towards more complicated and interesting topics (without having to drop any questions that are met with a blank stare). Although getting example code after an interview is an interesting experience. My predictions about a candidate's performance based on an interview have proven, with alarming regularity, to be way off the mark.