5 ms·

I hate it when an interviewer makes you write code on paper. It's like asking a truck driver to run around the room and make "put put" sounds.

caffeine 17y ago
Not really. It's more like asking a truck driver to read a sign at 20 feet, and to flex his hands and legs to make sure he's able to control the wheel and pedals. The idea with writing code on paper in an interview is to disqualify the blind-since-birth truck drivers who would point to a rubber duckie in a line-up and say "That one! That is a truck."
wwalker3 17y ago
I've tried it three ways: having interviewees write code on paper, on a whiteboard, and on a computer. Of the three, the computer fares worst. Many developers work in highly customized dev environments with special keyboards or monitor layouts. Using an unfamiliar setup, cold, in a stressful situation can make them sieze up or perform poorly. Paper is no good because you can't watch their problem-solving process without hovering over their shoulder, and they get no feedback from you if they're going into the weeds. The whiteboard seems to work best, because you get a good give-and-take effect between the candidate and interviewer.
- edw519 17y ago
  Agreed. When having interviewees code, I really don't care about the result. I care about the process. I have always learned much more about the candidate from the discussion about the exercise than from the actual result.
- fuelfive 17y ago
  For frogmetrics interviews, we did screen sharing sessions. You get to watch them code in their native environment, and see how they tackle the kinds of problems you'd be paying them to solve. In fact, the project we gave everybody was the next big feature we needed to implement, and the guy who won the job just brought his already-completed code along with him.
  - alain94040 17y ago
    Does it sound right to you that your next big feature can be implemented during an interview, before the candidate is even hired? I know, you didn't mean that it was all completed, but still... how much free work can you get from people before it's called abuse? If you had said "the guy who won the job just brought the best ideas with him", I'd be all happy.
    - drusenko 17y ago
      They didn't say that the code from candidates that weren't accepted was used, just the code from the one that was. I suppose you prefer that interview coding tasks be arbitrary and wasted effort rather than put to good use?
    - fuelfive 17y ago
      Some clarifications: 1. We only used code from the guy who won, and he never saw the other submissions. 2. The interview project was time boxed to 24 hours. The guy who won did it in about 9 hours. The actual feature is big, and has lots of edge cases: our devices need to be able to update themselves when new versions become available, roll back if there's an error, archive past versions, and never, ever end up in a state where they're bricked. The applicants did a small toy implementation of this functionality. The guy who won wrote good enough initial code to be used as a foundation for the rest.
- Erf 17y ago
  The setup I've found works best is to have two computers with browsers pointed to an Etherpad document, to allow pseudocode, and use problem domains that don't require the use of references. That allows face-to-face communication, and avoids the uncomfortable violation of personal space induced by over-the-shoulder glancing. Incidentally, this technique scales very effectively for phone interviews.
- _dan 17y ago
  I've found that trying to get candidates to write code on the spot is often more trouble than it's worth. It can take ages to cover any significant ground, and nerves are always going to affect people in different ways. If I find a promising-looking candidate, I send them a few really simple tasks to code up (nothing at all complicated, or particularly "academic", maybe an hour's work at the very most). I set no time limit, and even don't demand all the tasks are completed. This serves as a very effective filter for quickly discarding the completely-useless (which easily comprises 70% of the applications we get), and for the rest gives me a pretty good insight into their abilities. I've tried sending these tasks both before and after interviews, and I'm definitely in favour of sending them first and interviewing second. Apart from the obvious stupid-filtering benefits, I've found it helps immensely in keeping interviews relaxed. Having seen some code, I have a vague baseline for their abilities, so I can start with something I know they're familiar with and slowly work towards more complicated and interesting topics (without having to drop any questions that are met with a blank stare). Although getting example code after an interview is an interesting experience. My predictions about a candidate's performance based on an interview have proven, with alarming regularity, to be way off the mark.
Dilpil 17y ago
...on the other hand, if the truck driver can't complete a task like that you might begin to wonder.
- gaius 17y ago
  I think a truck driver would likely slap someone who asked them to do that, and he'd deserve it too.
dangrover 17y ago
I turn into a blathering idiot whenever someone is watching me code. Because I have to spend like 60% of my brain power worrying about what the person is thinking of each intermediate step and if I should show it or keep it to myself. But now that I started my own company, I get paid a little more than a guy of my age/qualifications should. Hopefully soon a lot more if I can keep building it up. I get paid based on the economic value of my products, not what any one individual thinks of me personally.
- mechanical_fish 17y ago
  I know this problem. The last time someone asked me to write a sort in an interview I ended up implementing bubble sort. "You know", I said, "I remember knowing how better sorts worked, once. But I'm blanking on it now, so let me just write the stupidest thing that could possibly work, and then after that I'll see if I've had time to remember anything." It didn't seem to be a big deal. Nor should it be -- the goal of your interviewer is to watch you do things like: test your code, remember to check the edge cases, manipulate pointers correctly, be able to figure out how your brain-dead bubblesort algorithm is likely to scale as N grows larger, knowing what exceptions are and understanding their proper role... It's possible that the interview is better if you're missing your higher brain functions while you're writing the demo code. You're trying to demonstrate your everyday habits, the things you do instinctively, when your memory, your cram session, and Stack Overflow all fail you. It's like asking a trumpeter to pick up their instrument and blow a few notes: according to Malcolm Gladwell, some orchestra conductors claim to be able to detect expert players with nothing more than this one test. ADDENDUM: Oh, look, Reg Braithwaite already said this, more or less: http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz.html http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz....
rstocker99 17y ago
It's actually more like asking a candidate airplane pilot to fly in a simulator. I'm personally a huge fan of interview questions that require candidates to do tasks that are as close as possible to what they'll actually need to do on the job. For devs this means coding, designing etc. For testers this means creating test cases and for a UI designer it means doing UI design. If someone made me interview an accountant I'd have them do some actual case questions for accounting. I wouldn't be able to tell if they did them particularly well but that's a whole other issue and is probably why no one has asked me to interview any accountants. Like many other folks that interview using this strategy I've come to it after seeing how many people have a great resume but are completely unable to do the actual work. This is often true even for people that pass reference checks with flying colors. After getting burned a few times you realize the benefits of getting people to demonstrate their abilities before hiring them. The counter point to getting developers to code during an interview (and presumably to accountants doing accounting problems in an interview) usually takes a form like "I can't write code in an interview setting b/c X, Y and Z but I'm a great coder under normal conditions". I'm sure this is true for some people and it is unfortunate that those high quality people miss out on jobs they are qualified for. It's also a loss for you as a hiring manager. But as a hiring manager you're better off with an interview process that mistakenly screens out some good people if it ensures fewer poor hires that you ultimately have to fire. A poor hire is very expensive in terms of time, team morale, money etc.

caffeine 17y ago

Not really. It's more like asking a truck driver to read a sign at 20 feet, and to flex his hands and legs to make sure he's able to control the wheel and pedals. The idea with writing code on paper in an interview is to disqualify the blind-since-birth truck drivers who would point to a rubber duckie in a line-up and say "That one! That is a truck."

wwalker3 17y ago

I've tried it three ways: having interviewees write code on paper, on a whiteboard, and on a computer. Of the three, the computer fares worst. Many developers work in highly customized dev environments with special keyboards or monitor layouts. Using an unfamiliar setup, cold, in a stressful situation can make them sieze up or perform poorly. Paper is no good because you can't watch their problem-solving process without hovering over their shoulder, and they get no feedback from you if they're going into the weeds. The whiteboard seems to work best, because you get a good give-and-take effect between the candidate and interviewer.

edw519 17y ago
Agreed. When having interviewees code, I really don't care about the result. I care about the process. I have always learned much more about the candidate from the discussion about the exercise than from the actual result.
fuelfive 17y ago
For frogmetrics interviews, we did screen sharing sessions. You get to watch them code in their native environment, and see how they tackle the kinds of problems you'd be paying them to solve. In fact, the project we gave everybody was the next big feature we needed to implement, and the guy who won the job just brought his already-completed code along with him.
- alain94040 17y ago
  Does it sound right to you that your next big feature can be implemented during an interview, before the candidate is even hired? I know, you didn't mean that it was all completed, but still... how much free work can you get from people before it's called abuse? If you had said "the guy who won the job just brought the best ideas with him", I'd be all happy.
  - drusenko 17y ago
    They didn't say that the code from candidates that weren't accepted was used, just the code from the one that was. I suppose you prefer that interview coding tasks be arbitrary and wasted effort rather than put to good use?
  - fuelfive 17y ago
    Some clarifications: 1. We only used code from the guy who won, and he never saw the other submissions. 2. The interview project was time boxed to 24 hours. The guy who won did it in about 9 hours. The actual feature is big, and has lots of edge cases: our devices need to be able to update themselves when new versions become available, roll back if there's an error, archive past versions, and never, ever end up in a state where they're bricked. The applicants did a small toy implementation of this functionality. The guy who won wrote good enough initial code to be used as a foundation for the rest.
Erf 17y ago
The setup I've found works best is to have two computers with browsers pointed to an Etherpad document, to allow pseudocode, and use problem domains that don't require the use of references. That allows face-to-face communication, and avoids the uncomfortable violation of personal space induced by over-the-shoulder glancing. Incidentally, this technique scales very effectively for phone interviews.
_dan 17y ago
I've found that trying to get candidates to write code on the spot is often more trouble than it's worth. It can take ages to cover any significant ground, and nerves are always going to affect people in different ways. If I find a promising-looking candidate, I send them a few really simple tasks to code up (nothing at all complicated, or particularly "academic", maybe an hour's work at the very most). I set no time limit, and even don't demand all the tasks are completed. This serves as a very effective filter for quickly discarding the completely-useless (which easily comprises 70% of the applications we get), and for the rest gives me a pretty good insight into their abilities. I've tried sending these tasks both before and after interviews, and I'm definitely in favour of sending them first and interviewing second. Apart from the obvious stupid-filtering benefits, I've found it helps immensely in keeping interviews relaxed. Having seen some code, I have a vague baseline for their abilities, so I can start with something I know they're familiar with and slowly work towards more complicated and interesting topics (without having to drop any questions that are met with a blank stare). Although getting example code after an interview is an interesting experience. My predictions about a candidate's performance based on an interview have proven, with alarming regularity, to be way off the mark.

Dilpil 17y ago

...on the other hand, if the truck driver can't complete a task like that you might begin to wonder.

gaius 17y ago
I think a truck driver would likely slap someone who asked them to do that, and he'd deserve it too.

dangrover 17y ago

I turn into a blathering idiot whenever someone is watching me code. Because I have to spend like 60% of my brain power worrying about what the person is thinking of each intermediate step and if I should show it or keep it to myself. But now that I started my own company, I get paid a little more than a guy of my age/qualifications should. Hopefully soon a lot more if I can keep building it up. I get paid based on the economic value of my products, not what any one individual thinks of me personally.

mechanical_fish 17y ago
I know this problem. The last time someone asked me to write a sort in an interview I ended up implementing bubble sort. "You know", I said, "I remember knowing how better sorts worked, once. But I'm blanking on it now, so let me just write the stupidest thing that could possibly work, and then after that I'll see if I've had time to remember anything." It didn't seem to be a big deal. Nor should it be -- the goal of your interviewer is to watch you do things like: test your code, remember to check the edge cases, manipulate pointers correctly, be able to figure out how your brain-dead bubblesort algorithm is likely to scale as N grows larger, knowing what exceptions are and understanding their proper role... It's possible that the interview is better if you're missing your higher brain functions while you're writing the demo code. You're trying to demonstrate your everyday habits, the things you do instinctively, when your memory, your cram session, and Stack Overflow all fail you. It's like asking a trumpeter to pick up their instrument and blow a few notes: according to Malcolm Gladwell, some orchestra conductors claim to be able to detect expert players with nothing more than this one test. ADDENDUM: Oh, look, Reg Braithwaite already said this, more or less: http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz.html http://weblog.raganwald.com/2007/01/dont-overthink-fizzbuzz....

rstocker99 17y ago

It's actually more like asking a candidate airplane pilot to fly in a simulator. I'm personally a huge fan of interview questions that require candidates to do tasks that are as close as possible to what they'll actually need to do on the job. For devs this means coding, designing etc. For testers this means creating test cases and for a UI designer it means doing UI design. If someone made me interview an accountant I'd have them do some actual case questions for accounting. I wouldn't be able to tell if they did them particularly well but that's a whole other issue and is probably why no one has asked me to interview any accountants. Like many other folks that interview using this strategy I've come to it after seeing how many people have a great resume but are completely unable to do the actual work. This is often true even for people that pass reference checks with flying colors. After getting burned a few times you realize the benefits of getting people to demonstrate their abilities before hiring them. The counter point to getting developers to code during an interview (and presumably to accountants doing accounting problems in an interview) usually takes a form like "I can't write code in an interview setting b/c X, Y and Z but I'm a great coder under normal conditions". I'm sure this is true for some people and it is unfortunate that those high quality people miss out on jobs they are qualified for. It's also a loss for you as a hiring manager. But as a hiring manager you're better off with an interview process that mistakenly screens out some good people if it ensures fewer poor hires that you ultimately have to fire. A poor hire is very expensive in terms of time, team morale, money etc.