8 ms·

I don’t like the timeboxing aspect. It’s putting pressure on the candidate. I don’t know why the website says there is less pressure when it’s time limited. Do

I don’t like the timeboxing aspect. It’s putting pressure on the candidate. I don’t know why the website says there is less pressure when it’s time limited. Do you have a study or other evidence to support this claim? For code exercises I set when recruiting I send a pdf out and then expect a zip file or link to GitHub or bitbucket with the solution. I might give an advisory along the lines of this shouldn’t take more than half a day. But if the candidate screws up and starts again that’s ok. Mistakes and wrong turns are taken all the time in the real world. It works very well. Simple. No complications. And free.

davidbanham 9y ago
Hey, thanks for the feedback. The idea is that the candidate knows they can do the best they can within the time limit, and then it's done. No agonizing over whether it's good enough or they should spend another few hours. No imagining how many hours and days other candidates for the same role might be putting in, wondering if they're going to stack up. I've used this approach with around 100 candidates hiring for various roles in my own business and for clients. The response I've gotten from candidates has been overwhelmingly positive. The problem I have with the traditional "don't spend more than about X time" approach is that it's not a level playing field. The responses to a challenge exist in relative terms, not absolute. It doesn't matter whether your response was great or not. It matters whether your response was in the top 5 of all the other candidates responding. It follows that you probably need to spend at least as much time, if not more, than all the other applicants for the role. How much time the other candidates _might_ be spending is basically unbounded, leading candidates to spend vastly too much time working on a challenge and then lying about it. If the candidate screws up a timed challenge that's also totally okay. It'll be a great conversation to have in the follow-up interview. "I started down this road, but if I was doing it again I'd do X, Y and Z because of A, B and C." Incidentally, I'd suggest that half a day is way too long. Especially if that half a day isn't actually enforced. When I'm hiring I usually set challenges of about 40 minutes, an hour and a half max. I'm not looking for a product that's ready to ship to production. I'm after a sense of whether the candidate can actually interpret a problem then string some code together, and something meaningful we can discuss in the next interview.
- carlmr 9y ago
  >Incidentally, I'd suggest that half a day is way too long. Especially if that half a day isn't actually enforced. When I'm hiring I usually set challenges of about 40 minutes, an hour and a half max. I'm not looking for a product that's ready to ship to production. I'm after a sense of whether the candidate can actually interpret a problem then string some code together, and something meaningful we can discuss in the next interview. While I agree with what you said about leveling the playing field, I find 40 minutes sound awfully short. In university I noticed I was doing much better on 3h exams than on 1.5 hour exams, often only taking 1.5h for the 3h exam, even though it had more material. I couldn't relax enough in the short exam to do my best, even though the time was long enough for a twice as long exam.
  - Xcelerate 9y ago
    > In university I noticed I was doing much better on 3h exams than on 1.5 hour exams Same here. I was the person who would do poorly on the 15 min physics quizzes we had in college just to get the highest grade in the class on the final, only because I had more time (half the answers on my quizzes would be blank just because I didn’t get to them). In high school, I would barely squeak by on the multiple choice AMC to then do fantastic on the 3 hour AIME. But I clearly knew the material, and I knew it very well, so are you going to penalize me for being slightly slower on a test? As an employee, I complete my real world tasks just as fast as everyone else, so why should a time constraint on a high pressure exam relate to the real world productivity and profitability of an employee? I would argue that it doesn’t—at all.
- kabouseng 9y ago
  But then again, a candidate that puts in way more effort is also a strong signal for future performance. I suppose it depends on the employer, does he prefer candidates that put in more effort but might be less talented, or very talented but might not have much resolve (unproven at least).
  - reificator 9y ago
    If I'm given a time limit on an assignment, I'm going to take the time allotted and it will reflect in my (admittedly easily faked) git history. If the employer is being honest, then we're all good. But if they're going to go with the candidate that puts in 5x the time allotted, then I clearly don't want to work for them either. It's an obvious red flag IMO. If they ask for n hours in the interview but expect more than that, then they're likely to do the same thing when you're on the job. Don't be surprised if a few months down the line you're working 12 hour days 6 days a week or you're an underperformer and the first to get cut.
joelhaasnoot 9y ago
In the past when doing timeboxed, we gave the candidate a weekend. The task was too much work for a weekend, but that's part of the job. Choosing to prioritize and what to prioritize is also an important skill. In our case: if you were going for the perfect code abstraction but didn't deliver any business value in the form of a working product, that says something about the candidate and their performance in a team.
- detaro 9y ago
  I hope you communicated that clearly beforehand?
  - mathgeek 9y ago
    This! I've done more than one code challenge in my time that was secretly testing for how far you get on a task that is underbudgeted on time. Your best applicants are going to spend too much of their own (unpaid) time finishing it and be annoyed at you before you tell them how clever you are for testing their ability to prioritize...or they're just going to not bother.
    - watwut 9y ago
      I would assume this is how the company will treat me after being hire.
      - scaryclam 9y ago
        Far too many companies forget that they are also being tested and interviewed. The sort of stupid "let's secretly test whether the candidate can figure out we're screwing with them" kind of thing lets good candidates know one thing: the company is toxic and won't treat them like a human. This is the exact sort of behavior to run away from.
  - jpmoyn 9y ago
    Yeah because most peoples biggest priority going into in a technical interview process is focusing on good code. If they want a candidate to "MVP" their coding challenge for business value reasons, they should make that very clear. It's not like it takes MORE skill to write hacky code.
mwcampbell 9y ago
I think your approach is reasonable. But I'm curious, why provide the challenge in PDF rather than HTML? Unless your PDF generator can create a tagged PDF, that's going to be less accessible for blind people to blind people than HTML, .docx or .odt, or even plain text. I don't mean to guilt-trip here; I figure you weren't aware of that problem. Is there a reason why PDF is better? For example, do your challenges tend to include graphics or complex math notation?

davidbanham 9y ago

Hey, thanks for the feedback. The idea is that the candidate knows they can do the best they can within the time limit, and then it's done. No agonizing over whether it's good enough or they should spend another few hours. No imagining how many hours and days other candidates for the same role might be putting in, wondering if they're going to stack up. I've used this approach with around 100 candidates hiring for various roles in my own business and for clients. The response I've gotten from candidates has been overwhelmingly positive. The problem I have with the traditional "don't spend more than about X time" approach is that it's not a level playing field. The responses to a challenge exist in relative terms, not absolute. It doesn't matter whether your response was great or not. It matters whether your response was in the top 5 of all the other candidates responding. It follows that you probably need to spend at least as much time, if not more, than all the other applicants for the role. How much time the other candidates _might_ be spending is basically unbounded, leading candidates to spend vastly too much time working on a challenge and then lying about it. If the candidate screws up a timed challenge that's also totally okay. It'll be a great conversation to have in the follow-up interview. "I started down this road, but if I was doing it again I'd do X, Y and Z because of A, B and C." Incidentally, I'd suggest that half a day is way too long. Especially if that half a day isn't actually enforced. When I'm hiring I usually set challenges of about 40 minutes, an hour and a half max. I'm not looking for a product that's ready to ship to production. I'm after a sense of whether the candidate can actually interpret a problem then string some code together, and something meaningful we can discuss in the next interview.

carlmr 9y ago
>Incidentally, I'd suggest that half a day is way too long. Especially if that half a day isn't actually enforced. When I'm hiring I usually set challenges of about 40 minutes, an hour and a half max. I'm not looking for a product that's ready to ship to production. I'm after a sense of whether the candidate can actually interpret a problem then string some code together, and something meaningful we can discuss in the next interview. While I agree with what you said about leveling the playing field, I find 40 minutes sound awfully short. In university I noticed I was doing much better on 3h exams than on 1.5 hour exams, often only taking 1.5h for the 3h exam, even though it had more material. I couldn't relax enough in the short exam to do my best, even though the time was long enough for a twice as long exam.
- Xcelerate 9y ago
  > In university I noticed I was doing much better on 3h exams than on 1.5 hour exams Same here. I was the person who would do poorly on the 15 min physics quizzes we had in college just to get the highest grade in the class on the final, only because I had more time (half the answers on my quizzes would be blank just because I didn’t get to them). In high school, I would barely squeak by on the multiple choice AMC to then do fantastic on the 3 hour AIME. But I clearly knew the material, and I knew it very well, so are you going to penalize me for being slightly slower on a test? As an employee, I complete my real world tasks just as fast as everyone else, so why should a time constraint on a high pressure exam relate to the real world productivity and profitability of an employee? I would argue that it doesn’t—at all.
kabouseng 9y ago
But then again, a candidate that puts in way more effort is also a strong signal for future performance. I suppose it depends on the employer, does he prefer candidates that put in more effort but might be less talented, or very talented but might not have much resolve (unproven at least).
- reificator 9y ago
  If I'm given a time limit on an assignment, I'm going to take the time allotted and it will reflect in my (admittedly easily faked) git history. If the employer is being honest, then we're all good. But if they're going to go with the candidate that puts in 5x the time allotted, then I clearly don't want to work for them either. It's an obvious red flag IMO. If they ask for n hours in the interview but expect more than that, then they're likely to do the same thing when you're on the job. Don't be surprised if a few months down the line you're working 12 hour days 6 days a week or you're an underperformer and the first to get cut.

joelhaasnoot 9y ago

In the past when doing timeboxed, we gave the candidate a weekend. The task was too much work for a weekend, but that's part of the job. Choosing to prioritize and what to prioritize is also an important skill. In our case: if you were going for the perfect code abstraction but didn't deliver any business value in the form of a working product, that says something about the candidate and their performance in a team.

detaro 9y ago
I hope you communicated that clearly beforehand?
- mathgeek 9y ago
  This! I've done more than one code challenge in my time that was secretly testing for how far you get on a task that is underbudgeted on time. Your best applicants are going to spend too much of their own (unpaid) time finishing it and be annoyed at you before you tell them how clever you are for testing their ability to prioritize...or they're just going to not bother.
  - watwut 9y ago
    I would assume this is how the company will treat me after being hire.
    - scaryclam 9y ago
      Far too many companies forget that they are also being tested and interviewed. The sort of stupid "let's secretly test whether the candidate can figure out we're screwing with them" kind of thing lets good candidates know one thing: the company is toxic and won't treat them like a human. This is the exact sort of behavior to run away from.
- jpmoyn 9y ago
  Yeah because most peoples biggest priority going into in a technical interview process is focusing on good code. If they want a candidate to "MVP" their coding challenge for business value reasons, they should make that very clear. It's not like it takes MORE skill to write hacky code.

mwcampbell 9y ago

I think your approach is reasonable. But I'm curious, why provide the challenge in PDF rather than HTML? Unless your PDF generator can create a tagged PDF, that's going to be less accessible for blind people to blind people than HTML, .docx or .odt, or even plain text. I don't mean to guilt-trip here; I figure you weren't aware of that problem. Is there a reason why PDF is better? For example, do your challenges tend to include graphics or complex math notation?