4 ms·

> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable

> I can say "you shouldn't spend more than two hours on this" until I'm blue in the face, but candidates think they need to polish far beyond what is reasonable > I'd prefer to design a better in person interview than take the easy route that is disrespectful of candidates' time. Why don't you just let candidates decide for themselves how much time they want to spend on the task? You can set the expectations, saying that you expect that the task of a given complexity will to take between 1 and 2 hours; but candidates should be free to spend as little or as much time on a task as they like. Some candidates will value their time and spend no more than is advised; others will be sufficiently obsessed by the task to spend more time. Why should the latter group feel guilty about exceeding the recommended time limit?

jnwatson 6y ago
You're trying to evaluate a candidate's ability to accomplish something with finite resources. If you allow someone to put additional time into it, you're biasing the results towards the more desperate and less in-demand individual.
- azangru 6y ago
  - If the more desperate and less in-demand individual writes bad code, or makes poor judgements, you will be able to see it in their submission, won't you? - Whereas if such an individual, because of their desperation or lack of demand, has enough time and motivation to elegantly solve the problem, then what's not to like? I think a greater concern would be that with take-home assignments it's impossible to be sure you are actually evaluating the right individual. How can you be sure they didn't receive help from someone else?
  - jnwatson 6y ago
    The programming test is just one aspect of a candidate, in my estimate about a third of the total "score". Take-home exams bias towards evaluating programming ability but bias against other desirable candidate qualities such as experience. Their desperation to get the job doesn't necessarily carry over to their day-to-day programming. If they take a month to do a week-long task, I would want to know. In terms of cheating, that can be easily resolved by asking detailed questions about the candidate's submission. It is also helpful to show that you took the time to review it. (There's nothing as frustrating as taking a day for a take home test and getting no response).
    - azangru 6y ago
      > but bias against other desirable candidate qualities such as experience Won't you see, in the code submission, at least some evidence of experience? The way the candidate organises their code; the overall style; the third-party libraries they choose to use (or not to use); the tests they write (or don't write) — doesn't it all, combined, speak of experience or of lack thereof? There are still other stages of the interview, where you can assess candidate's experience more directly, by asking about what they worked on. I agree that it's useful to know when a task that should have taken x long takes 5x or 10x; but it may be because the candidate set a higher bar for themselves.

jnwatson 6y ago

You're trying to evaluate a candidate's ability to accomplish something with finite resources. If you allow someone to put additional time into it, you're biasing the results towards the more desperate and less in-demand individual.

azangru 6y ago
- If the more desperate and less in-demand individual writes bad code, or makes poor judgements, you will be able to see it in their submission, won't you? - Whereas if such an individual, because of their desperation or lack of demand, has enough time and motivation to elegantly solve the problem, then what's not to like? I think a greater concern would be that with take-home assignments it's impossible to be sure you are actually evaluating the right individual. How can you be sure they didn't receive help from someone else?
- jnwatson 6y ago
  The programming test is just one aspect of a candidate, in my estimate about a third of the total "score". Take-home exams bias towards evaluating programming ability but bias against other desirable candidate qualities such as experience. Their desperation to get the job doesn't necessarily carry over to their day-to-day programming. If they take a month to do a week-long task, I would want to know. In terms of cheating, that can be easily resolved by asking detailed questions about the candidate's submission. It is also helpful to show that you took the time to review it. (There's nothing as frustrating as taking a day for a take home test and getting no response).
  - azangru 6y ago
    > but bias against other desirable candidate qualities such as experience Won't you see, in the code submission, at least some evidence of experience? The way the candidate organises their code; the overall style; the third-party libraries they choose to use (or not to use); the tests they write (or don't write) — doesn't it all, combined, speak of experience or of lack thereof? There are still other stages of the interview, where you can assess candidate's experience more directly, by asking about what they worked on. I agree that it's useful to know when a task that should have taken x long takes 5x or 10x; but it may be because the candidate set a higher bar for themselves.