4 ms·

> In my experience, relevant, realistic coding challenges are by far the best indicator of ability. They should take roughly 2 hours, to be done in the candidat

> In my experience, relevant, realistic coding challenges are by far the best indicator of ability. They should take roughly 2 hours, to be done in the candidate’s own time That's a great idea for the company, but from the perspective of the candidate, what a gamble of their time. They give away two hours of coding for free for a job they may not get? If you're going to demand that the candidate do some coding for you, at the very least give them a token amount of money in exchange for their time. Yes, when you're hiring a lot of people, the cost can add up, but you shouldn't be demanding such a large slice of someone's time without being prepared to compensate them fairly.

Peroni 13y ago
...you shouldn't be demanding such a large slice of someone's time without being prepared to compensate them fairly. The compensation is successfully landing the job. If you aren't willing to commit two hours of your time to landing said job then clearly you don't want the job.
- mathattack 13y ago
  I'm curious how much this narrows down the field, and if the right folks drop out. That will help decide if you're on the right track. I assume that you're filling your needs, so it does work. Do half the people decline the coding assignment? A quarter? How many people pass the assignment? My instinct is it's no different than interviewing for 2 hours. If it's for a coding job, it's probably more efficient for everyone involved.
  - Peroni 13y ago
    I've genuinely never had a single person refuse to take part in our technical test or request compensation in return either. If someone refused or demanded compensation, I would no longer pursue said candidate.
    - mathattack 13y ago
      I think that answers whether or not it's worth doing. Thanks for sharing your insights into the process. Hiring is a very hard thing to do right. (Maybe impossible to do right, only possible to do less wrong)
  - tomblomfield 13y ago
    We've never had anyone explicitly decline. Maybe 5 or 10% drop off the radar at this stage - perhaps they didn't feel able to complete it successfully? If it were feasible, I'd actually prefer to pay candidates consulting-rates to do a (useful) 4-5 day project as a final interview. We've actually done this 3 or 4 times - but only when the candidate is thinking about moving his family from abroad to come work for us. We feel like there's an increased obligation on us to make sure we're sure in these cases. Normally, though, it's just not practical. People are normally in jobs already, and asking them to take 4 or 5 days of holiday isn't fair. Especially if they were in the final stages of applying to 3 or 4 different companies. That's your entire year's holiday allowance down the drain.
    - mathattack 13y ago
      Thank you for sharing. Hiring is very tough, and I appreciate the war stories. At what point does a 4-5 day project morph into a 10 week internship or contract project? (Even after 10 weeks it can be tough to decide on an offer!)
geetee 13y ago
I'd gladly give up 2 hours of my time over a 4 hour interview with a whiteboarding exercise that I inevitably fuck up.
bonemachine 13y ago
This may not have occurred to you, but it's an investment of time both ways. The company most likely spent >2 hours of staff time filtering resumes, & handling communications before issuing the code challenge. If a fizzbuzz-order phone screen was done before the longer coding challenge, for multiple candidates, then most likely >> 2 hours. What sucks though is when the hiring manager pretends (as managers often do) that the challenge will take "only 2 hours", but in reality -- to get it production-grade, well-document, and (as happens quite often) provide fixes or workarounds to bugs in the hastily drawn up spec -- it ends up taking 5-10 hours. And as a reward, you get sent back to the internal recruiter, with their finite attention span and short-term memory capacity. And the usual random delays, inattentiveness, and cognitively detached communication loops ensue, for week after week. Fun for everyone!
- walshemj 13y ago
  Tough - running a startup is not all kittens and unicorns there is this thing called hard work - this attitude is just like the car dealers in Texas who dont like Tesla selling direct.
  - bonemachine 13y ago
    "If you don't like it, tough" can work as an attitude, sort of, if you're a car dealership battling with another car dealership over some piece of marketing territory in Texas. But if you're a startup with an iffy business plan trying to convince sentient human beings with high-end to spend their salad years sweating on your vision, in as competitive a labor market as what we have now... not so much.
    - walshemj 13y ago
      That was my point that some start ups seem to think people should walk over burning coals to work for them - to much watching of sirallen of the "apprentice"/dragons den type of programs. Its a market if you load costs (time or money) on to potential employees they are going to mark you down.
solve 13y ago
I've done a few take-home interview coding challenges -- absolutely would NEVER do them again. Both that I did ended up just getting tossed in the trash. I was able to get in contact with my interviewers later, not one of them had even seen my take-home work. If you're a company using this tactic, hopefully you're aiming for only the most desperate or green candidates, because that's exactly what you're going to get.
- coreymaass 13y ago
  I've done a few as well. It left a bad taste in my mouth every time, too. It never showed what I could do, and I felt it was wasting everyone's time - the company's and mine. Now I usually make a counter offer. I ask them to find a small project they can hire me for. It's a better use of everyone's time.
  - tomblomfield 13y ago
    We spend a long time (30+ mins) reviewing every single coding challenge we receive. We also use it as the basis of an hour-long interview - probing on the choice of libraries, asking extension questions, etc. But it's important that you weed out no-hopers at the CV- and phone-screen stage, otherwise you're going to get hundreds of coding challenges and you won't be able to give them due attention.
mseebach 13y ago
Well, what isn't a gamble of the candidates time? I'm assuming that the two hour test is done from home, and depending on travel time, that's a lot smaller investment of time than travelling in for an on-site interview, plus it's likely to give significantly better signal than a two-hour on-site session early in the process. Yet another way to look at it is as leveling the paying field. A two hour on-site costs the company at least two hours engineering time, so you have to sort more aggressively in incoming applications - this puts those with "high value brands" (top universities and well known employers/internships) at an advantage over a maybe equally qualified candidate with an off-the-beaten-path CV. A complete-from-home test is much, much quicker and comfortable to "grade" and can thus be given out much more broadly.

Peroni 13y ago

...you shouldn't be demanding such a large slice of someone's time without being prepared to compensate them fairly. The compensation is successfully landing the job. If you aren't willing to commit two hours of your time to landing said job then clearly you don't want the job.

mathattack 13y ago
I'm curious how much this narrows down the field, and if the right folks drop out. That will help decide if you're on the right track. I assume that you're filling your needs, so it does work. Do half the people decline the coding assignment? A quarter? How many people pass the assignment? My instinct is it's no different than interviewing for 2 hours. If it's for a coding job, it's probably more efficient for everyone involved.
- Peroni 13y ago
  I've genuinely never had a single person refuse to take part in our technical test or request compensation in return either. If someone refused or demanded compensation, I would no longer pursue said candidate.
  - mathattack 13y ago
    I think that answers whether or not it's worth doing. Thanks for sharing your insights into the process. Hiring is a very hard thing to do right. (Maybe impossible to do right, only possible to do less wrong)
- tomblomfield 13y ago
  We've never had anyone explicitly decline. Maybe 5 or 10% drop off the radar at this stage - perhaps they didn't feel able to complete it successfully? If it were feasible, I'd actually prefer to pay candidates consulting-rates to do a (useful) 4-5 day project as a final interview. We've actually done this 3 or 4 times - but only when the candidate is thinking about moving his family from abroad to come work for us. We feel like there's an increased obligation on us to make sure we're sure in these cases. Normally, though, it's just not practical. People are normally in jobs already, and asking them to take 4 or 5 days of holiday isn't fair. Especially if they were in the final stages of applying to 3 or 4 different companies. That's your entire year's holiday allowance down the drain.
  - mathattack 13y ago
    Thank you for sharing. Hiring is very tough, and I appreciate the war stories. At what point does a 4-5 day project morph into a 10 week internship or contract project? (Even after 10 weeks it can be tough to decide on an offer!)

geetee 13y ago

I'd gladly give up 2 hours of my time over a 4 hour interview with a whiteboarding exercise that I inevitably fuck up.

bonemachine 13y ago

This may not have occurred to you, but it's an investment of time both ways. The company most likely spent >2 hours of staff time filtering resumes, & handling communications before issuing the code challenge. If a fizzbuzz-order phone screen was done before the longer coding challenge, for multiple candidates, then most likely >> 2 hours. What sucks though is when the hiring manager pretends (as managers often do) that the challenge will take "only 2 hours", but in reality -- to get it production-grade, well-document, and (as happens quite often) provide fixes or workarounds to bugs in the hastily drawn up spec -- it ends up taking 5-10 hours. And as a reward, you get sent back to the internal recruiter, with their finite attention span and short-term memory capacity. And the usual random delays, inattentiveness, and cognitively detached communication loops ensue, for week after week. Fun for everyone!

walshemj 13y ago
Tough - running a startup is not all kittens and unicorns there is this thing called hard work - this attitude is just like the car dealers in Texas who dont like Tesla selling direct.
- bonemachine 13y ago
  "If you don't like it, tough" can work as an attitude, sort of, if you're a car dealership battling with another car dealership over some piece of marketing territory in Texas. But if you're a startup with an iffy business plan trying to convince sentient human beings with high-end to spend their salad years sweating on your vision, in as competitive a labor market as what we have now... not so much.
  - walshemj 13y ago
    That was my point that some start ups seem to think people should walk over burning coals to work for them - to much watching of sirallen of the "apprentice"/dragons den type of programs. Its a market if you load costs (time or money) on to potential employees they are going to mark you down.

solve 13y ago

I've done a few take-home interview coding challenges -- absolutely would NEVER do them again. Both that I did ended up just getting tossed in the trash. I was able to get in contact with my interviewers later, not one of them had even seen my take-home work. If you're a company using this tactic, hopefully you're aiming for only the most desperate or green candidates, because that's exactly what you're going to get.

coreymaass 13y ago
I've done a few as well. It left a bad taste in my mouth every time, too. It never showed what I could do, and I felt it was wasting everyone's time - the company's and mine. Now I usually make a counter offer. I ask them to find a small project they can hire me for. It's a better use of everyone's time.
- tomblomfield 13y ago
  We spend a long time (30+ mins) reviewing every single coding challenge we receive. We also use it as the basis of an hour-long interview - probing on the choice of libraries, asking extension questions, etc. But it's important that you weed out no-hopers at the CV- and phone-screen stage, otherwise you're going to get hundreds of coding challenges and you won't be able to give them due attention.

mseebach 13y ago

Well, what isn't a gamble of the candidates time? I'm assuming that the two hour test is done from home, and depending on travel time, that's a lot smaller investment of time than travelling in for an on-site interview, plus it's likely to give significantly better signal than a two-hour on-site session early in the process. Yet another way to look at it is as leveling the paying field. A two hour on-site costs the company at least two hours engineering time, so you have to sort more aggressively in incoming applications - this puts those with "high value brands" (top universities and well known employers/internships) at an advantage over a maybe equally qualified candidate with an off-the-beaten-path CV. A complete-from-home test is much, much quicker and comfortable to "grade" and can thus be given out much more broadly.