3 ms·

Many people here will disagree with what I'm about to say, but if you're going to give a coding challenge... I like hackerrank/codility style coding challenges

Many people here will disagree with what I'm about to say, but if you're going to give a coding challenge... I like hackerrank/codility style coding challenges. gasp They're timed. They're run against a standardized test suite. And they have support for many languages. It's really unfortunate the number of companies that give custom take home coding challenges that run the gamut. The problems with which are manifold: - Inconsistent technology and language choices by candidates - Time spent varies wildly and is easily underestimated by companies - and 4-5 hours for an unpaid coding challenge in an early round with a company is a significant investment - Good candidates will de-prioritize your company against others that move faster I know that I only answered a small subset of the question, but it's very difficult to lump all developers together and come up with a single, wholistic, and ideal interview process. From the nature of the role (IC/Lead/Manager) to the technologies/specialty to the company's mission, the process will need to be customized to some degree. However, do not think that your custom coding challenge will somehow be the perfect key to assessing developers.

shawn 8y ago
Timed challenges rule out skilled hackers that do challenges to learn new things. (They do exist.) It's often the funnest part of looking for a job. From a comment below: I was given a week to do it, but it only took me an hour or so to meet all of their explicit requirements. I liked that there was no time pressure in that regard. Ditto.
azangru 8y ago
When we were looking for a developer for our team, we created a set of small coding challenges testing the candidate's knowledge of the most relevant aspects of the language. We wrote automated tests for these challenges, and the main bulk of our interview was having the candidate solve them. The advantages to that approach, as I see it, are: - the developer is tested for knowledge that is most relevant for their current job (as opposed to writing general-purpose algorithms) - the developer is showing their familiarity with a testing environment - writing working code in a code editor seems to me more practical than writing (pseudo)code on paper or on a whiteboard There are, of course, disadvantages to this approach as well. The developer may have hard time working on an unfamiliar machine and not in their favorite code editor (though what we had was quite standard). This might be a tad stressful (although I don't know how this compares with solving a general coding problem on a whiteboard, or with a task asking you to think not just of a way to solve a problem, but of a performant way too). And it made us focus on the candidate's coding skills too much (which some hiring managers might find objectionable).
- mcv 8y ago
  The advantage of using something like codility, or making it a take-home thing, is that they can do it in their own time, free from the stressful environment of an interview.
  - umvi 8y ago
    The disadvantage of that is that the interviewer no longer knows if the candidate actually solved the problem or if he got his CS roommate to solve it for him.
    - toomanyrichies 8y ago
      Isn't that one of the reasons to ask follow-up questions like "why did you decide to use data structure X here instead of Y"? If the candidate didn't write the code, their lack of coherent answers should make that obvious pretty quickly.
      - mcv 8y ago
        Exactly. That's why you let them show their code to the other devs who get to ask questions. I don't believe someone who can't code can explain someone else's code in a convincing way.
  - pytester 8y ago
    I hate doing take home projects. * It's way too easy to set a project that takes too long. I was given one the other day that would have taken about two to three full days. That simply doesn't happen if you do the test sitting in front of them - the time spent (and potentially wasted) is always capped because the test setter doesn't want to waste too much of their own time. * Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you. * Take home projects take employer skin out of the game. It's way too easy to give out twenty tests and ignore ten of them. I was actually half way through a homework test the other day when I was told that surprise they had an open offer and the candidate had just taken the job. I doubt they'd have wasted their own time testing me IRL knowing what they knew. From the employer's perspective: * In the last case I also realized after googling that the answers were online and I could just have copied someone else's. * You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test. I suspect this is partly why the take home tests I've been given are generally of a lower quality than the in person. * It's valuable to talk through the solution with the candidate while they do it (describing trade offs they made, pitfalls they're cognizant may crop up, potential downsides to their approach).
mcv 8y ago
I recently did some codility tests, and they were totally fine. I'm also fine with take-home challenges, though. But you're absolutely right that they're often vastly underestimated, and you shouldn't give someone a take-home assignment when there's only a small change they'll get hired. It should only be done once you're sure you want to hire this candidate if they're as good as they seem to be. A take-home assignment is too big to sift the chaff from the wheat in a large group of candidates. But once you're sure you'll hire them if they do well on it, a take-home test is totally fine. Though such a take-home programming assignment should also involve presenting the result to a group of developers to explain their choices. If you're not willing to commit to that, you can't expect them to commit to putting in the time to do the assignment.
ummonk 8y ago
Yeah, those challenges are explicitly timeboxed, and I know there is a relatively objective grading criteria (which is often as simple as “passes enough of the hidden test cases”). I refuse to do other types of take home challenges, but am fine with these.
- spyspy 8y ago
  The hidden test cases can be super frustrating, especially when you're passing 7/10 and the timer keeps ticking down. It's impossible to debug besides just staring at the code and guessing what else they might be looking for. Then to ultimately have the company immediately go, "Sorry you didn't check enough green boxes. You've failed. Goodbye."
  - kamalkishor1991 8y ago
    Although sometimes this is true, most of the time you can't pass all the test because you have written a messed up code that you yourself can't understand. Learning to write the simplest code that works first time one of the most critical skills in programming and in real life also there are no test cases written for you.
andrei_says_ 8y ago
Timing a code challenge is a bit like chess boxing. Increased cortisol levels diminish or shut down the reasoning function of the brain, so expecting someone to code fast under the threat to lose a potential employnentbopportunity is not exactly productive.

shawn 8y ago

Timed challenges rule out skilled hackers that do challenges to learn new things. (They do exist.) It's often the funnest part of looking for a job. From a comment below: I was given a week to do it, but it only took me an hour or so to meet all of their explicit requirements. I liked that there was no time pressure in that regard. Ditto.

azangru 8y ago

When we were looking for a developer for our team, we created a set of small coding challenges testing the candidate's knowledge of the most relevant aspects of the language. We wrote automated tests for these challenges, and the main bulk of our interview was having the candidate solve them. The advantages to that approach, as I see it, are: - the developer is tested for knowledge that is most relevant for their current job (as opposed to writing general-purpose algorithms) - the developer is showing their familiarity with a testing environment - writing working code in a code editor seems to me more practical than writing (pseudo)code on paper or on a whiteboard There are, of course, disadvantages to this approach as well. The developer may have hard time working on an unfamiliar machine and not in their favorite code editor (though what we had was quite standard). This might be a tad stressful (although I don't know how this compares with solving a general coding problem on a whiteboard, or with a task asking you to think not just of a way to solve a problem, but of a performant way too). And it made us focus on the candidate's coding skills too much (which some hiring managers might find objectionable).

mcv 8y ago
The advantage of using something like codility, or making it a take-home thing, is that they can do it in their own time, free from the stressful environment of an interview.
- umvi 8y ago
  The disadvantage of that is that the interviewer no longer knows if the candidate actually solved the problem or if he got his CS roommate to solve it for him.
  - toomanyrichies 8y ago
    Isn't that one of the reasons to ask follow-up questions like "why did you decide to use data structure X here instead of Y"? If the candidate didn't write the code, their lack of coherent answers should make that obvious pretty quickly.
    - mcv 8y ago
      Exactly. That's why you let them show their code to the other devs who get to ask questions. I don't believe someone who can't code can explain someone else's code in a convincing way.
- pytester 8y ago
  I hate doing take home projects. * It's way too easy to set a project that takes too long. I was given one the other day that would have taken about two to three full days. That simply doesn't happen if you do the test sitting in front of them - the time spent (and potentially wasted) is always capped because the test setter doesn't want to waste too much of their own time. * Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you. * Take home projects take employer skin out of the game. It's way too easy to give out twenty tests and ignore ten of them. I was actually half way through a homework test the other day when I was told that surprise they had an open offer and the candidate had just taken the job. I doubt they'd have wasted their own time testing me IRL knowing what they knew. From the employer's perspective: * In the last case I also realized after googling that the answers were online and I could just have copied someone else's. * You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test. I suspect this is partly why the take home tests I've been given are generally of a lower quality than the in person. * It's valuable to talk through the solution with the candidate while they do it (describing trade offs they made, pitfalls they're cognizant may crop up, potential downsides to their approach).

mcv 8y ago

I recently did some codility tests, and they were totally fine. I'm also fine with take-home challenges, though. But you're absolutely right that they're often vastly underestimated, and you shouldn't give someone a take-home assignment when there's only a small change they'll get hired. It should only be done once you're sure you want to hire this candidate if they're as good as they seem to be. A take-home assignment is too big to sift the chaff from the wheat in a large group of candidates. But once you're sure you'll hire them if they do well on it, a take-home test is totally fine. Though such a take-home programming assignment should also involve presenting the result to a group of developers to explain their choices. If you're not willing to commit to that, you can't expect them to commit to putting in the time to do the assignment.

ummonk 8y ago

Yeah, those challenges are explicitly timeboxed, and I know there is a relatively objective grading criteria (which is often as simple as “passes enough of the hidden test cases”). I refuse to do other types of take home challenges, but am fine with these.

spyspy 8y ago
The hidden test cases can be super frustrating, especially when you're passing 7/10 and the timer keeps ticking down. It's impossible to debug besides just staring at the code and guessing what else they might be looking for. Then to ultimately have the company immediately go, "Sorry you didn't check enough green boxes. You've failed. Goodbye."
- kamalkishor1991 8y ago
  Although sometimes this is true, most of the time you can't pass all the test because you have written a messed up code that you yourself can't understand. Learning to write the simplest code that works first time one of the most critical skills in programming and in real life also there are no test cases written for you.

andrei_says_ 8y ago

Timing a code challenge is a bit like chess boxing. Increased cortisol levels diminish or shut down the reasoning function of the brain, so expecting someone to code fast under the threat to lose a potential employnentbopportunity is not exactly productive.