4 ms·

When we were looking for a developer for our team, we created a set of small coding challenges testing the candidate's knowledge of the most relevant aspects of

When we were looking for a developer for our team, we created a set of small coding challenges testing the candidate's knowledge of the most relevant aspects of the language. We wrote automated tests for these challenges, and the main bulk of our interview was having the candidate solve them. The advantages to that approach, as I see it, are: - the developer is tested for knowledge that is most relevant for their current job (as opposed to writing general-purpose algorithms) - the developer is showing their familiarity with a testing environment - writing working code in a code editor seems to me more practical than writing (pseudo)code on paper or on a whiteboard There are, of course, disadvantages to this approach as well. The developer may have hard time working on an unfamiliar machine and not in their favorite code editor (though what we had was quite standard). This might be a tad stressful (although I don't know how this compares with solving a general coding problem on a whiteboard, or with a task asking you to think not just of a way to solve a problem, but of a performant way too). And it made us focus on the candidate's coding skills too much (which some hiring managers might find objectionable).

mcv 8y ago
The advantage of using something like codility, or making it a take-home thing, is that they can do it in their own time, free from the stressful environment of an interview.
- umvi 8y ago
  The disadvantage of that is that the interviewer no longer knows if the candidate actually solved the problem or if he got his CS roommate to solve it for him.
  - toomanyrichies 8y ago
    Isn't that one of the reasons to ask follow-up questions like "why did you decide to use data structure X here instead of Y"? If the candidate didn't write the code, their lack of coherent answers should make that obvious pretty quickly.
    - mcv 8y ago
      Exactly. That's why you let them show their code to the other devs who get to ask questions. I don't believe someone who can't code can explain someone else's code in a convincing way.
- pytester 8y ago
  I hate doing take home projects. * It's way too easy to set a project that takes too long. I was given one the other day that would have taken about two to three full days. That simply doesn't happen if you do the test sitting in front of them - the time spent (and potentially wasted) is always capped because the test setter doesn't want to waste too much of their own time. * Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you. * Take home projects take employer skin out of the game. It's way too easy to give out twenty tests and ignore ten of them. I was actually half way through a homework test the other day when I was told that surprise they had an open offer and the candidate had just taken the job. I doubt they'd have wasted their own time testing me IRL knowing what they knew. From the employer's perspective: * In the last case I also realized after googling that the answers were online and I could just have copied someone else's. * You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test. I suspect this is partly why the take home tests I've been given are generally of a lower quality than the in person. * It's valuable to talk through the solution with the candidate while they do it (describing trade offs they made, pitfalls they're cognizant may crop up, potential downsides to their approach).
  - mcv 8y ago
    > It's way too easy to set a project that takes too long. They do often take longer than they claim. That's true. > Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you. In my experience, you're usually assigned someone you can call or mail with questions. Asking questions is good. Though I usually figure it out on my own. > Take home projects take employer skin out of the game. It's way too easy to give out twenty tests and ignore ten of them Only if they do it wrong. The right way is to give this assignment only to candidates you will definitely want to hire, unless they produce terrible code. The candidate doesn't mail in their code, but presents it to a group of developers who ask questions about it. This gives the employer skin in the game: they're not going to keep their developers off their work for dozens of candidates who are probably not going to make it; they only bring them out to assess a very likely hire. I guess some employers use these coding assignments badly. You're totally right to avoid those. > I could just have copied someone else's. But could you have been able to explain the choices of that someone else? > You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test. Maybe, but to many people, coding on the spot is more stressful than it is at home. Although I've also done some really enjoyable pair programming on what was probably my only long interview day (which had an interview, pair programming, and a game to get to know each other). > It's valuable to talk through the solution with the candidate while they do it But it's also valuable to talk through the solution after they've done it. That's when they have made those trade-offs.
saiya-jin 8y ago
You will favor those that used exactly same tools/frameworks/testing, but generally smarter person might get a disadvantage. Ie give me Intellij Idea and I would be a tad slower since I use Eclipse primarily. This difference would be mitigated 2 days after hiring, but would be visible during interview. One tiny issue with the env can maybe block their effort measurably. I get the advantages, I guess everybody does, but there are disadvantages too.

mcv 8y ago

The advantage of using something like codility, or making it a take-home thing, is that they can do it in their own time, free from the stressful environment of an interview.

umvi 8y ago
The disadvantage of that is that the interviewer no longer knows if the candidate actually solved the problem or if he got his CS roommate to solve it for him.
- toomanyrichies 8y ago
  Isn't that one of the reasons to ask follow-up questions like "why did you decide to use data structure X here instead of Y"? If the candidate didn't write the code, their lack of coherent answers should make that obvious pretty quickly.
  - mcv 8y ago
    Exactly. That's why you let them show their code to the other devs who get to ask questions. I don't believe someone who can't code can explain someone else's code in a convincing way.
pytester 8y ago
I hate doing take home projects. * It's way too easy to set a project that takes too long. I was given one the other day that would have taken about two to three full days. That simply doesn't happen if you do the test sitting in front of them - the time spent (and potentially wasted) is always capped because the test setter doesn't want to waste too much of their own time. * Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you. * Take home projects take employer skin out of the game. It's way too easy to give out twenty tests and ignore ten of them. I was actually half way through a homework test the other day when I was told that surprise they had an open offer and the candidate had just taken the job. I doubt they'd have wasted their own time testing me IRL knowing what they knew. From the employer's perspective: * In the last case I also realized after googling that the answers were online and I could just have copied someone else's. * You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test. I suspect this is partly why the take home tests I've been given are generally of a lower quality than the in person. * It's valuable to talk through the solution with the candidate while they do it (describing trade offs they made, pitfalls they're cognizant may crop up, potential downsides to their approach).
- mcv 8y ago
  > It's way too easy to set a project that takes too long. They do often take longer than they claim. That's true. > Take home projects inevitably have issues ("bugs", if you will). You can't just "resolve" those problems if the person who set it isn't sitting in front of you. In my experience, you're usually assigned someone you can call or mail with questions. Asking questions is good. Though I usually figure it out on my own. > Take home projects take employer skin out of the game. It's way too easy to give out twenty tests and ignore ten of them Only if they do it wrong. The right way is to give this assignment only to candidates you will definitely want to hire, unless they produce terrible code. The candidate doesn't mail in their code, but presents it to a group of developers who ask questions about it. This gives the employer skin in the game: they're not going to keep their developers off their work for dozens of candidates who are probably not going to make it; they only bring them out to assess a very likely hire. I guess some employers use these coding assignments badly. You're totally right to avoid those. > I could just have copied someone else's. But could you have been able to explain the choices of that someone else? > You lose a valuable source of feedback which allows you to iteratively improve the test if you're not watching the candidate while they do the test. Maybe, but to many people, coding on the spot is more stressful than it is at home. Although I've also done some really enjoyable pair programming on what was probably my only long interview day (which had an interview, pair programming, and a game to get to know each other). > It's valuable to talk through the solution with the candidate while they do it But it's also valuable to talk through the solution after they've done it. That's when they have made those trade-offs.

saiya-jin 8y ago

You will favor those that used exactly same tools/frameworks/testing, but generally smarter person might get a disadvantage. Ie give me Intellij Idea and I would be a tad slower since I use Eclipse primarily. This difference would be mitigated 2 days after hiring, but would be visible during interview. One tiny issue with the env can maybe block their effort measurably. I get the advantages, I guess everybody does, but there are disadvantages too.