4 ms·

Have every candidate work on the same problems, and let them do it from home To be fair, I've seen a lot of good arguments against programming assignments.

Have every candidate work on the same problems, and let them do it from home To be fair, I've seen a lot of good arguments against programming assignments. I think at the end of the day, the employer needs to conduct some method of determining if the employee has the technical capabilities needed for the specific job they're being hired for. However, there are MANY other factors too like "how well they get along with the team" that need to be considered. That's what they're trying to determine with the pairings and other stuff you mentioned I think. Whether that works is another story.

tptacek 11y ago
Every team I've talked to starts out with some X-factor they think they need to assess for. It's my belief that if you can have the discipline to stop filtering for X-factors, you'll build better, more effective teams, because those factors are really just vectors for personal biases. Regardless of whether you agree with me about that, I think we can all stipulate that if work-sample technical evaluations work (and: they do), most of what companies try to evaluate in on-site interviews is stupid. No part of working effectively with a team requires timed recall of how to implement a stable quicksort, or reversing a doubly linked list at a whiteboard. At the very least, using work sample tests allows you to build an on-site interview process that honestly engages with "team fit" (or whatever your X-factor is). My guess, though, is that when more teams adopt work samples and then go through the motions of trying to design a pure team-fit interview, they're going to realize --- once they don't have "implement Bellman-Ford on this whiteboard" to fall back on --- how unequipped they always were to evaluate team-fit in the first place.
- bcg1 11y ago
  The problem could also be that many teams simply don't have the time or possess the knowledge to do an assessment like you're suggesting (which I agree is a good way to assess candidates). It requires someone to actually design an assignment (which is a task that many coders might not be good at), and requires one or more people to evaluate it. The ridiculous whiteboard coding of puzzles probably stems from laziness or inability to implement what you're suggesting. I know that personally, if someone asked me to developed a work sample evaluation for my job, it would take me many hours to come up with something, and frankly I even once complete we would have no way of knowing if it is actually a good predictor of whether or not someone is the right employee.
  - tptacek 11y ago
    Hours? It could take a week, and if you're going to hire more than one person this quarter, it will still be worth it just in the time savings from not having developers deliver bad interviews.
    - benmccann 11y ago
      Have you done this? My worry would be that sooner or later you'll have a candidate who posts the assignment online ruining your weeks of effort. It also seems like it makes it much easier to cheat if you make it a take home.
      - tptacek 11y ago
        Yes, I ran a process like this for several years. I ran recruiting for the largest software security firm in the US; before that firm bought us, I used this process to more than double the size of my company. When I left, to work on a recruiting startup, not one of the people I'd overseen hiring had quit or been fired. I'm religious about this topic for a reason. :)
        dpritchett 11y ago
        Was the 0% turnover due to great teammates, great projects, or above-market comp? I've never seen 0%!
        tptacek 11y ago
        We paid the market median to new hires; we definitely didn't buy our way to that turnover (NCC pays better than a lot of early stage startups, but not better than late-stage ones). No, I attribute the turnover to the recruiting mechanism. We found great people who were sorely mispriced by the market, and we took advantage of that to create a win-win scenario: people without the resume to get a similar job at a competing firm got an extremely impactful resume bullet and a good-paying job, and we got people who genuinely wanted to be on our team and weren't applying as a once-a-year lateral job-hopping gamble. (I have no problem with people job-hopping, by the way, but every employer is trying to minimize their exposure to that.)
dclowd9901 11y ago
If you substitute the candidate's time in an in person technical interview with the estimated time for the take home assignment, I can't see the issue. I would gladly give 3 hours of bullshit algorithm white boarding exercises for 3 hours of homework.

tptacek 11y ago

Every team I've talked to starts out with some X-factor they think they need to assess for. It's my belief that if you can have the discipline to stop filtering for X-factors, you'll build better, more effective teams, because those factors are really just vectors for personal biases. Regardless of whether you agree with me about that, I think we can all stipulate that if work-sample technical evaluations work (and: they do), most of what companies try to evaluate in on-site interviews is stupid. No part of working effectively with a team requires timed recall of how to implement a stable quicksort, or reversing a doubly linked list at a whiteboard. At the very least, using work sample tests allows you to build an on-site interview process that honestly engages with "team fit" (or whatever your X-factor is). My guess, though, is that when more teams adopt work samples and then go through the motions of trying to design a pure team-fit interview, they're going to realize --- once they don't have "implement Bellman-Ford on this whiteboard" to fall back on --- how unequipped they always were to evaluate team-fit in the first place.

bcg1 11y ago
The problem could also be that many teams simply don't have the time or possess the knowledge to do an assessment like you're suggesting (which I agree is a good way to assess candidates). It requires someone to actually design an assignment (which is a task that many coders might not be good at), and requires one or more people to evaluate it. The ridiculous whiteboard coding of puzzles probably stems from laziness or inability to implement what you're suggesting. I know that personally, if someone asked me to developed a work sample evaluation for my job, it would take me many hours to come up with something, and frankly I even once complete we would have no way of knowing if it is actually a good predictor of whether or not someone is the right employee.
- tptacek 11y ago
  Hours? It could take a week, and if you're going to hire more than one person this quarter, it will still be worth it just in the time savings from not having developers deliver bad interviews.
  - benmccann 11y ago
    Have you done this? My worry would be that sooner or later you'll have a candidate who posts the assignment online ruining your weeks of effort. It also seems like it makes it much easier to cheat if you make it a take home.
    - tptacek 11y ago
      Yes, I ran a process like this for several years. I ran recruiting for the largest software security firm in the US; before that firm bought us, I used this process to more than double the size of my company. When I left, to work on a recruiting startup, not one of the people I'd overseen hiring had quit or been fired. I'm religious about this topic for a reason. :)
      - dpritchett 11y ago
        Was the 0% turnover due to great teammates, great projects, or above-market comp? I've never seen 0%!
        tptacek 11y ago
        We paid the market median to new hires; we definitely didn't buy our way to that turnover (NCC pays better than a lot of early stage startups, but not better than late-stage ones). No, I attribute the turnover to the recruiting mechanism. We found great people who were sorely mispriced by the market, and we took advantage of that to create a win-win scenario: people without the resume to get a similar job at a competing firm got an extremely impactful resume bullet and a good-paying job, and we got people who genuinely wanted to be on our team and weren't applying as a once-a-year lateral job-hopping gamble. (I have no problem with people job-hopping, by the way, but every employer is trying to minimize their exposure to that.)

dclowd9901 11y ago

If you substitute the candidate's time in an in person technical interview with the estimated time for the take home assignment, I can't see the issue. I would gladly give 3 hours of bullshit algorithm white boarding exercises for 3 hours of homework.