3 ms·
The author is absolutely right that interviews that involve someone to code over the phone, on paper or on a whiteboard bias against a certain part of the devel
by kasey_junk 12y ago
The author is absolutely right that interviews that involve someone to code over the phone, on paper or on a whiteboard bias against a certain part of the developer population. Unless it is something people actually need to do for the position, you would be wise to remove "paper" coding from your hiring pipeline.
But, there is also a large part of the developer population that has the same reaction that the author did to "paper" coding, to any pair programming, including pairing with devs they've worked with for years. Developers in this population, who know that they are in it, generally will decline any job that actually involves large amounts of pair programming so you should reveal that you do lots of pairing early in your hiring pipeline.
If you do not actually do pair programming a lot in practice, this technique is no better, and in some ways worse, than "paper" programming.
My policy on any programming test is that it should be take home, non-timed and non-observed. Unless your environment has real, hard requirements about "how" someone writes code putting fake constraints on that part of the filtering process only biases against potentially great hires.
- dsacco 12y agoDevil's advocate here - how do you control against candidates plagiarizing code or cheating? Do you ask them about the code sample onsite in such a way as to determine if they understand it? I'm not sure where I stand on this, but I've heard many hiring managers complain that great phone and work sample candidates often fizz out entirely when tested on the sample in person - and not due to jitters, but due to e.g. not being able to comment their own code or explain one of their functions. I will say that I think a "homework" style coding test has always felt like a more fair assessment and been more enjoyable for me personally. I think a homework style work sample is probably one of the best methods for hiring, as long as it 1) won't take longer than an hour or so and 2) is technically exhaustive of typical job functions. That said, I know brain teasers are hated, and I dislike them too, but I really think most programmers should know standard algorithms and data structures. I don't think the current model of whiteboard coding is perfect, but knowledge of algorithm design is great for optimizing performant code.
- kasey_junk 12y agoIf you are in an environment that is large enough for your programming problems to be widely available, you are also large enough to build mechanical plagiarism detection systems (which are a pretty widely researched field oddly enough). As an aside, if you have that problem with submitted work, you also have it with whiteboard/paper problems. How do you distinguish between a "good" candidate and a "well coached" one in the whiteboard setting? I personally think if your hiring pipeline is generating lots of "cheaters" you need to address the candidate collection not the candidate evaluation. You are fishing in the wrong ponds, so to speak. I personally have no problem with candidates getting help on submitting their programming problem. I view effective help getting as a fundamental tool in a developers toolkit, so that in and of itself doesn't bother me. I think you can do an onsite code review, where the candidate presents their code sample, teaches you how to use it, explains the tradeoffs, responds to criticism, etc. This should be just like what you are doing for your "real" code reviews (if you don't do "real" code reviews, how do you know that any of your colleagues understand the code they write?) Over the years, I've grown a bias against even caring about that second step, because I have found so few people that do well on the programming problem part and then do poorly on the second part due to anything other than stage fright, but I understand I have an extreme position on that.
- dsacco 12y agoHow do you feel about testing algorithm design as a component of, or alongside a more "project based" homework test (project based meaning something like "Develop a few classes to interact with an API like so..."). Do you think developers need any grounding in foundation if it isn't explicitly part of their job description? Finally, do you administer the homework test via a third party like HackerRank or develop your own problem and simply email it to them?
- kasey_junk 12y agoI quite like algorithm design to be one of the judgement criteria for judging submitted work samples. Designing the work sample is very difficult, but the good ones have some box canyons that are easy to fall into without proper attention to algorithms. I personally have found my computer science academic background invaluable in my career, and think that foundational computer science is useful in nearly any software development position. That said, some of the very best developers I've worked with do not have that foundation, so I do not think it should be used as a selection criteria. I have historically used email for work samples and then fed them into whatever code review system the company is using. If I were setting up a big hiring pipeline right now I would probably due research about outsourcing the mechanics of collection/testing/code review collaboration to a 3rd party. That said, sending candidates through your real workflow can expose problems on your side, which is a nice side benefit.
- baggachipz 12y agoI couldn't agree with you more. To pile on here, I'd like to add that when you force somebody to do an observed pair programming interview, several things still do not match what will most likely be the day-to-day process. First, and most obviously, their environment will not be what they're used to, at all. Any pair tools out there are most likely very different from their normal setup. This immediately sows discomfort, no question. Second, let's be honest here: We all use documentation and google extensively when programming. That's simply due to the fact that we can't possibly remember all syntax and available functionality. But we know where to look. Doing that during an observed interview is usually a no-no. As an example: I recently applied at Trello. I nailed their programming challenge to be able to submit the resume, and I gave a thorough explanation for my solution in the submission. I got a call from HR and they scheduled an online interview where I would be sharing a code window with the interviewer and I'd have to solve a succession of problems. Shit. All of these teasers were easier than the initial problem I solved to get the interview in the first place, but with a spotlight on me and an implicit time limit that I could feel ticking past with the interviewer's silence, I went down in flames. I have no doubt that I'd be perfectly capable for the job itself, but that doesn't matter because I couldn't dance the right jig. I have since accepted employment with a company whose interview process cared more about what I have done and what I can do, rather than forcing me to participate in an unnatural process.
- vilda 12y agoYour method has one huge drawback - time consumption on candidate's side. Before doing this test I would recommend a simple preliminary test to quickly eliminate obvious misfits. It's also in align with Joel's recommendation[1]. At the end, there's too much factors you can't uncover before hiring, that at some point more testing in meaningless. My experience is that everybody has a preferred way how to work and a preferred topics, and allowing them to do just then yields most benefits. [1] http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html http://www.joelonsoftware.com/articles/GuerrillaInterviewing...
- kasey_junk 12y agoI absolutely think that your work sample project should have reasonable time requirements. I shoot for "a candidate that we would want to hire should be able to do this in 2-5 hours over the course of a week or 2". If a candidate is unwilling to invest that much time in our process, that is perfectly within their rights and I respect that. But I'm not going to sacrifice my time for them either. If I need to cut time commitments from my hiring pipeline I will do it at the interview component. I would gladly trade 4 hours of work sample time for a half day of onsite interviews for instance. As far as eliminating obvious misfits, I'd argue that a mechanical suite of tests (I like to provide say 70% of them to the candidate) gets much lower false negatives and much higher true positives than any "simpler" test you can devise. The Guerrilla guide is a very good starting point btw, but I think it misses the central problem, which is, interviewing is not a reliable way to filter candidates for software development jobs, no matter how you do it.