3 ms·

Devil's advocate here - how do you control against candidates plagiarizing code or cheating? Do you ask them about the code sample onsite in such a way as to de

by dsacco 12y ago

Devil's advocate here - how do you control against candidates plagiarizing code or cheating? Do you ask them about the code sample onsite in such a way as to determine if they understand it? I'm not sure where I stand on this, but I've heard many hiring managers complain that great phone and work sample candidates often fizz out entirely when tested on the sample in person - and not due to jitters, but due to e.g. not being able to comment their own code or explain one of their functions. I will say that I think a "homework" style coding test has always felt like a more fair assessment and been more enjoyable for me personally. I think a homework style work sample is probably one of the best methods for hiring, as long as it 1) won't take longer than an hour or so and 2) is technically exhaustive of typical job functions. That said, I know brain teasers are hated, and I dislike them too, but I really think most programmers should know standard algorithms and data structures. I don't think the current model of whiteboard coding is perfect, but knowledge of algorithm design is great for optimizing performant code.

kasey_junk 12y ago
If you are in an environment that is large enough for your programming problems to be widely available, you are also large enough to build mechanical plagiarism detection systems (which are a pretty widely researched field oddly enough). As an aside, if you have that problem with submitted work, you also have it with whiteboard/paper problems. How do you distinguish between a "good" candidate and a "well coached" one in the whiteboard setting? I personally think if your hiring pipeline is generating lots of "cheaters" you need to address the candidate collection not the candidate evaluation. You are fishing in the wrong ponds, so to speak. I personally have no problem with candidates getting help on submitting their programming problem. I view effective help getting as a fundamental tool in a developers toolkit, so that in and of itself doesn't bother me. I think you can do an onsite code review, where the candidate presents their code sample, teaches you how to use it, explains the tradeoffs, responds to criticism, etc. This should be just like what you are doing for your "real" code reviews (if you don't do "real" code reviews, how do you know that any of your colleagues understand the code they write?) Over the years, I've grown a bias against even caring about that second step, because I have found so few people that do well on the programming problem part and then do poorly on the second part due to anything other than stage fright, but I understand I have an extreme position on that.
- dsacco 12y ago
  How do you feel about testing algorithm design as a component of, or alongside a more "project based" homework test (project based meaning something like "Develop a few classes to interact with an API like so..."). Do you think developers need any grounding in foundation if it isn't explicitly part of their job description? Finally, do you administer the homework test via a third party like HackerRank or develop your own problem and simply email it to them?
  - kasey_junk 12y ago
    I quite like algorithm design to be one of the judgement criteria for judging submitted work samples. Designing the work sample is very difficult, but the good ones have some box canyons that are easy to fall into without proper attention to algorithms. I personally have found my computer science academic background invaluable in my career, and think that foundational computer science is useful in nearly any software development position. That said, some of the very best developers I've worked with do not have that foundation, so I do not think it should be used as a selection criteria. I have historically used email for work samples and then fed them into whatever code review system the company is using. If I were setting up a big hiring pipeline right now I would probably due research about outsourcing the mechanics of collection/testing/code review collaboration to a 3rd party. That said, sending candidates through your real workflow can expose problems on your side, which is a nice side benefit.
- Ntrails 12y ago
  I'd usually say the first part isn't that interesting compared to the second. Any take home challenge to some extent just tests how willing someone is to burn several hours generating code to maybe pass a stage in an interview. How would you feel about, say, providing a candidate with a chunk of [source] code and 30 minutes to read through and grok it. Then they can lead a discussion/review about their thoughts on its structure, and so on. This can move in to their preferred implementation, or just focus on consequences and risks etc. Essentially switching positions from the usual stage 2 where you're reading code and leading. I know reading code is in some ways easier than writing - but to me understanding foreign source is sometimes harder than writing a fresh tool in your own style etc. Being able to distill intent, aim etc.

by dsacco 12y ago

kasey_junk 12y ago

If you are in an environment that is large enough for your programming problems to be widely available, you are also large enough to build mechanical plagiarism detection systems (which are a pretty widely researched field oddly enough). As an aside, if you have that problem with submitted work, you also have it with whiteboard/paper problems. How do you distinguish between a "good" candidate and a "well coached" one in the whiteboard setting? I personally think if your hiring pipeline is generating lots of "cheaters" you need to address the candidate collection not the candidate evaluation. You are fishing in the wrong ponds, so to speak. I personally have no problem with candidates getting help on submitting their programming problem. I view effective help getting as a fundamental tool in a developers toolkit, so that in and of itself doesn't bother me. I think you can do an onsite code review, where the candidate presents their code sample, teaches you how to use it, explains the tradeoffs, responds to criticism, etc. This should be just like what you are doing for your "real" code reviews (if you don't do "real" code reviews, how do you know that any of your colleagues understand the code they write?) Over the years, I've grown a bias against even caring about that second step, because I have found so few people that do well on the programming problem part and then do poorly on the second part due to anything other than stage fright, but I understand I have an extreme position on that.

dsacco 12y ago
How do you feel about testing algorithm design as a component of, or alongside a more "project based" homework test (project based meaning something like "Develop a few classes to interact with an API like so..."). Do you think developers need any grounding in foundation if it isn't explicitly part of their job description? Finally, do you administer the homework test via a third party like HackerRank or develop your own problem and simply email it to them?
- kasey_junk 12y ago
  I quite like algorithm design to be one of the judgement criteria for judging submitted work samples. Designing the work sample is very difficult, but the good ones have some box canyons that are easy to fall into without proper attention to algorithms. I personally have found my computer science academic background invaluable in my career, and think that foundational computer science is useful in nearly any software development position. That said, some of the very best developers I've worked with do not have that foundation, so I do not think it should be used as a selection criteria. I have historically used email for work samples and then fed them into whatever code review system the company is using. If I were setting up a big hiring pipeline right now I would probably due research about outsourcing the mechanics of collection/testing/code review collaboration to a 3rd party. That said, sending candidates through your real workflow can expose problems on your side, which is a nice side benefit.
Ntrails 12y ago
I'd usually say the first part isn't that interesting compared to the second. Any take home challenge to some extent just tests how willing someone is to burn several hours generating code to maybe pass a stage in an interview. How would you feel about, say, providing a candidate with a chunk of [source] code and 30 minutes to read through and grok it. Then they can lead a discussion/review about their thoughts on its structure, and so on. This can move in to their preferred implementation, or just focus on consequences and risks etc. Essentially switching positions from the usual stage 2 where you're reading code and leading. I know reading code is in some ways easier than writing - but to me understanding foreign source is sometimes harder than writing a fresh tool in your own style etc. Being able to distill intent, aim etc.