2 ms·
If you are in an environment that is large enough for your programming problems to be widely available, you are also large enough to build mechanical plagiarism
by kasey_junk 12y ago
If you are in an environment that is large enough for your programming problems to be widely available, you are also large enough to build mechanical plagiarism detection systems (which are a pretty widely researched field oddly enough). As an aside, if you have that problem with submitted work, you also have it with whiteboard/paper problems. How do you distinguish between a "good" candidate and a "well coached" one in the whiteboard setting? I personally think if your hiring pipeline is generating lots of "cheaters" you need to address the candidate collection not the candidate evaluation. You are fishing in the wrong ponds, so to speak.
I personally have no problem with candidates getting help on submitting their programming problem. I view effective help getting as a fundamental tool in a developers toolkit, so that in and of itself doesn't bother me.
I think you can do an onsite code review, where the candidate presents their code sample, teaches you how to use it, explains the tradeoffs, responds to criticism, etc. This should be just like what you are doing for your "real" code reviews (if you don't do "real" code reviews, how do you know that any of your colleagues understand the code they write?)
Over the years, I've grown a bias against even caring about that second step, because I have found so few people that do well on the programming problem part and then do poorly on the second part due to anything other than stage fright, but I understand I have an extreme position on that.
- dsacco 12y agoHow do you feel about testing algorithm design as a component of, or alongside a more "project based" homework test (project based meaning something like "Develop a few classes to interact with an API like so..."). Do you think developers need any grounding in foundation if it isn't explicitly part of their job description? Finally, do you administer the homework test via a third party like HackerRank or develop your own problem and simply email it to them?
- kasey_junk 12y agoI quite like algorithm design to be one of the judgement criteria for judging submitted work samples. Designing the work sample is very difficult, but the good ones have some box canyons that are easy to fall into without proper attention to algorithms. I personally have found my computer science academic background invaluable in my career, and think that foundational computer science is useful in nearly any software development position. That said, some of the very best developers I've worked with do not have that foundation, so I do not think it should be used as a selection criteria. I have historically used email for work samples and then fed them into whatever code review system the company is using. If I were setting up a big hiring pipeline right now I would probably due research about outsourcing the mechanics of collection/testing/code review collaboration to a 3rd party. That said, sending candidates through your real workflow can expose problems on your side, which is a nice side benefit.
- Ntrails 12y agoI'd usually say the first part isn't that interesting compared to the second. Any take home challenge to some extent just tests how willing someone is to burn several hours generating code to maybe pass a stage in an interview. How would you feel about, say, providing a candidate with a chunk of [source] code and 30 minutes to read through and grok it. Then they can lead a discussion/review about their thoughts on its structure, and so on. This can move in to their preferred implementation, or just focus on consequences and risks etc. Essentially switching positions from the usual stage 2 where you're reading code and leading. I know reading code is in some ways easier than writing - but to me understanding foreign source is sometimes harder than writing a fresh tool in your own style etc. Being able to distill intent, aim etc.
- kasey_junk 12y ago> how willing someone is to burn several hours generating code to maybe pass a stage in an interview. We are well into the area where it is my biases talking and not any sort of well researched finding, but I don't believe in developers who can turn on/off writing good code. Either a developer writes good code, or they don't. Further, I take getting my next job very seriously. I take hiring my next colleague equally seriously. I'd hope that this seriousness (in combination with me asking for a reasonable amount of work sample) would encourage everyone involved to put their best foot forward. > How would you feel about, say, providing a candidate with a chunk of [source] code and 30 minutes to read through and grok it. I have never tried this so don't have any experiential recommendations. My instinct is to think that this would be problematic, not because it is wrong to judge someones ability to learn foreign code, rather because the evaluation step would be very hard to filter for someone who learned foreign code quickly, from someone who is great at talking about stuff generally. If I wanted to judge someones foreign code ability, I would as part of the code project, expect them to interact with a provided set of foreign code. I'd set explicit boundaries about how much they could or could not change it and then judge the candidate on how their new code interacted with the legacy code as well as what and how they changed the legacy system. > I know reading code is in some ways easier than writing I don't think that is in the least bit true. I think evaluating a candidates ability to interact with legacy code is a hugely valuable thing to filter on in most environments, it's just very hard to design a fair and repeatable work sample problem around it.