5 ms·

Despite some of the comments here, I'm actually a fan of the 'homework' format. I'm not always great at hiring, but I know how to gauge the 'homework' interview

Despite some of the comments here, I'm actually a fan of the 'homework' format. I'm not always great at hiring, but I know how to gauge the 'homework' interviews, and I've usually been right about the interviewee's abilities afterwards, WAY moreso than whiteboarding or riddles or any of the other alternatives. That being said, I would add some constraints: 1. The 'task' shouldn't take more than 30 min to understand, and no more than 1-2 hours to do. If it takes more than that, you should make some obfuscated dummy-code endpoints that do everything you're not directly evaluating on. 2. You should let the interviewee know that it should only take 2 hours as well. If it takes more, then there was either a misunderstanding in the task, or the interviewee doesn't have the right knowledge/experience/qualifications etc. 3. You should do your homework as well, meaning you've thoroughly reviewed the code submitted and have detailed questions ready to go for the interviewee. This should also be the 'control' to make sure the interviewee really did write the code (you'll know really quickly if they didn't) 4. No other coding tasks in the interview process. You can have them review code or suggest approaches to solve problems (things that people do on the job with others) 5. The interview should be shorter than if they were asked to do whiteboarding or live coding or anything like that. Most employers have (on paper) "trial periods" of around 3 months. A 1-2 hour coding assignment followed by 1 hour of well-crafted questions about the assignment (plus the usual interview stuff) should be enough to know whether or not you want to give the employee a chance for 3 months. (I realize most employers don't use the evaluation period for evaluative tasks, they just have the person start and hope for the best, but if done right, it's actually a pretty good system)

bgdam 8y ago
Well when done this way with all the constraints you specified, then yes the homework interview works very well. Unfortunately, the vast majority of hiring managers do not come anywhere close to any of the points you mentioned. Just last week, I was interviewing for an opportunity as a VP of Engineering for a post-early-stage startup. The CTO gave me a take home assignment involving building not one but three complete applications (to be delivered in a docker container so it could be easily run). When I told him that I didn't have the kind of time it would take to perform this, he told me all the other candidates for the same role had no issues with it. I ended up withdrawing from the process. If senior roles itself are being hired like this, then I can only imagine how much worse it would be for individual contributor engineering roles.
- gowld 8y ago
  Senior roles should have more demanding technical interviews than junior roles. The company had a bullshit-preventing immune system. It's not a culture fit for you. Win-win.
  - bgdam 8y ago
    When hiring for a senior role, you should also consider that the person, who is already in a similar role, would not have the time to spend on your humongous homework app. When literally the only free time I have for myself and my family is a few hours everyday and then the weekends, I am definitely not going to spend the majority of that on an interview, not would I expect my candidates to do so if I were the one doing the hiring. So yes, I suppose in a way it was not a culture fit for me, and that was indeed why I ended up withdrawing. I would also urge anybody interviewing for any role to evaluate whether a company which requires you to spend a substantial portion of your personal time, before you're even an employee, has the kind of culture you're okay with.
- mywittyname 8y ago
  There's a trade-off for companies. This kind of hiring process will filter out 99% of low-quality candidates, while also filtering out <99% of quality candidates. So it increases their odds of a not selecting a poor choice. This is great for companies who are willing to sacrifice getting the best candidate for assurance they won't get a bad one. This makes sense for a small startup, where a poor candidate hurts growth more than a great candidate helps it, but less so for a Google-class company where a single great candidate is like a billion dollar lottery ticket and the bad candidates cost almost nothing.
animal531 8y ago
I'm going to disagree with you on a number of reasons. 1. A lot of studies out there show that the most important factor for a hiring success is culture, not skill. 2. Are you even measuring skill with your test? It doesn't matter if it should only take 2h for a candidate to complete your test, if they're nervous or want to impress they're going to spend 12h on it. How can you tell the difference? 3. What about the really good candidate that doesn't want to spend 2h on your test, because he also went to 2 other interviews in the same day where no tests were demanded of him. You're again not testing for skill, but willingness (and against other employer's testing regimes). 4. What is your test about? As a random example implement Web Service X that does Y. Is it important for you that candidates know how to implement X from the ground up? Will they be doing a lot of Y? Why spend 2 hours effectively answering 2 questions, where you could have interviewed him in 2 hours covering his whole history of computing experience, what he loves about it, why, what code he's written, what he's into, what he thinks about X,Y,Z etc. 5. Your job is not to impress him with your knowledge (not that I'm saying this is what you're doing, just a general remark on interviewers out there), but to determine what his is, where he's strong and where not, and how the person will fit into your structure. Making your test the big determining factor is a failure. 6. There's a reason fizzbuzz for example makes a great starter test. It immediately weeds out people that can't complete it, and allows you to see the interviewee reason through it. He goes for example, well, I take this and that, then I have to do this; oh yeah and so so etc. It allows you to see a lot more than just the end answer. After that you can apply more interesting/harder questions, but focus rather on how they do it than if they can complete it in the allotted 5 minutes.
- shortoncash 8y ago
  Regarding #2, I believe your observations and concerns are correct and valid. However, in these situations where the candidate spills his technical heart out, you really start to see the difference between good candidates and bad candidates. Good candidates just have a deep sense of the problem space, can enumerate issues, document such things, etc.
- cimmanom 8y ago
  I find code review-style discussion of a simple coding challenge to be great for assessing both culture fit and skill. What we administer is like a web app equivalent to FizzBuzz in terms of difficulty level. But unlike a whiteboard FizzBuzz, it doesn't put the developer on the spot. As an extra bonus, unlike FizzBuzz, you'll get different results from the kid who just successfully finished a CS degree and the grizzled veteran who's built and maintained complex systems for decades. The right challenge can be completed in an hour or so by a mid-level developer in a brute-force manner. It will weed out those who simply can't code (or don't grok how the web works or how apps are constructed); and also highlight candidates who are capable of more sophisticated work. If being up to speed on a specific technology from Day 1 isn't important, you can give the developer their choice of framework or language or whatever. Then when you do a code review in person you can get a feel for both how the developer approaches code (a skill) -- but more importantly, how they take critique. Do they get defensive? Do they riff off your ideas? Do they suggest improvements/refactorings they would have made if they had more time? You can also explore other cultural factors - for instance, how did they approach constraint trade-offs in terms of time vs. completeness vs. sophistication? How clearly are they able to communicate about technical concepts?
clavalle 8y ago
Your criteria are almost exactly what I assign. I start with a phone screen. Then onto a (relatively short 45 min to an hour and a half) face-to-face. This is a 'let's get to know each other and our expectations' low-stress interview just to make sure we understand each other and what it is we're actually getting into. Then, a technical discussion. Approaches. Experience. Past projects. Some 'did you lie on your resume' questions but nothing that would require a whiteboard or anything. This culminates in getting a paragraph of requirements that will lead to the take-home test. The requirements are intentionally vague and missing critical information to see if the candidate recognizes this and asks the right questions (I'm convinced that 60% of any job is asking the right questions). Then, when I feel like they've hit a few of the important gaps I pull out the detailed specs that tries to answer all of the questions they might have, serve as a reference sheet, and make it clear what not to do (like turning a two hour task into an over-engineered weekend project just to try to impress -- I've got to review all of this code after all!). This is not a real world test but it is a somewhat simplified version of a real-world thing they will be expected to do. It shouldn't take more than 2 hours. I point out that if they have any code they can show me from a real, working project it can act as a substitute. What I am looking for is knowledge, approach to the problem, and code style. Afterward, I am looking for how they handle constructive criticism. This has saved me and I think applicants so much time with much higher quality information than a long interview or a nerve-wracking whiteboard session. I give them five days to a week to finish. Most do it within a day or two. If a busy person can't find the time that they'd normally spend in an interview session essentially any time they can fit it in over the next week or so...well, too bad. Oh, I also give them explicit permission to post their solution on github so that if we don't end up hiring at least they'll have some code to show the next potential hiring manager.

bgdam 8y ago

Well when done this way with all the constraints you specified, then yes the homework interview works very well. Unfortunately, the vast majority of hiring managers do not come anywhere close to any of the points you mentioned. Just last week, I was interviewing for an opportunity as a VP of Engineering for a post-early-stage startup. The CTO gave me a take home assignment involving building not one but three complete applications (to be delivered in a docker container so it could be easily run). When I told him that I didn't have the kind of time it would take to perform this, he told me all the other candidates for the same role had no issues with it. I ended up withdrawing from the process. If senior roles itself are being hired like this, then I can only imagine how much worse it would be for individual contributor engineering roles.

gowld 8y ago
Senior roles should have more demanding technical interviews than junior roles. The company had a bullshit-preventing immune system. It's not a culture fit for you. Win-win.
- bgdam 8y ago
  When hiring for a senior role, you should also consider that the person, who is already in a similar role, would not have the time to spend on your humongous homework app. When literally the only free time I have for myself and my family is a few hours everyday and then the weekends, I am definitely not going to spend the majority of that on an interview, not would I expect my candidates to do so if I were the one doing the hiring. So yes, I suppose in a way it was not a culture fit for me, and that was indeed why I ended up withdrawing. I would also urge anybody interviewing for any role to evaluate whether a company which requires you to spend a substantial portion of your personal time, before you're even an employee, has the kind of culture you're okay with.
mywittyname 8y ago
There's a trade-off for companies. This kind of hiring process will filter out 99% of low-quality candidates, while also filtering out <99% of quality candidates. So it increases their odds of a not selecting a poor choice. This is great for companies who are willing to sacrifice getting the best candidate for assurance they won't get a bad one. This makes sense for a small startup, where a poor candidate hurts growth more than a great candidate helps it, but less so for a Google-class company where a single great candidate is like a billion dollar lottery ticket and the bad candidates cost almost nothing.

animal531 8y ago

I'm going to disagree with you on a number of reasons. 1. A lot of studies out there show that the most important factor for a hiring success is culture, not skill. 2. Are you even measuring skill with your test? It doesn't matter if it should only take 2h for a candidate to complete your test, if they're nervous or want to impress they're going to spend 12h on it. How can you tell the difference? 3. What about the really good candidate that doesn't want to spend 2h on your test, because he also went to 2 other interviews in the same day where no tests were demanded of him. You're again not testing for skill, but willingness (and against other employer's testing regimes). 4. What is your test about? As a random example implement Web Service X that does Y. Is it important for you that candidates know how to implement X from the ground up? Will they be doing a lot of Y? Why spend 2 hours effectively answering 2 questions, where you could have interviewed him in 2 hours covering his whole history of computing experience, what he loves about it, why, what code he's written, what he's into, what he thinks about X,Y,Z etc. 5. Your job is not to impress him with your knowledge (not that I'm saying this is what you're doing, just a general remark on interviewers out there), but to determine what his is, where he's strong and where not, and how the person will fit into your structure. Making your test the big determining factor is a failure. 6. There's a reason fizzbuzz for example makes a great starter test. It immediately weeds out people that can't complete it, and allows you to see the interviewee reason through it. He goes for example, well, I take this and that, then I have to do this; oh yeah and so so etc. It allows you to see a lot more than just the end answer. After that you can apply more interesting/harder questions, but focus rather on how they do it than if they can complete it in the allotted 5 minutes.

shortoncash 8y ago
Regarding #2, I believe your observations and concerns are correct and valid. However, in these situations where the candidate spills his technical heart out, you really start to see the difference between good candidates and bad candidates. Good candidates just have a deep sense of the problem space, can enumerate issues, document such things, etc.
cimmanom 8y ago
I find code review-style discussion of a simple coding challenge to be great for assessing both culture fit and skill. What we administer is like a web app equivalent to FizzBuzz in terms of difficulty level. But unlike a whiteboard FizzBuzz, it doesn't put the developer on the spot. As an extra bonus, unlike FizzBuzz, you'll get different results from the kid who just successfully finished a CS degree and the grizzled veteran who's built and maintained complex systems for decades. The right challenge can be completed in an hour or so by a mid-level developer in a brute-force manner. It will weed out those who simply can't code (or don't grok how the web works or how apps are constructed); and also highlight candidates who are capable of more sophisticated work. If being up to speed on a specific technology from Day 1 isn't important, you can give the developer their choice of framework or language or whatever. Then when you do a code review in person you can get a feel for both how the developer approaches code (a skill) -- but more importantly, how they take critique. Do they get defensive? Do they riff off your ideas? Do they suggest improvements/refactorings they would have made if they had more time? You can also explore other cultural factors - for instance, how did they approach constraint trade-offs in terms of time vs. completeness vs. sophistication? How clearly are they able to communicate about technical concepts?

clavalle 8y ago

Your criteria are almost exactly what I assign. I start with a phone screen. Then onto a (relatively short 45 min to an hour and a half) face-to-face. This is a 'let's get to know each other and our expectations' low-stress interview just to make sure we understand each other and what it is we're actually getting into. Then, a technical discussion. Approaches. Experience. Past projects. Some 'did you lie on your resume' questions but nothing that would require a whiteboard or anything. This culminates in getting a paragraph of requirements that will lead to the take-home test. The requirements are intentionally vague and missing critical information to see if the candidate recognizes this and asks the right questions (I'm convinced that 60% of any job is asking the right questions). Then, when I feel like they've hit a few of the important gaps I pull out the detailed specs that tries to answer all of the questions they might have, serve as a reference sheet, and make it clear what not to do (like turning a two hour task into an over-engineered weekend project just to try to impress -- I've got to review all of this code after all!). This is not a real world test but it is a somewhat simplified version of a real-world thing they will be expected to do. It shouldn't take more than 2 hours. I point out that if they have any code they can show me from a real, working project it can act as a substitute. What I am looking for is knowledge, approach to the problem, and code style. Afterward, I am looking for how they handle constructive criticism. This has saved me and I think applicants so much time with much higher quality information than a long interview or a nerve-wracking whiteboard session. I give them five days to a week to finish. Most do it within a day or two. If a busy person can't find the time that they'd normally spend in an interview session essentially any time they can fit it in over the next week or so...well, too bad. Oh, I also give them explicit permission to post their solution on github so that if we don't end up hiring at least they'll have some code to show the next potential hiring manager.