9 ms·
We use a 3 step process, sans whiteboard: 1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our te
by 11thEarlOfMar 6y ago
We use a 3 step process, sans whiteboard:
1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. If they express interest by being prepared and asking good questions, we send them home with
2. a programming task. Choose 1 of 5 tasks. The tasks are not abstract problems or puzzles, but come out of the designs we've implemented. We ask for 100 lines of code and not more than 2 hours. They take as much calendar time in days as they need, most have full time jobs, and let us know when you're done. The candidate then comes back, typically in 2 to 7 days and hosts a code review in front of our team. I give strict instructions before hand: The purpose of the review is to learn how they thought through the problem and how they solved it, not to criticize their style and approach.
3. If we decide we like them to this point, the third interview is with managers from other functions. How well does the candidate communicate, come across to non-developers, express interest in the company and role, etc. It's a check point to look for concerning weaknesses, as well as get buy in from the broader organization.
We like this approach because it allows the candidate to code in a much more natural environment. They can take the time they need and comfortably solve. No one spends their professional career coding on a white board. This approach so far has yielded excellent results. We ask developers how much time they took in the programming task and why they chose the one they solved. The programming task is telling, not in the quality of their code and how long they took, but how much did they get into it? We have had the range from candidates who did not complete it at all and opted out, to candidates who stumped 40 year veterans with elegant code. In every case, we learn how well they can express their thoughts, and importantly, their level of love for the discipline. This is as important to me as any other attribute.
- victor9000 6y agoThis sounds like a pretty reasonable process, are you guys hiring?
- 11thEarlOfMar 6y agoDigital Dynamics, a small company in Scotts Valley, Ca. We make specialty industrial control computers.
- naavis 6y agoDo you have some kind of a screening process before the first interview?
- 11thEarlOfMar 6y agoTypically an introductory phone call: ‘is this the kind of job you’d be interested in.’ Maybe that’s step 0.
- deleted 6y ago[deleted]
- fiblye 6y agoA lot of people in other comments below are criticizing take-home tasks, but honestly, this sounds nice and balanced. I'd much rather have a couple days to think about a problem and a few hours to implement it over having to take time off work, pay to go all the way to your company at a time that works for both of us, and deal with a white board interview. It's a way bigger investment. So long as it's a small, confined, practical problem and not a full app, this sounds good. Plus, if your company gives someone what they might think are ridiculous questions or they realize it's just not what they're good at, they can easily reject proceeding any farther with the interview. If someone walks into a crappy whiteboard interview, they feel kind of stuck. Seems to be better for both sides. Less pressure.
- wildrhythms 6y agoI oppose take-home tasks as an interviewer. I don't think employees or prospective employees should be taking work home with them, ever. Instead, collaborating on a problem with the interviewee, discussing the problem and potential solutions, observing how the interviewee recognizes when they go off-track, how they respond to hints and suggestions... these are all better signals than the culmination of some homework task, and easily captured in a 30-45 minute interview.
- 11thEarlOfMar 6y agoWe have both elements. The code they wrote is subsequently reviewed live with 3 or more of our engineers and all of these aspects are explored. I think what gives our approach balance is that we are also committing to making a time investment. Reviews typically last an hour, which means that the 2 hours we ask from the candidate is matched with 3-5 hours of our engineers' time to review and discuss with them.
- tchaffee 6y ago> collaborating on a problem with the interviewee That's not how developers spend the vast majority of their time. It's a poor signal in my experience. What it selects for is candidates who are immediately comfortable collaborating in real time with someone they have never met. That doesn't mean you aren't getting good candidates. What is means is that I spend far less than you finding good candidates because I also hire people who are great developers and who would fail at your interview.
- pythonbase 6y agoGreat approach. How do you shortlist candidates for the first step?
- 11thEarlOfMar 6y agoResumes & phone calls if they look interesting.
- nilkn 6y agoHow do you avoid awarding candidates who violate your request of no more than two hours and spend considerable time coming up with something amazing? They may still produce 100 lines of code — but those 100 lines may end up being unusually elegant, beautiful code that stumps 40 year veterans. How do you avoid punishing someone who took your requested time constraint seriously (or otherwise didn’t have enough free time to violate it anyway)? That’s not to say I don’t think this process has promise, but I’d need to be convinced that the above two issues can be fairly dealt with first.
- tziki 6y agoThis. I just can't see this not penalizing conscientious candidates.
- IggleSniggle 6y agoI'm of the opinion that this is fair enough as long as the pay isn't crazy high. The truth is, no interview is ever truly fair, and all interviews can be gamed. "Grinding leetcode" is much the same problem you are describing wrt spending 40 hours on a 2 hour project. People game non-technical interviews too. And resumes. It sucks, but you only need to be better perceived than the next best competitor. Most folks don't have 40 hours to spend on a 2 hour max coding challenge. Hopefully the ones that do are already employed and not currently competing with you, or are so inexperienced that 40 hours produces horrific over-engineered abominations instead of clean efficient readable code.
- wvenable 6y agoI reject entire the premise of this question. Anyone spending huge amounts of time is either going to produce way too much or they just took too long. There is nothing magical about programming where taking 8 hours to solve a 2 hour problem is going to stump 40 year veterans or be unusually beautiful. That's just unrealistic.
- dmos62 6y agoThat's not my experience at all. To a limited extent, you can get something more beautiful by just putting more work in it.
- frequentnapper 6y agoDamn, can I do one just for fun and then let's code review together? :p I recently had a humiliating experience. Just was approached on LinkedIn and at first they sent me a take home. I learned the front-end tech which was required for it, and then coded it. They just wanted me to fill out a doc but I put it all up on stackblitz. Another portion was back-end which I also put up on another online env so they could see not only the code but how everything was laid out. Get to the over the screen share interview - just nervous and both the interviewers were impatient, and just the atmosphere was so nerve-wracking - they did nothing to make it more relaxed. Just seemed like the senior guy was stressed out and I was wasting his time. My brain kind of shut off and the seemingly simple problem at that time just seemed insurmountable because I felt myself second guessing my every thought. Been developing for a number of years for a few companies, and I guess I haven't prepped for the tech interview - I thought my existing skills are what they need and should be enough to evaluate but boy was I wrong. Needless to say they didn't go with me. It's not like I needed the job, or probably wouldn't have taken it anyway since what their maximum range was lower than what I make, but the startup seemed quite interesting. Just such a bad experience.
- JauntyHatAngle 6y agoThat's annoying, but sounds like you avoided a bad team environment anyway? If they're not placating you at all than I doubt that'd be a great environment to work in. Or they'd already selected someone.
- ShamelessC 6y agoHow do you feel about paying candidates for the time they spend on the assignment?
- lukey_q 6y agoI created a similar task at my company as the parent commenter, building a small frontend to an existing API that can be completed in a couple of hours. There's no time limit for turning it in (we suggest two weeks but don't enforce it) and we specifically tell people that a good, concise solution to the problem at hand is perfectly fine, and even might be better than a sloppier solution that tries to add bells and whistles. When I set it up the goal was to avoid a lot of the issues I've personally had or heard about with coding challenges: they take way too long, they're too abstract, or you feel like the company might just be using you to get a couple of days of free work on their actual codebase. For this reason I feel fine about not paying people, as it's just a small part of the application process and not free work for us. I think it's worked pretty well so far. It happens after the initial phone screen and before the first technical interview. We send people the challenge, they return it whenever they want, and if we like it we set up a technical interview. Since the challenge uses our tech stack and is similar to the work we actually do, a large part of the first technical interview is discussing their solution. Why they chose certain patterns, why they added a certain library, how they'd consider testing it, why a certain function might be slow with 10k entities, and so on.
- Polylactic_acid 6y agoI once applied at a company that started with a programming test before any human communication. I think I did a pretty good job on the test. After that I was rejected when I assume they saw my resume. They gave me $50 store credit on their website (custom sticker printing). I assume it cost them a lot less than $50 but it was still pretty cool.
- justinlloyd 6y agoThis past week one of the companies on "who's hiring" sent me three challenges that "would only take 2 hours" before I ever got to talk to a human being or even got a human being responding with an email. This was a fully automated system apparently. "Take this programming test, we'll let you know if you're good enough to talk with a human." Whilst the challenges were not particularly complex, you also had to ensure that the person who was going to run your code could stand up your environment, so part of the second challenge was "thoroughly document how to run your code, the environment needed, etc, etc." Life is too short to deal with this kind of stonewalling.
- friendzis 6y ago> The tasks <...> come out of the designs we've implemented. I guess the tasks are something like "take this scaffolding and implement thing x". What would you think if the candidate responded saying the scaffolding does not allow to implement X properly without sacrificing things y and z, and instead of turning in code for review would show up for a discussion about the compromises and deficiencies in the scaffolding?
- cutemonster 6y ago> if the candidate responded saying Responded how, when? (At what point in the process)
- friendzis 6y ago> At what point in the process > > instead of turning in code for review I.e. something like instead of turning in code example they turn in a motivated analysis of why the scaffolding provided cannot support "proper" (and then analysis of so called proper) implementation of the thing asked, provide analysis for changes required in the scaffolding. Basically, the candidate would turn "provide and review code" process into "let's talk about architecture, implementation and tradeoffs". Would you discard such a candidate for not following the process because you expect seniority or would higher level thinking be a plus?
- tziki 6y agoWhy not time the programming task? Since it's supposed to take about two hours you could ask them to choose a two hour slot, email them the task at that time and have them return it two hours later. This wouldn't penalize people who choose to abide by the time guideline vs more dishonest candidates.
- ProZsolt 6y agoWhen I have limited amount of time even if it's more than enough I get stressed. When I'm on a job I have time to think about the problem, come up a rough solution in my head then start coding. If I have a hard limit I don't think that much because what if I don't have enough time to write it down? In the sort thinking phase I came up with a subpar architecture, but even if I have an extra half hour in the endI can't really change it or worst You cant finish it because I didn't think of an edge case.
- twic 6y agoIf your slot starts, and then something comes up - your elderly grandmother phones, your internet goes down, your roof springs a leak - that would be pretty annoying.
- cutemonster 6y agoWhat about visiting a coworking space and blocking your grandmother's number? Temporarily only
- genericlogic 6y agoA key point in your process is that you have a conversation before the homework phase. The last time I was job hunting I would be given homework without ever having a chance to talk to someone. I did that once, sent my work into an automated system and never heard anything back besides an automated thank you. I think if I was interviewing using your process I would submit code because I would guaranteed a chance to talk about it with other developers.
- ProZsolt 6y agoYes, if you aren't planing to invest in me. I won't do your homework. In this way you will only get desperate candidates.
- bryanrasmussen 6y ago>1. First interview, the candidate interviews us. What market we serve, what our development processes are, what our technology stack is. I somehow feel this rewards opinionated developers. Also right now I'm looking at a couple places because their recruiters called me up and said they were willing to meet my requirements which are basically pay - I don't care about most of these things, if they want me to do stuff and are willing to pay my rate and if they, after looking over my cv believe I can handle their stack, that's it. I don't really have any questions for them. So I guess you don't have people looking for people for you - you only have people coming in that think your company is special enough that they really really want to be there and are invested about finding out everything just to confirm that commitment on their part is good? on edit: this of course supposes that 90% of the companies out there are very similar and do not have any problems that are so interesting that I should feel prompted to ask. If I ever applied to the Internet Archive I'm sure I would have things I would like to ask.
- awhitby 6y agoAren't opinions something an employer should want in developers? And not just opinions on tech stack but all the way up to requirements etc. If I hire a developer "to build X," I would rather work with the developer who explains that X is the wrong thing and we should actually build Y, than the one who shrugs, asks no more questions, and builds a really excellent X - that later turns out to be the wrong thing to have built.
- swsieber 6y agoThere's a difference between opinionated and having an opinion. I think Rust is much more pleasant to work with (an opinion), but I'm not going to let it stop me from using other things (not opinionated)
- deleted 6y ago[deleted]
- jjk166 6y agoAt the interview stage, the candidate has been looking at your use case for at most a couple of hours with very little context. Assuming you've done any work on X and that you are competent, X is probably a good choice or at least there's good reason to believe it is a good choice. It might be nice if the candidate asks why you chose X over Y, but they shouldn't seriously propose that you should switch over to Y. Best case scenario they're assuming their little glimpse gives them a deeper understanding of the problem than you, which suggests they're cocky, worst case they may be deliberately selling you on a potentially worse solution that they just happen to be more comfortable with, ie they're inflexible. A good developer should wait until they know what they are talking about before they form an opinion on what would be the right tool for the job.
- gjm11 6y agoSomething seems badly off to me with the phrase "candidates who stumped 40 year veterans with elegant code". I don't want code that stumps anyone. I want code that delights 40-year veterans, for sure. But the signal of genuine elegance is "oh, that's a lovely way to do it" much more often than it's "how the hell does this work?".
- 11thEarlOfMar 6y agoIt was more of 'I didn't know the C language could do that'. The elegance was that what it did was obvious.
- quantified 6y agoMuch better thought out than any I have performed in on either side. Thank you for sharing, this would be a good template for others who need one to follow (which many do). You also hiring?
- GordonS 6y agoDo you have a few examples of tasks you assigned? I struggle to come up with tasks that I think have the right level of complexity, but which can also be completed in a reasonable amount of time.