5 ms·
There is a lot of discussion in the comments on this thread about take home tests not being useful. I believe we've have a pretty good hiring process that make
by rsyring 7y ago
There is a lot of discussion in the comments on this thread about take home tests not being useful. I believe we've have a pretty good hiring process that makes significant use of take home tests (16-18 hours of them in fact). At the same time, I assure you that we care deeply about making the best use of our and the applicants time. This is what we do to make that happen:
- We make a commitment to reply to every candidate who applies as instructed. Not every reply can be detailed, but candidates are never applying to a black hole.
- We have a very detailed job description that answers a lot of the questions many developers have about a company up front. There is always a chance that we are lying or don't live up to what we say, but the info is at least there so the applicants can make an informed decision about whether they are really interested in working for us. The details about our application process are also stated clearly up front so that candidates know what they are getting into.
- We ask pretty in-depth follow questions about technical experience to make sure it aligns with what we are looking for. This filters out a decent amount of candidates who would be unlikely to pass the rest of our process saving them and us time. Other than roughly validating that the candidates work experience falls within the range we are looking for, we find resumes to be mostly useless.
- The initial "take home" exercise we ask applicants to complete takes 60-90 minutes. It's not a coding exercise and they are given a document that helps them prep for the work they are going to be asked to do. We have found over time that this is more predictive of success in our organization that doing an initial phone screen. Candidates are given relatively detailed feedback on this exercise, usually within 3 business days of submitting their work.
- The next step is a Zoom interview. We have a few very basic coding tasks that we ask them to do as part of this interview. These tasks are representative of real world skills a developer would need and not in any way convoluted or academic. The environment could be challenging for some due to the fact that we are watching over their shoulder, but again the tasks are very basic and the goal is simply to see that the tasks are able to be finished in a reasonable time. This isn't to tell us if the candidate can do the job, it just helps us filter out candidates who are likely to fail badly on the next part of our interview process. It saves them time and us time and money. We have occasionally passed candidates on this step who seemed like their bad performance might have been from nervousness and, to date, those candidates have never performed well on the rest of our skills tests.
The candidate has an opportunity to ask any questions they would like during this interview.
We also talk about money at this stage. Our salary range and benefits are published on the job description so we ask the candidate what their expectations are from a compensation perspective. If they are reluctant to share details, that's fine, but we at least confirm that they understand that our published compensation is what we are actually planning on paying and ask that if that is not suitable to them, that they let us know that now before moving along in the process.
Only if everything is aligning at this stage, do we then ask them to commit to a significant more time working through our skills tests process. Our increasing "ask" of the candidate's time is deliberately progressive. We are trying to only ask more of them when we have had the chance to vet them for obvious mismatches and vice versa.
- We then move into a two part skills test process (16-18 hours). All candidates who move on to these tests are compensated at around $50 per hour. All tests are timed and represent real world tasks our developers do on a daily basis. The first three have to be scheduled and proctored but the fourth is really "take home" and can be done at the developers leisure.
If they pass all of those tests, the work on the fourth test goes through a code review process and is then reviewed with the candidate as part of their second video interview, in a very similar way to how we would do a code review for one of our projects.
- Finally, if all that goes well, we do an on-site interview which is mostly a work day (or two) with the project they are working on, again, very representative of the work our developers do on a daily basis. The costs of travel and lodging are compensated at this stage, but we are not paying them hourly. The work they do for us is not customer based work and does not generate revenue for us.
I can't say this is a perfect process and it is a lot to ask of an applicant. But, we also put a lot of time and money into this process for those who get to the later stages. So it's a "give, give" if you will.
I'm sure some will take issue with this process and some don't apply because they don't like the intensity or commitment involved. But I honestly can't fathom trying to decide on someone's technical ability by spending less time seeing how they handle actual programming tasks. So, we do ask a lot from applicants, but our skills tests work out ok specifically because:
- As much as possible, our process is designed to be "evidence based." We rely heavily on skills tests and evaluations that look at how the candidate does work that is very similar to the work we actually need them to do in the job.
- We work hard to provide the candidate with all the info they need from us to determine if we'd be a good fit for them before we ask for more than a few hours of their time
- All programming tasks are done on their own machine with the editor/tools of their choice (we do choose the language the skills tests are in which align with the technologies which are posted as required in the job descriptions)
- All programming tasks are highly representative of actual work a candidate would do if they were hired. The fact that they are timed is probably the one thing that is most "contrived" about the tests.
- The skills tests are paid at ~$50 per hour
- The grading of the candidates work is pretty straight forward. We have a rubric of sorts for what we are looking for and, to the best of our ability, that rubric aligns with the same things we'd be looking for during a code review of one of our employees that was working on a customer project. There are no trick questions and there are no academic or algorithmic oriented questions (because our day to day tasks don't usually involve those skills)
- While it's possible someone could get another developer to take the skills tests for them, the on-site work day(s) would likely reveal their deceit rather quickly. And, if not, we are a small organization and would figure it out pretty quickly if they made it all the way through and actually started working with us.
I'd love to have a simpler and less time consuming process for everyone's sake (it takes a lot of our time to administer this process too). But without the validation that comes with the candidate participating in these exercises for us, hiring becomes more of a guess than an evaluation. At least, I haven't figured out how to do it differently yet and the advice I read hear on HN and elsewhere about dev hiring hasn't convinced me of a better way to do it.
We do offer some candidates the option of skipping the on-site interview and instead work with us in a 30-90 day contract to hire situation. This is actually, probably, the best scenario for us but it obviously doesn't work for all candidates to leave a stable employment situation for such an offer.
- grafporno 7y agoWhat's the salary like to jump through all these hoops?
- howmuchaftertax 7y ago80-105k plus 1-2k "profit sharing" "bonus" for usa-only remote https://www.level12.io/careers/full-stack-web-app-developer-mid/ https://www.level12.io/careers/full-stack-web-app-developer-...
- hak8or 7y agoIs this really normal for web developer? I don't work in web dev, but I would absolutely not put up with this for only a 100k salary and getting paid roughly one grand for an interview process that takes in total days. Reason why is because I've had interviews at other companies which are far less interrogative from the company towards me. For example, for my current job (embedded systems) I had: - 30 minute phone interview with a technically competent hr rep - a quick very easy online quiz (multiple choice/few sentence answers) -one more 30 minute phone interview with a developer (open ended technical discussion) -onsite for roughly 2 to 3 hours with a dew developers, all very open ended technical discussions It was honestly the best hiring process I encountered. I didn't feel like I was even tested during the interviews, it was genuinely enjoyable technical conversations.
- rsyring 7y agoAs best I can tell, our process is not typical. But it's also not fundamentally a bad thing. It obviously doesn't interest you and I'm sure there are others who wouldn't apply for the same reason. But, there are others out there who appreciate a more thorough approach and might view the process you described as inadequate. Also, our salary range is higher for senior devs. I think what matters is whether the process being used by the organization is resulting in the hiring of developers that are a good match. I'm glad we are doing things this way even though it makes hiring take longer and likely filters out candidates who could do the work. It's an imperfect balancing act, one I'm always looking to improve. The main point of the post I made above was to show how IMO take home exams can be executed well, not to argue that everyone needs such an elaborate application process.