6 ms·

Studies [1] show that a work sample test is the best predictor of candidate performance on the job, which is why many software engineering teams use take-home t

Studies [1] show that a work sample test is the best predictor of candidate performance on the job, which is why many software engineering teams use take-home tests as one step in their hiring process. But designing an effective test is difficult and time-consuming. For example, candidates are reluctant to complete tests that are too long or not engaging enough. But make them too short and teams won’t get the signal they need for a proper evaluation. To encourage more thoughtful test design (and hopefully save future candidates from the worst offenders), my team compiled the largest library of non-“whiteboard” take-home tests that real engineering teams have used. You’ll find the challenges that Stripe and Microsoft gave to their full-stack candidates, front-end tests from Tailwind and Rivian, and back-end ones from Basecamp and Revolut. Whether you’re looking to evaluate an Android, DevOps, or Data Science candidate, a bootcamp grad, or senior engineer, we found a few options for each. Having built 20+ tests ourselves, we also rated the design of each test. The criteria for a 5-star rating: 1. Tests for skills highly relevant to those required for the position 2. Includes a well-written description of the prompt and even motivation for using a take-home test 3. Sets clear expectations for candidates (e.g. time requirements, evaluation criteria, submission details) 4. Asks for a reasonable time commitment from candidates (<4 hours) A few notes: - We found most of these test prompts in public GitHub repos, usually owned by the hiring team but occasionally in the candidate-owned submission. We sifted through hundreds of tests and filtered out those overly focused on algorithms (aka LeetCode), leaving us with 142 tests in the library. - The larger and more recognizable companies didn’t always have the best tests. Some of the most interesting prompts we found were from smaller teams (e.g. YC startups). This shouldn’t be surprising. Startups need to design candidate-friendly hiring experiences to compete for talent against more established players. - There were common themes among the tests we found. For example, front-end candidates were often given a Figma design + content feed to implement, while back-end candidates had to implement an API given a set of requirements. Data scientists were usually given a data set to clean, analyze, and submit a Jupyter notebook with their findings. - We’ll continue to update this library and add descriptions of each test so it’s easier to compare. Have feedback, or another take-home test we should add? We’d love to hear from you! [1] The Validity and Utility of Selection Methods in Personnel Psychology (https://www.researchgate.net/publication/232564809_The_Validity_and_Utility_of_Selection_Methods_in_Personnel_Psychology https://www.researchgate.net/publication/232564809_The_Valid...)

noisenotsignal 4y ago
While this certainly sounds like reasonable criteria for rating a take home, does this truly account for general candidate reluctance to do take homes due to the time commitment? Even 2-4 hours is more time than conventional Leetcode-style initial assessments, and many may rather use that time to interview with 2-4 companies instead of just one. I recently interviewed with Ramp and enjoyed their model of practical problems, like a take home, but condensed into just one hour with less of the end-to-end expectations of a true take home.
- ryandrake 4y ago
  If you are interviewing for 10-20 jobs in a particular week, a 2-4 hour take home test is a full time job 40 hours worth of work. I would accept a 2-4 hour time commitment but only much later in the interview process, when much of the interviewing was done and the list was whittled down to 2-3 serious companies.
  - lost953 4y ago
    Seriously, if a company wants me to spend more than an hour of my time on something for them, they better be including a check as well.
    - dymk 4y ago
      Less competition for everybody else looking at the position, then.
- mellavora 4y ago
  > Even 2-4 hours is more time than conventional Leetcode-style initial assessments, Hard disagree on this one. Unless your day-to-day work includes many leetcode style problems, you need to put in significant time training on leetcode if you want to pass the interview. You should be able to complete a take-home based on your current skill set. Yes, it may take 4-6 hours (or 8-10), and yes, that is a big ask of a candidate, but leetcode can take 5 to 10x prep time and you still can muff the interview.

noisenotsignal 4y ago

While this certainly sounds like reasonable criteria for rating a take home, does this truly account for general candidate reluctance to do take homes due to the time commitment? Even 2-4 hours is more time than conventional Leetcode-style initial assessments, and many may rather use that time to interview with 2-4 companies instead of just one. I recently interviewed with Ramp and enjoyed their model of practical problems, like a take home, but condensed into just one hour with less of the end-to-end expectations of a true take home.

ryandrake 4y ago
If you are interviewing for 10-20 jobs in a particular week, a 2-4 hour take home test is a full time job 40 hours worth of work. I would accept a 2-4 hour time commitment but only much later in the interview process, when much of the interviewing was done and the list was whittled down to 2-3 serious companies.
- lost953 4y ago
  Seriously, if a company wants me to spend more than an hour of my time on something for them, they better be including a check as well.
  - dymk 4y ago
    Less competition for everybody else looking at the position, then.
mellavora 4y ago
> Even 2-4 hours is more time than conventional Leetcode-style initial assessments, Hard disagree on this one. Unless your day-to-day work includes many leetcode style problems, you need to put in significant time training on leetcode if you want to pass the interview. You should be able to complete a take-home based on your current skill set. Yes, it may take 4-6 hours (or 8-10), and yes, that is a big ask of a candidate, but leetcode can take 5 to 10x prep time and you still can muff the interview.