3 ms·

Giving people take home tests or leet code challenges is a backwards hiring process. Why as a manager are you wasting your time and the candidate's time "discov

Giving people take home tests or leet code challenges is a backwards hiring process. Why as a manager are you wasting your time and the candidate's time "discovering" if they can code when there's millions of developers who are actively working in the open on OSS platforms like GitHub? I don't give tests, I don't give technical interviews. I scout amazing developers and try to convince them to join my team. Not only do you get top notch people like that (who very much appreciate not being "tested"), you also save a ton of time. Your job as a manager is to proactively search for the best people possible to join your team, hire them and keep them as happy as humanly possible.

matthewmacleod 4y ago
You will certainly be able to find excellent developers that way. You'll also automatically rule out the vast majority of amazing developers that aren't "actively working in the open on OSS platforms".
- wootland 4y ago
  If I can quickly hire as many amazing candidates as needed using this method, then I don't really have a need to try to find the needle in the haystack that are the great devs in the global community of all devs using technical interviews. That's a very wasteful process.
PragmaticPulp 4y ago
> Why as a manager are you wasting your time and the candidate's time "discovering" if they can code when there's millions of developers who are actively working in the open on OSS platforms like GitHub? The number of developers who have something recent and meaningful on GitHub is extremely small. I’ve interviewed and hired a lot of great developers. Very few of them had much of anything on GitHub. Often the most I could find were one-line patches to projects somewhere. If you limit your hiring to GitHub OSS superstars, you’re going to miss out on a massive number of excellent developers.
- wootland 4y ago
  The truth is that I've never had a problem sourcing candidates this way for the past decade+. There are tens of millions of developers on GitHub. You will find people who are very accomplished in the exact area you need, and it's easy to track them down by navigating through the projects. Extra bonus points for hiring the maintainers of libraries/software you're actually using.
  - deleted 4y ago
    [deleted]
  - olliej 4y ago
    You will also exclude every person who works at a job that doesn’t allow that, and people who aren’t coding on weekends. The latter likely disproportionately impacts people with children, so that’s great as well.
    - wootland 4y ago
      I also hire like this with referrals from current (and past) team members. This helps provide an on-ramp for people with less of a public persona. If someone I've worked with in the past says "you should hire X, I worked with them at my last co and they're an amazing developer", I'll hire X. X tends to be excellent and you have a great chance of landing them with a hot referral.
- JohnFen 4y ago
  This is true. I have no code on GitHub or the like, personally. Although I have plenty of code that I can provide to a company if they want to see it. Some have, most haven't. I don't think it's an issue that some companies want to use GitHub as a shortcut to vetting developers. It does limit the pool they're fishing in, but that's what all vetting processes do.
jkubicek 4y ago
> there's millions of developers who are actively working in the open on OSS platforms like GitHub? There's an order of magnitude more qualified developers who don't have the time/aptitude/patience for contributing to OSS. You're shutting yourself off from the bulk of the hiring market by limiting candidates to OSS contributors.
- wootland 4y ago
  It's true that there are great devs that aren't doing OSS. There are so many devs doing OSS though that it's totally possible to hire exclusively this way. I know because I've done it for half a dozen teams, in all manner of different companies, to great success.
Mizoguchi 4y ago
You can learn a lot about a candidate, particularly those applying to senior positions, by looking at how they work under uncertainty and time/resource constraints, because ultimately that's how real life is. I have seen software engineers getting lost trying to solve a trivial problem because they like to always write code that scales well or showcase their knowledge of software design. Take home projects are great to learn about a candidate's decision making process given the use case and constraints of the final solution. IMO that would be hard to assess by looking at a repository. As far as having an entire team of top performers, IMO most companies don't need that. Where I work most of the stuff we do never gets released as a final product, we may work on a new feature for six months and then gets killed, because the business requirement changed, because we partnered with another company that already provides that functionality or because our VP changed his vision. Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them. We do have a few way above average coders leading the way, but most of our team is made of good coders, not stars. So far that's working great for us.
- wootland 4y ago
  > You can learn a lot about a candidate, particularly those applying to senior positions, by looking at how they work under uncertainty and time/resource constraints, because ultimately that's how real life is. Starting a project from scratch, shipping it and supporting it is quite a bit more difficult than regular company drudgery. I've never had an issue with someone hired via this method struggling to perform once in the door. > Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them. You should always pay a fair salary, but you'd be surprised that when you use this method you can hire some amazing people for "regular" pay. Respect goes a long way with top notch developers. A lot of the best people won't even do the leetcode dog and pony show, even if it results in very high pay. People will usually compensate for what they view as a toxic environment by demanding a higher salary (and quitting a couple of years in).
whack 4y ago
This is fascinating. What is your process for scouting amazing developers? Do you just go to random repositories, look at random commits, and identify people who are writing clean code?
- wootland 4y ago
  I start with anyone building with my company's OSS tools or contributing to them in a meaningful way. I also look at anyone working on libraries or OSS software that we're using in our projects. Beyond that I scout for people who launch projects from scratch (large or small) as it's a great way to see how people solve problems and how good their product sense is. Sometimes I'll just browse through the top charts on GH in the language we're using. Between GH, Twitter, HN and Reddit I see dozens of projects a day (more than I have time to investigate). One nice thing about doing this is it tends to snowball. Once you hire a high profile maintainer or several OSS enthusiasts, it makes it much easier to hire more. OSS developers will typically be skeptical of "business people", so it's best to either have an OSS presence as a manager or reach out through another OSS dev. I also stress that people can keep working on OSS in their free time (and keep ownership over their personal code). I go into the first meeting prepared to offer the person a job and also offer them the opportunity to interview anyone else at the company they may want to talk to. The results have been very positive, even if they don't take the job. They're usually flattered that you've done the research on their work and value it enough to offer them a job sight unseen. I would estimate that I have ~75% acceptance rate.
olliej 4y ago
A huge number of software engineers work for companies that don’t allow them to have side projects, so they don’t/can’t have code anywhere in public. Similarly for developers where software development is their job, they may simply not feel like continuing to work in their non-work time (or they may have a family they would like to spend time with, etc). But similarly take home tests aren’t going to give you anything interviews aren’t going to show - if you’re concerned about people lying about their competence you should assume that they’re able to pay someone to do the work for them anyway. After all the threat model for a take home test is dishonesty. I really think a lot of people think the goal of an interview is just to determine “can this person do the mechanical part of writing code” which is the easy bit.

matthewmacleod 4y ago

You will certainly be able to find excellent developers that way. You'll also automatically rule out the vast majority of amazing developers that aren't "actively working in the open on OSS platforms".

wootland 4y ago
If I can quickly hire as many amazing candidates as needed using this method, then I don't really have a need to try to find the needle in the haystack that are the great devs in the global community of all devs using technical interviews. That's a very wasteful process.

PragmaticPulp 4y ago

> Why as a manager are you wasting your time and the candidate's time "discovering" if they can code when there's millions of developers who are actively working in the open on OSS platforms like GitHub? The number of developers who have something recent and meaningful on GitHub is extremely small. I’ve interviewed and hired a lot of great developers. Very few of them had much of anything on GitHub. Often the most I could find were one-line patches to projects somewhere. If you limit your hiring to GitHub OSS superstars, you’re going to miss out on a massive number of excellent developers.

wootland 4y ago
The truth is that I've never had a problem sourcing candidates this way for the past decade+. There are tens of millions of developers on GitHub. You will find people who are very accomplished in the exact area you need, and it's easy to track them down by navigating through the projects. Extra bonus points for hiring the maintainers of libraries/software you're actually using.
- deleted 4y ago
  [deleted]
- olliej 4y ago
  You will also exclude every person who works at a job that doesn’t allow that, and people who aren’t coding on weekends. The latter likely disproportionately impacts people with children, so that’s great as well.
  - wootland 4y ago
    I also hire like this with referrals from current (and past) team members. This helps provide an on-ramp for people with less of a public persona. If someone I've worked with in the past says "you should hire X, I worked with them at my last co and they're an amazing developer", I'll hire X. X tends to be excellent and you have a great chance of landing them with a hot referral.
JohnFen 4y ago
This is true. I have no code on GitHub or the like, personally. Although I have plenty of code that I can provide to a company if they want to see it. Some have, most haven't. I don't think it's an issue that some companies want to use GitHub as a shortcut to vetting developers. It does limit the pool they're fishing in, but that's what all vetting processes do.

jkubicek 4y ago

> there's millions of developers who are actively working in the open on OSS platforms like GitHub? There's an order of magnitude more qualified developers who don't have the time/aptitude/patience for contributing to OSS. You're shutting yourself off from the bulk of the hiring market by limiting candidates to OSS contributors.

wootland 4y ago
It's true that there are great devs that aren't doing OSS. There are so many devs doing OSS though that it's totally possible to hire exclusively this way. I know because I've done it for half a dozen teams, in all manner of different companies, to great success.

Mizoguchi 4y ago

You can learn a lot about a candidate, particularly those applying to senior positions, by looking at how they work under uncertainty and time/resource constraints, because ultimately that's how real life is. I have seen software engineers getting lost trying to solve a trivial problem because they like to always write code that scales well or showcase their knowledge of software design. Take home projects are great to learn about a candidate's decision making process given the use case and constraints of the final solution. IMO that would be hard to assess by looking at a repository. As far as having an entire team of top performers, IMO most companies don't need that. Where I work most of the stuff we do never gets released as a final product, we may work on a new feature for six months and then gets killed, because the business requirement changed, because we partnered with another company that already provides that functionality or because our VP changed his vision. Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them. We do have a few way above average coders leading the way, but most of our team is made of good coders, not stars. So far that's working great for us.

wootland 4y ago
> You can learn a lot about a candidate, particularly those applying to senior positions, by looking at how they work under uncertainty and time/resource constraints, because ultimately that's how real life is. Starting a project from scratch, shipping it and supporting it is quite a bit more difficult than regular company drudgery. I've never had an issue with someone hired via this method struggling to perform once in the door. > Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them. You should always pay a fair salary, but you'd be surprised that when you use this method you can hire some amazing people for "regular" pay. Respect goes a long way with top notch developers. A lot of the best people won't even do the leetcode dog and pony show, even if it results in very high pay. People will usually compensate for what they view as a toxic environment by demanding a higher salary (and quitting a couple of years in).

whack 4y ago

This is fascinating. What is your process for scouting amazing developers? Do you just go to random repositories, look at random commits, and identify people who are writing clean code?

wootland 4y ago
I start with anyone building with my company's OSS tools or contributing to them in a meaningful way. I also look at anyone working on libraries or OSS software that we're using in our projects. Beyond that I scout for people who launch projects from scratch (large or small) as it's a great way to see how people solve problems and how good their product sense is. Sometimes I'll just browse through the top charts on GH in the language we're using. Between GH, Twitter, HN and Reddit I see dozens of projects a day (more than I have time to investigate). One nice thing about doing this is it tends to snowball. Once you hire a high profile maintainer or several OSS enthusiasts, it makes it much easier to hire more. OSS developers will typically be skeptical of "business people", so it's best to either have an OSS presence as a manager or reach out through another OSS dev. I also stress that people can keep working on OSS in their free time (and keep ownership over their personal code). I go into the first meeting prepared to offer the person a job and also offer them the opportunity to interview anyone else at the company they may want to talk to. The results have been very positive, even if they don't take the job. They're usually flattered that you've done the research on their work and value it enough to offer them a job sight unseen. I would estimate that I have ~75% acceptance rate.

olliej 4y ago

A huge number of software engineers work for companies that don’t allow them to have side projects, so they don’t/can’t have code anywhere in public. Similarly for developers where software development is their job, they may simply not feel like continuing to work in their non-work time (or they may have a family they would like to spend time with, etc). But similarly take home tests aren’t going to give you anything interviews aren’t going to show - if you’re concerned about people lying about their competence you should assume that they’re able to pay someone to do the work for them anyway. After all the threat model for a take home test is dishonesty. I really think a lot of people think the goal of an interview is just to determine “can this person do the mechanical part of writing code” which is the easy bit.