11 ms·

Taking that a step further, we often actively discourage looking at OSS contributions during resume review for the same reason we don't offer take home intervie

Taking that a step further, we often actively discourage looking at OSS contributions during resume review for the same reason we don't offer take home interview assignments: it's biased against people who don't have a whole lot of extra time at home. When we have done either of the above, the singles who work part time have a bunch of time to perfect their work suddenly have a lot to show over the single parents who may be working full time or more. I say "often" because OSS contributions can still be an indicator of something, but it's not really clear what. Maybe it indicates drive to contribute to OSS, maybe technical ability, maybe no hobbies or commitments outside their day job. In our experience it's often the latter, but even so, it's biased against people who don't have the time to contribute even if they desired to do so. So we typically just stick with the resume for actual experience and college coursework, if any, but not the college itself. Using these heuristics we've managed to build a pretty strong pipeline of people with all backgrounds of education or experience.

Uberphallus 6y ago
Thank you! When I was single I had a lot of free time to tinker and go through coding tests, etc, etc, and cater to whatever hiring shenanigans were in place. First marriage, then a kid, and I find myself grabbing my laptop after work maybe once a fortnight. I'm always "open for new challenges" and I regularly apply to positions in interesting (to me) projects, but more often than not, at some point in their process they gimme some "take home assignments" that are a literal week of unpaid work. I've heard of companies that pay for those assignments, but I've never stumbled upon one. I always drop out of that, thinking, "good luck with that particular choice of candidate sampling". I might not have been the best candidate, but most seniors I talk to are turned off by these things as well. Timed coding tests are fine, up to 2-3h, I can squeeze one of those in most weeks. But, and I'm not making it up, "implement this subset of the MQTT spec in your language of choice" as just a step in the hiring process? Hell nah.
- austincheney 6y ago
  So I am married with kids and I have two separate careers in unrelated industries to balance, only one of which is full time though. Still I find time to spend with the wife and kids and I still contribute daily to open source. It is all about budgeting and balance. What are you willing to sacrifice. I don't have social time outside the family and I don't watch much television unless I am traveling away from the family. I balance open source against things like gardening and house maintenance but gardening and house maintenance only take so much time. The biggest killers to my open source contributions are general life soul sucking killers. For me the worst is long driving commutes to an office. I can feel my soul bleeding away.
  - notsuoh 6y ago
    That's wonderful! You sound like a very driven person to be able to do all of that. Of course, your experience and ability shouldn't be considered the norm with everyone or expected of anyone, especially because with finite time in the day and differing situations, we must strive to judge everyone on the same plane.
    - austincheney 6y ago
      > Of course, your experience and ability shouldn't be considered the norm It is the norm in my line of work. I mean in my other line of work that isn't a software development.
      - notsuoh 6y ago
        Oh sure, sorry I should have clarified, I was talking about software development. I thought that was implied, but happy to be explicit.
        austincheney 6y ago
        I also find it strange how some people are so incredibly emotional about this subject like being punched in the face or having their car stolen at gun point. I like programming as hobby so I choose to program outside of work. By no means is that sentiment meant to suggest any form of hostility. There isn’t even any competition implied.
  - bcrosby95 6y ago
    I doubt I would be able to juggle 2 jobs with my current pandemic induced schedule. Basically, my time is spoken for during the week from 630am until 8pm. And that gets me 6 hours of work. I need to find 10 hours outside that time to get myself up to a 40 hour workweek. I'm pretty lucky in that my wife is a stay at home mom. My sister in law has a dual income family. They basically never see eachother, because one has to work during the day during the week, and the other works nights and all weekend. They're lucky one of their jobs is so flexible.
- pydry 6y ago
  I wrote open source initially because I wanted to skip those assignments but still be fair to the hirers and because it took less time and was WAY more fun than take home tests. What I discovered was that they'd be willing to make me work 5+ hours on their assignment but wouldn't spend 5 minutes reading my code. Though it wasn't my intention it inadvertently gave me a quick and easy way to filter companies which actively disrespect candidates' time.
  - notsuoh 6y ago
    It takes more than five minutes to review code, especially when it's of a volume you describe. You're also really trying to replace the metrics against which a company measures candidates and insert your own metric and then complaining that they don't use your metric. I wouldn't expect you to use my metric when interviewing me for your company, so don't think it's fair to try to insert your own metric for others (though you're welcome to vote with your feet).
    - pydry 6y ago
      It takes more than 5 minutes to thoroughly review code but you can learn a lot from only spending 5 minutes. Ignoring it completely also sends a signal to the candidate. IME this red flag is usually paired with 5 or 6 other red flags.
deergomoo 6y ago
We're having a bit of a debate about this internally as the company I work for is struggling to hire good candidates. For a while we've had a take-home test and that has increased our success rate somewhat. But as you say, we really don't want to lay unreasonable expectations on people who may have other obligations, but we've had many people in the past who've interviewed very well but turned out to be completely incompetent when assigned to a real project. Maybe we're just really bad at interviews? It probably doesn't help that management here is almost entirely non-technical. There's usually a developer or two sitting in on the interview to try and balance it out, but I don't think any of us would consider ourselves particularly good at interviewing people.
- notsureaboutpg 6y ago
  >For a while we've had a take-home test and that has increased our success rate somewhat. But as you say, we really don't want to lay unreasonable expectations on people who may have other obligations These are mutually exclusive things. Candidates who have more free time to code are more likely to be better at it than those who don't (all else equal). Candidates who have OSS maintenance / leadership experience are more likely to work well in teams (all else equal). If you choose not to weight those things, to balance the playing field for people who have more obligations outside of work, then you'll also have lesser quality candidates (again, generally). So if you're struggling to hire good candidates, maybe it's a good idea to weight these things (and bias against people with less free time outside work). Once you have good candidates, and this is no longer a pressure on your business, then it might be a good time to try and balance the playing field for new hires. But you cannot balance the playing field and also hire the best candidates. The best candidates will have unfair advantages in general. You can either lean towards having the best or lean towards balancing the playing field.
  - sokoloff 6y ago
    We should try to eliminate irrelevant biases in the hiring process but we are fundamentally trying to select people to hire who will join our company and write good code. That quality is not evenly distributed in the population. Some of the gymnastics thinking that I see seems to suggest that we’d seek to hire fluent English speakers by interviewing evenly across all populations. By all means, I’d be more than happy to hire someone fluent in English from China, but if I’m looking for a fluent English speaker, I shouldn’t spent 18.5% of my recruiting efforts/budget in China out of "fairness".
PragmaticPulp 6y ago
I prefer to look at everything the candidate has to offer. Don’t penalize people for not having OSS contributions, of course, but it doesn’t make sense to ignore them. Whatever credentials the candidate brings to the table should be taken into consideration. Not everyone has the opportunity to go to colleges or get a first job at a well-known company. If someone chooses to prove themselves via OSS contributions, let them.
pydry 6y ago
>I say "often" because OSS contributions can still be an indicator of something, but it's not really clear what. It's a fairly clear signal of skill quality and attitude. Reading open source commits/PRs and issue trackers tells you quite a lot about a developer which you can't see without some sort of a test (often not even then). >it's biased against people who don't have the time Surely any career that requires a high level of skill and practice honing that skill is biased against people who don't have the time? What's special about open source?
- bauerd 6y ago
  It's not about merit but about bias
  - californical 6y ago
    You generally become better with more practice though, right? Whole 10,000 hours thing? Surely someone who spends more time developing will be better at it. Now of course this doesn’t matter past a certain point — people could spend all their time working on something that doesn’t help them grow. Then once you have those 10,000 hours of actual growth, you are probably close to a Senior developer level. Which after 5 years in an office job working 40 hour weeks, you’re there anyways. But at the earlier levels of developers, it seems that working on projects outside of work would definitely help you improve faster!
    - notsuoh 6y ago
      I trust that all that practice would make someone interview really well, no need to look at how much practice they've had as a metric.
    - watwut 6y ago
      > You generally become better with more practice though, right? Whole 10,000 hours thing? Except that 10000 hours thing is non scientific nonsense. To improve, you have to practice right way. If you code for 8 hours at work, coding 2 further hours wont make you improve more. Similarly, if you want to improve in music or running, just playing songs or jogging means you will hit plateau pretty fast. After that, you have to train on correct selection of exercises. At that point reading some theory will have much bigger impact, because yoi are doing something new. And even exercising will likely make you improve more due to what it does to body then further 2 hours of the same.
WalterSear 6y ago
> for the same reason we don't offer take home interview assignments: it's biased against people who don't have a whole lot of extra time at home. This is just another single dimension hiring credential, that will result in limiting your hiring pool to people like yourself. My code ran on 70+ million machines last month, but I've come to decline any timed or proctored technical interviews. It's not that I'm too good for whiteboarding or timed tests, or that my options are so open that there isn't significant cost in doing so - quite the opposite: I'm come to find the process so traumatic that going through with it isn't worth it for anyone involved: those jobs just aren't open to people like me.
- notsuoh 6y ago
  We look at CV and then have an interview process. We don't do proctored timed interview coding questions in the usual sense, though we may walk through code. I understand the reluctance of a senior engineer such as yourself to go though any interview process, but to be honest I've interviewed plenty of engineers with decades of experience and many have completely fallen flat. Interviews aren't just about technical knowledge, but also to make sure people will get along and are reasonable to work with. I think we'll agree that there has to be _some_ interview process. iirc node's Left Pad package is downloaded like 20M/mo.
  - WalterSear 6y ago
    Yes, that's very different, and much more reasonable than most places that don't do take home technical examinations. There's obviously a fundamental necessity to evaluate a candidate's technical skills directly, rather than relying on credentials - my issue is with the false expedience and conflationism of timed and artificially performative technical evaluations, and my personal difficulties with the social requirements inherent to them.
- iguanayou 6y ago
  Agree
908B64B197 6y ago
> we often actively discourage looking at OSS contributions during resume review for the same reason we don't offer take home interview assignments: it's biased against people who don't have a whole lot of extra time at home. When we have done either of the above, the singles who work part time have a bunch of time to perfect their work suddenly have a lot to show over the single parents who may be working full time or more. Or their company ships a product that has a huge dependency on that particular OSS project, so they are doing the work on company time.
polishdude20 6y ago
How do you interview people who are changing industries then? The only way a cook or a librarian can get out of that and into software could very well be side projects and open source contributions. With your heuristics, you'd only consider their cooking experience and say "well that's not software" and pass.
- notsuoh 6y ago
  Specifically I don't hire entry level people, so that hasn't been an issue. But if I did, I would likely look at projects and such, general aptitude, etc. because that's all there is.
keb_ 6y ago
This is bad for applicants who may be at dead-end jobs that don't provide much meaningful development experience, so instead choose to self-learn and/or contribute to OSS to fill the void. Not saying your strategy is bad, just that you might miss some good candidates.

Uberphallus 6y ago

Thank you! When I was single I had a lot of free time to tinker and go through coding tests, etc, etc, and cater to whatever hiring shenanigans were in place. First marriage, then a kid, and I find myself grabbing my laptop after work maybe once a fortnight. I'm always "open for new challenges" and I regularly apply to positions in interesting (to me) projects, but more often than not, at some point in their process they gimme some "take home assignments" that are a literal week of unpaid work. I've heard of companies that pay for those assignments, but I've never stumbled upon one. I always drop out of that, thinking, "good luck with that particular choice of candidate sampling". I might not have been the best candidate, but most seniors I talk to are turned off by these things as well. Timed coding tests are fine, up to 2-3h, I can squeeze one of those in most weeks. But, and I'm not making it up, "implement this subset of the MQTT spec in your language of choice" as just a step in the hiring process? Hell nah.

austincheney 6y ago
So I am married with kids and I have two separate careers in unrelated industries to balance, only one of which is full time though. Still I find time to spend with the wife and kids and I still contribute daily to open source. It is all about budgeting and balance. What are you willing to sacrifice. I don't have social time outside the family and I don't watch much television unless I am traveling away from the family. I balance open source against things like gardening and house maintenance but gardening and house maintenance only take so much time. The biggest killers to my open source contributions are general life soul sucking killers. For me the worst is long driving commutes to an office. I can feel my soul bleeding away.
- notsuoh 6y ago
  That's wonderful! You sound like a very driven person to be able to do all of that. Of course, your experience and ability shouldn't be considered the norm with everyone or expected of anyone, especially because with finite time in the day and differing situations, we must strive to judge everyone on the same plane.
  - austincheney 6y ago
    > Of course, your experience and ability shouldn't be considered the norm It is the norm in my line of work. I mean in my other line of work that isn't a software development.
    - notsuoh 6y ago
      Oh sure, sorry I should have clarified, I was talking about software development. I thought that was implied, but happy to be explicit.
      - austincheney 6y ago
        I also find it strange how some people are so incredibly emotional about this subject like being punched in the face or having their car stolen at gun point. I like programming as hobby so I choose to program outside of work. By no means is that sentiment meant to suggest any form of hostility. There isn’t even any competition implied.
- bcrosby95 6y ago
  I doubt I would be able to juggle 2 jobs with my current pandemic induced schedule. Basically, my time is spoken for during the week from 630am until 8pm. And that gets me 6 hours of work. I need to find 10 hours outside that time to get myself up to a 40 hour workweek. I'm pretty lucky in that my wife is a stay at home mom. My sister in law has a dual income family. They basically never see eachother, because one has to work during the day during the week, and the other works nights and all weekend. They're lucky one of their jobs is so flexible.
pydry 6y ago
I wrote open source initially because I wanted to skip those assignments but still be fair to the hirers and because it took less time and was WAY more fun than take home tests. What I discovered was that they'd be willing to make me work 5+ hours on their assignment but wouldn't spend 5 minutes reading my code. Though it wasn't my intention it inadvertently gave me a quick and easy way to filter companies which actively disrespect candidates' time.
- notsuoh 6y ago
  It takes more than five minutes to review code, especially when it's of a volume you describe. You're also really trying to replace the metrics against which a company measures candidates and insert your own metric and then complaining that they don't use your metric. I wouldn't expect you to use my metric when interviewing me for your company, so don't think it's fair to try to insert your own metric for others (though you're welcome to vote with your feet).
  - pydry 6y ago
    It takes more than 5 minutes to thoroughly review code but you can learn a lot from only spending 5 minutes. Ignoring it completely also sends a signal to the candidate. IME this red flag is usually paired with 5 or 6 other red flags.

deergomoo 6y ago

We're having a bit of a debate about this internally as the company I work for is struggling to hire good candidates. For a while we've had a take-home test and that has increased our success rate somewhat. But as you say, we really don't want to lay unreasonable expectations on people who may have other obligations, but we've had many people in the past who've interviewed very well but turned out to be completely incompetent when assigned to a real project. Maybe we're just really bad at interviews? It probably doesn't help that management here is almost entirely non-technical. There's usually a developer or two sitting in on the interview to try and balance it out, but I don't think any of us would consider ourselves particularly good at interviewing people.

notsureaboutpg 6y ago
>For a while we've had a take-home test and that has increased our success rate somewhat. But as you say, we really don't want to lay unreasonable expectations on people who may have other obligations These are mutually exclusive things. Candidates who have more free time to code are more likely to be better at it than those who don't (all else equal). Candidates who have OSS maintenance / leadership experience are more likely to work well in teams (all else equal). If you choose not to weight those things, to balance the playing field for people who have more obligations outside of work, then you'll also have lesser quality candidates (again, generally). So if you're struggling to hire good candidates, maybe it's a good idea to weight these things (and bias against people with less free time outside work). Once you have good candidates, and this is no longer a pressure on your business, then it might be a good time to try and balance the playing field for new hires. But you cannot balance the playing field and also hire the best candidates. The best candidates will have unfair advantages in general. You can either lean towards having the best or lean towards balancing the playing field.
- sokoloff 6y ago
  We should try to eliminate irrelevant biases in the hiring process but we are fundamentally trying to select people to hire who will join our company and write good code. That quality is not evenly distributed in the population. Some of the gymnastics thinking that I see seems to suggest that we’d seek to hire fluent English speakers by interviewing evenly across all populations. By all means, I’d be more than happy to hire someone fluent in English from China, but if I’m looking for a fluent English speaker, I shouldn’t spent 18.5% of my recruiting efforts/budget in China out of "fairness".

PragmaticPulp 6y ago

I prefer to look at everything the candidate has to offer. Don’t penalize people for not having OSS contributions, of course, but it doesn’t make sense to ignore them. Whatever credentials the candidate brings to the table should be taken into consideration. Not everyone has the opportunity to go to colleges or get a first job at a well-known company. If someone chooses to prove themselves via OSS contributions, let them.

pydry 6y ago

>I say "often" because OSS contributions can still be an indicator of something, but it's not really clear what. It's a fairly clear signal of skill quality and attitude. Reading open source commits/PRs and issue trackers tells you quite a lot about a developer which you can't see without some sort of a test (often not even then). >it's biased against people who don't have the time Surely any career that requires a high level of skill and practice honing that skill is biased against people who don't have the time? What's special about open source?

bauerd 6y ago
It's not about merit but about bias
- californical 6y ago
  You generally become better with more practice though, right? Whole 10,000 hours thing? Surely someone who spends more time developing will be better at it. Now of course this doesn’t matter past a certain point — people could spend all their time working on something that doesn’t help them grow. Then once you have those 10,000 hours of actual growth, you are probably close to a Senior developer level. Which after 5 years in an office job working 40 hour weeks, you’re there anyways. But at the earlier levels of developers, it seems that working on projects outside of work would definitely help you improve faster!
  - notsuoh 6y ago
    I trust that all that practice would make someone interview really well, no need to look at how much practice they've had as a metric.
  - watwut 6y ago
    > You generally become better with more practice though, right? Whole 10,000 hours thing? Except that 10000 hours thing is non scientific nonsense. To improve, you have to practice right way. If you code for 8 hours at work, coding 2 further hours wont make you improve more. Similarly, if you want to improve in music or running, just playing songs or jogging means you will hit plateau pretty fast. After that, you have to train on correct selection of exercises. At that point reading some theory will have much bigger impact, because yoi are doing something new. And even exercising will likely make you improve more due to what it does to body then further 2 hours of the same.

WalterSear 6y ago

> for the same reason we don't offer take home interview assignments: it's biased against people who don't have a whole lot of extra time at home. This is just another single dimension hiring credential, that will result in limiting your hiring pool to people like yourself. My code ran on 70+ million machines last month, but I've come to decline any timed or proctored technical interviews. It's not that I'm too good for whiteboarding or timed tests, or that my options are so open that there isn't significant cost in doing so - quite the opposite: I'm come to find the process so traumatic that going through with it isn't worth it for anyone involved: those jobs just aren't open to people like me.

notsuoh 6y ago
We look at CV and then have an interview process. We don't do proctored timed interview coding questions in the usual sense, though we may walk through code. I understand the reluctance of a senior engineer such as yourself to go though any interview process, but to be honest I've interviewed plenty of engineers with decades of experience and many have completely fallen flat. Interviews aren't just about technical knowledge, but also to make sure people will get along and are reasonable to work with. I think we'll agree that there has to be _some_ interview process. iirc node's Left Pad package is downloaded like 20M/mo.
- WalterSear 6y ago
  Yes, that's very different, and much more reasonable than most places that don't do take home technical examinations. There's obviously a fundamental necessity to evaluate a candidate's technical skills directly, rather than relying on credentials - my issue is with the false expedience and conflationism of timed and artificially performative technical evaluations, and my personal difficulties with the social requirements inherent to them.
iguanayou 6y ago
Agree

908B64B197 6y ago

> we often actively discourage looking at OSS contributions during resume review for the same reason we don't offer take home interview assignments: it's biased against people who don't have a whole lot of extra time at home. When we have done either of the above, the singles who work part time have a bunch of time to perfect their work suddenly have a lot to show over the single parents who may be working full time or more. Or their company ships a product that has a huge dependency on that particular OSS project, so they are doing the work on company time.

polishdude20 6y ago

How do you interview people who are changing industries then? The only way a cook or a librarian can get out of that and into software could very well be side projects and open source contributions. With your heuristics, you'd only consider their cooking experience and say "well that's not software" and pass.

notsuoh 6y ago
Specifically I don't hire entry level people, so that hasn't been an issue. But if I did, I would likely look at projects and such, general aptitude, etc. because that's all there is.

keb_ 6y ago

This is bad for applicants who may be at dead-end jobs that don't provide much meaningful development experience, so instead choose to self-learn and/or contribute to OSS to fill the void. Not saying your strategy is bad, just that you might miss some good candidates.