3 ms·

A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review. I'm building a developer

A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review. I'm building a developer screens sharing app that, among other things, automates non-whiteboard interviews. Typically we can review a 60 minute take-home in 10 minutes, without cloning the repo. http://paircast.io http://paircast.io If you're an engineering manager that wants to move away from the leetcode challenges, please get in touch with me. ian@haxor.sh Also see http://they.whiteboarded.me/ http://they.whiteboarded.me/

dominotw 6y ago
> Typically we can review a 60 minute take-home in 10 minutes So a candidate a invests 60 minutes of their time and a multi million dollar company invests 10 minutes. We should all boycott these asymmetrical "take home" tests.
- sokoloff 6y ago
  This is an almost inextricably linked effect of allowing anyone who thinks (or wishes) they have the skills to compete for a $250K/yr job. If you put a grand prize out there, lots of people are motivated to go after it, including the ultra-long shots. On of my favorite interviews was for Acclaim in the mid-90s. Take home (download a zip file with a skeleton app, read the code, write the strategy element that’s stubbed out). I don’t recall how much time they allowed but it was at least 24 hours. Mine was “navigate an airplane from left of screen to right of screen against a fixed set of blocked squares[mountains]” Probably spent a little over an hour on it, enjoyed doing it, and remember it 25 years later. They can score such a test (at least initially) totally mechanically, weeding out the “couldn’t get anything to work” subset of the population in zero minutes of engineer time per new application. It tests something quite close to what you’d do as a game programmer for them. You can do it at home, at your own pace, in your normal environment. If you pass, then you might have to get an airplane and travel to interview, which is a multi-day disruption to your life. I’m on the employer side now; I see the value in asymmetric tests at the initial filtering stage for entry-level, because I don’t see any workable alternatives that don’t introduce other, likely worse, forms of credential-checking. I’m sure we miss some good candidates this way; false negatives are bad but less costly than false positives.
  - dominotw 6y ago
    > This is an almost inextricably linked effect of allowing anyone who thinks (or wishes) they have the skills to compete for a $250K/yr job. 250k+ jobs just have whiteboarding. Google, facebook, amazon ect don't have take homes. I've seen take homes mostly from smaller shops. Netflix might be an exception in the FAANGs with take homes > because I don’t see any workable alternatives that don’t introduce other, likely worse, forms of credential-checking. what do you think of compensating candidates for their time?
    - sokoloff 6y ago
      I don't see that the benefit exceeding the paperwork required to create and pay 0.75-1.5 hr contracts in the 51 different states plus District of Columbia plus many local taxing jurisdictions for just the United States, to say nothing of the hundreds of other countries that may each have their own requirements, filing requirements, or other traps. What do you think of asking applicants to compensate the company for the time spent reviewing their application? (Many colleges have application fees as an analog.) I would be opposed to this type of friction-introduction as well. Applicants cover their costs (including time) and company covers their costs (including time) seems to be working reasonably and gives both actors choice about how to balance their inputs and expected returns. If either side demands too much, the market moves away from them.
swagonomixxx 6y ago
> A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review. Literally this. You can pretty much do all of the interviews required for FANG without talking to a human being, which from the FANG perspective, is absolutely GREAT, because it takes all of the existing social biases (e.g preferring a certain race, preferring a certain gender, etc.) out of the window and focuses ONLY on the algorithmic knowledge of the applicant. Of course, there might be some interview here and there about soft questions, but for the most part, they don't really give a shit about that, and it's unlikely that you'd be rejected based on that, unless you _really_ suck at bull-shitting success/failure stories (which I am - can't stand doing it). I think the last time I applied to Amazon (over a year ago at this point) I didn't even get phone screened. It was just a link to 2 graph algorithm problems that were the equivalent of a medium difficultly on Leetcode, and I was immediately graded. It's so fricking easy for FANG nowadays.
novok 6y ago
The issue with take homes is they tend to be time unbounded for the interviewee, not the amount of time it take to review them AND there is still a whiteboard algo interview to do. When your doing whiteboard interviews the interviewer is there the entire time interacting with you, so it's taking up 1.5hrs of engineer time at least for every 1hr interacting with the interviewee. Reviewing a take home takes less time.
anon_tor_12345 6y ago
>A big reason we use leetcode style challenges is because they can be automatically scored but they're not? every screen i've been through has been on something that doesn't even compile the code (e.g. coderpad.io for FB or docs at G) and the interviewer then writes a summary afterwards (ala hiring packet at G) with ratings along some axes.
PragmaticPulp 6y ago
> A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review. Take-home problems are automatically scored as well. The company defines an API or interface to the code and creates a test suite to run against it. The benefit of take-home problems is that you have the option of reviewing the code in-depth if you need more perspective. If a candidate only passes 7/10 tests, I'll still dive into the code to see what went wrong. If it's an honest mistake, I ignore the failure and amend the instructions to make it more clear for the next candidate.

dominotw 6y ago

> Typically we can review a 60 minute take-home in 10 minutes So a candidate a invests 60 minutes of their time and a multi million dollar company invests 10 minutes. We should all boycott these asymmetrical "take home" tests.

sokoloff 6y ago
This is an almost inextricably linked effect of allowing anyone who thinks (or wishes) they have the skills to compete for a $250K/yr job. If you put a grand prize out there, lots of people are motivated to go after it, including the ultra-long shots. On of my favorite interviews was for Acclaim in the mid-90s. Take home (download a zip file with a skeleton app, read the code, write the strategy element that’s stubbed out). I don’t recall how much time they allowed but it was at least 24 hours. Mine was “navigate an airplane from left of screen to right of screen against a fixed set of blocked squares[mountains]” Probably spent a little over an hour on it, enjoyed doing it, and remember it 25 years later. They can score such a test (at least initially) totally mechanically, weeding out the “couldn’t get anything to work” subset of the population in zero minutes of engineer time per new application. It tests something quite close to what you’d do as a game programmer for them. You can do it at home, at your own pace, in your normal environment. If you pass, then you might have to get an airplane and travel to interview, which is a multi-day disruption to your life. I’m on the employer side now; I see the value in asymmetric tests at the initial filtering stage for entry-level, because I don’t see any workable alternatives that don’t introduce other, likely worse, forms of credential-checking. I’m sure we miss some good candidates this way; false negatives are bad but less costly than false positives.
- dominotw 6y ago
  > This is an almost inextricably linked effect of allowing anyone who thinks (or wishes) they have the skills to compete for a $250K/yr job. 250k+ jobs just have whiteboarding. Google, facebook, amazon ect don't have take homes. I've seen take homes mostly from smaller shops. Netflix might be an exception in the FAANGs with take homes > because I don’t see any workable alternatives that don’t introduce other, likely worse, forms of credential-checking. what do you think of compensating candidates for their time?
  - sokoloff 6y ago
    I don't see that the benefit exceeding the paperwork required to create and pay 0.75-1.5 hr contracts in the 51 different states plus District of Columbia plus many local taxing jurisdictions for just the United States, to say nothing of the hundreds of other countries that may each have their own requirements, filing requirements, or other traps. What do you think of asking applicants to compensate the company for the time spent reviewing their application? (Many colleges have application fees as an analog.) I would be opposed to this type of friction-introduction as well. Applicants cover their costs (including time) and company covers their costs (including time) seems to be working reasonably and gives both actors choice about how to balance their inputs and expected returns. If either side demands too much, the market moves away from them.

swagonomixxx 6y ago

> A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review. Literally this. You can pretty much do all of the interviews required for FANG without talking to a human being, which from the FANG perspective, is absolutely GREAT, because it takes all of the existing social biases (e.g preferring a certain race, preferring a certain gender, etc.) out of the window and focuses ONLY on the algorithmic knowledge of the applicant. Of course, there might be some interview here and there about soft questions, but for the most part, they don't really give a shit about that, and it's unlikely that you'd be rejected based on that, unless you _really_ suck at bull-shitting success/failure stories (which I am - can't stand doing it). I think the last time I applied to Amazon (over a year ago at this point) I didn't even get phone screened. It was just a link to 2 graph algorithm problems that were the equivalent of a medium difficultly on Leetcode, and I was immediately graded. It's so fricking easy for FANG nowadays.

novok 6y ago

The issue with take homes is they tend to be time unbounded for the interviewee, not the amount of time it take to review them AND there is still a whiteboard algo interview to do. When your doing whiteboard interviews the interviewer is there the entire time interacting with you, so it's taking up 1.5hrs of engineer time at least for every 1hr interacting with the interviewee. Reviewing a take home takes less time.

anon_tor_12345 6y ago

>A big reason we use leetcode style challenges is because they can be automatically scored but they're not? every screen i've been through has been on something that doesn't even compile the code (e.g. coderpad.io for FB or docs at G) and the interviewer then writes a summary afterwards (ala hiring packet at G) with ratings along some axes.

PragmaticPulp 6y ago

> A big reason we use leetcode style challenges is because they can be automatically scored, while take-homes take a lot more to review. Take-home problems are automatically scored as well. The company defines an API or interface to the code and creates a test suite to run against it. The benefit of take-home problems is that you have the option of reviewing the code in-depth if you need more perspective. If a candidate only passes 7/10 tests, I'll still dive into the code to see what went wrong. If it's an honest mistake, I ignore the failure and amend the instructions to make it more clear for the next candidate.