4 ms·

This conclusively tells us that the Leetcode grind has been (without any dispute) been gamed to the ground and is no longer an accurate measure of exceptional p

by rvz 3y ago

This conclusively tells us that the Leetcode grind has been (without any dispute) been gamed to the ground and is no longer an accurate measure of exceptional performance in the role. Even the interviewers would struggle with the questions themselves. Why waste each other's time in the interview when I (if I was the interviewer) can just ask for relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool. I don't need to test you if you have already made significant contributions in the open. Easy assumptions can be made with the very least: * Has knowledge of Git. * Knows how to code in X language in a large project. * Has done code reviews on other people's code. * Is able to maintain a sophisticated project with external contributors. Everything else beyond that is secondary or optional and it's a very efficient evaluation and hard to fake. When there are too many candidates in the pipeline, Leetcoding them all is a waste of everyone's time. Overall leetcode optimizes to be gamed and is now a solved problem by ChatGPT.

randmeerkat 3y ago
> …can just ask for relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool. Get your hiring done now while you can, when the economy rebounds you won’t be able to hire anyone. Also give your team a raise, because they’ll probably be the first to go once new options open up.
bpye 3y ago
Not everyone works in the open. I do have open source side projects and contributions I’ve made on my own time - but almost everything I’ve done at work is closed source.
leononame 3y ago
> can just ask for relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool Not everyone spends their free time contributing (to major nonetheless) to open source projects. There are a lot of great engineers that have enough work on their desks with their day job and there are also plenty of idiots in open source. Asking for relevant projects or asking for GitHub profiles to gauge relevant projects yourself is what people were already doing years ago and it wasn't a great hiring strategy. Turns out judging a software engineers skills is extremely hard.
- angarg12 3y ago
  This. Focusing your hiring on open source contributions biases the process and misses huge slices of the software engineering population. I made the best work of my life (by a long long shot) to private companies closed source.
- thwarted 3y ago
  Not everyone spends their free time contributing (to major nonetheless) to open source projects. and elsewhere in these comments, we see: I would expect candidates for programming jobs to demonstrate first class ChatGPT or other code copilot skills. Not everyone spends their free time learning first class ChatGPT or code copilot skills. It's interesting that this age old mantra about open source contributions being inappropriate for a hiring manager to expect because of what people do or do not do in their free time now does not apply to random other skill/experience that one must also acquire in their free time. If a company has, for example, "git experience required" on the job posting you'll either need to actively demonstrate that you know how to use git or passively demonstrate by having a on-line accessible corpus of git-related work that shows your experience. You don't get a pass on that requirement just because the companies you've been working at don't use git and, well, you don't want to spend your free time learning git. And it is appropriate for the hiring manager to list that as a requirement despite claims that they'll be disqualifying some percentage of the candidates by including that as a requirement.
  - mythhabit 3y ago
    I think part of the issue is also the complete lack of thrust. If you state that a candidate must know git, and the candidate say they do know git for basic committing, branching, merging ect, then fine - let's move on. I've learned that prior work history, talk and basic problem solving is in no way indicative of performance at all. I've found that the only guaranteed process to find excellent candidates, is hiring interns and part-timers currently finishing education, and pick the ones with the right mindset and intelligence.
- zelphirkalt 3y ago
  That is a fair view. Of course there is also a tendency of great engineers doing project in their free time, since they love what they do. There is also the fact that there is a higher chance of someone having learned a lot more, if they continuously worked on projects in their free time. There is also a higher chance to find people, who are not just into software development jobs, because they pay well, but because this is what they want to do. So for hiring engineers, I can understand when they hire with a certain bias. Sure, no one should be expecting us to do extra rounds, and sure, one can be a great engineer even without extra rounds, but the tendency is for that to take more years without that drive to explore in ones free time. And that's OK! I think it is fair though, if a company wants to hire a more driven/curious/exploring person.
vkou 3y ago
If you want to eliminate 90% of candidates in a pool, a simpler solution is to take your stack of resumes, and shred the top 90% of them.
wkirby 3y ago
> is no longer an accurate measure of exceptional performance in the role It never was. No real-world job performance has ever been accurately measured by solving leetcode puzzles for one simple reason: problem solving is only ever going to be about 50% of your performance, and these puzzles don't address collaboration or communication skills.
rmbyrro 3y ago
It'd be very easy to game open contributions
ProjectArcturis 3y ago
>relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool. I have 20 years experience in very high level data science work. I do not have a public git repo because I've worked at for-profit companies and I don't do additional free work in my spare time.
- kuchenbecker 3y ago
  How dare you have a personal life. I'm the same, my git repo is a graveyard of projects all set to hidden.
Aurornis 3y ago
> can just ask for relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool. Eliminating 90% of your candidate pool sounds like a great way to slow your hiring to a crawl. Very few people have noteworthy and/or relevant GitHub activity. You'd probably be eliminating more like 95%. GitHub activity also has a high false positive rate in my experience. A lot of the GitHub profile superstars I've worked with were always spending their time working on open source things or something that they could put on their profile. They avoided doing anything internal to the company as much as possible because they knew they couldn't leverage it for their next job.
plasticchris 3y ago
Reminds me of the hiring committee at Google that rejected their own packets (with the names changed).
- deleted 3y ago
  [deleted]
- neilv 3y ago
  Found it, and I'm pleasantly surprised that it seems credible (much more credible than the LinkedIn post): https://www.youtube.com/watch?v=r8RxkpUvxK0?t=8m20s https://www.youtube.com/watch?v=r8RxkpUvxK0?t=8m20s from "Moishe Lettvin - What I Learned Doing 250 Interviews at Google"
  - plasticchris 3y ago
    Yeah, that’s the talk!
throwaway2037 3y ago
> Is able to maintain a sophisticated project with external contributors. To add one more nail into this coffin: Well, then you will eliminate 99.999% of candidates. Or more likely, you will get none. Seriously, read that sentence again: "maintain a sophisticated [open source] project". How many of those exist in open source? A few thousand at most. And there are millions of developers in the world.
hiAndrewQuinn 3y ago
Alternatively, an external vetting company which provides on-site, locked down locations for doing LeetCode problems in and then publishes their results online could be very useful. The real trick of course would be trust.
BeFlatXIII 3y ago
> Knows how to code in X language in a large project Or split this into “knows how to play nice on a large project” and “can readily learn new languages” if your company is willing to invest in training.

by rvz 3y ago

randmeerkat 3y ago

> …can just ask for relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool. Get your hiring done now while you can, when the economy rebounds you won’t be able to hire anyone. Also give your team a raise, because they’ll probably be the first to go once new options open up.

bpye 3y ago

Not everyone works in the open. I do have open source side projects and contributions I’ve made on my own time - but almost everything I’ve done at work is closed source.

leononame 3y ago

> can just ask for relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool Not everyone spends their free time contributing (to major nonetheless) to open source projects. There are a lot of great engineers that have enough work on their desks with their day job and there are also plenty of idiots in open source. Asking for relevant projects or asking for GitHub profiles to gauge relevant projects yourself is what people were already doing years ago and it wasn't a great hiring strategy. Turns out judging a software engineers skills is extremely hard.

angarg12 3y ago
This. Focusing your hiring on open source contributions biases the process and misses huge slices of the software engineering population. I made the best work of my life (by a long long shot) to private companies closed source.
thwarted 3y ago
Not everyone spends their free time contributing (to major nonetheless) to open source projects. and elsewhere in these comments, we see: I would expect candidates for programming jobs to demonstrate first class ChatGPT or other code copilot skills. Not everyone spends their free time learning first class ChatGPT or code copilot skills. It's interesting that this age old mantra about open source contributions being inappropriate for a hiring manager to expect because of what people do or do not do in their free time now does not apply to random other skill/experience that one must also acquire in their free time. If a company has, for example, "git experience required" on the job posting you'll either need to actively demonstrate that you know how to use git or passively demonstrate by having a on-line accessible corpus of git-related work that shows your experience. You don't get a pass on that requirement just because the companies you've been working at don't use git and, well, you don't want to spend your free time learning git. And it is appropriate for the hiring manager to list that as a requirement despite claims that they'll be disqualifying some percentage of the candidates by including that as a requirement.
- mythhabit 3y ago
  I think part of the issue is also the complete lack of thrust. If you state that a candidate must know git, and the candidate say they do know git for basic committing, branching, merging ect, then fine - let's move on. I've learned that prior work history, talk and basic problem solving is in no way indicative of performance at all. I've found that the only guaranteed process to find excellent candidates, is hiring interns and part-timers currently finishing education, and pick the ones with the right mindset and intelligence.
zelphirkalt 3y ago
That is a fair view. Of course there is also a tendency of great engineers doing project in their free time, since they love what they do. There is also the fact that there is a higher chance of someone having learned a lot more, if they continuously worked on projects in their free time. There is also a higher chance to find people, who are not just into software development jobs, because they pay well, but because this is what they want to do. So for hiring engineers, I can understand when they hire with a certain bias. Sure, no one should be expecting us to do extra rounds, and sure, one can be a great engineer even without extra rounds, but the tendency is for that to take more years without that drive to explore in ones free time. And that's OK! I think it is fair though, if a company wants to hire a more driven/curious/exploring person.

vkou 3y ago

If you want to eliminate 90% of candidates in a pool, a simpler solution is to take your stack of resumes, and shred the top 90% of them.

wkirby 3y ago

> is no longer an accurate measure of exceptional performance in the role It never was. No real-world job performance has ever been accurately measured by solving leetcode puzzles for one simple reason: problem solving is only ever going to be about 50% of your performance, and these puzzles don't address collaboration or communication skills.

rmbyrro 3y ago

It'd be very easy to game open contributions

ProjectArcturis 3y ago

>relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool. I have 20 years experience in very high level data science work. I do not have a public git repo because I've worked at for-profit companies and I don't do additional free work in my spare time.

kuchenbecker 3y ago
How dare you have a personal life. I'm the same, my git repo is a graveyard of projects all set to hidden.

Aurornis 3y ago

> can just ask for relevant projects or commits on GitHub of a major open source project and that eliminates the 90% of candidates in the pool. Eliminating 90% of your candidate pool sounds like a great way to slow your hiring to a crawl. Very few people have noteworthy and/or relevant GitHub activity. You'd probably be eliminating more like 95%. GitHub activity also has a high false positive rate in my experience. A lot of the GitHub profile superstars I've worked with were always spending their time working on open source things or something that they could put on their profile. They avoided doing anything internal to the company as much as possible because they knew they couldn't leverage it for their next job.

plasticchris 3y ago

Reminds me of the hiring committee at Google that rejected their own packets (with the names changed).

deleted 3y ago
[deleted]
neilv 3y ago
Found it, and I'm pleasantly surprised that it seems credible (much more credible than the LinkedIn post): https://www.youtube.com/watch?v=r8RxkpUvxK0?t=8m20s https://www.youtube.com/watch?v=r8RxkpUvxK0?t=8m20s from "Moishe Lettvin - What I Learned Doing 250 Interviews at Google"
- plasticchris 3y ago
  Yeah, that’s the talk!

throwaway2037 3y ago

> Is able to maintain a sophisticated project with external contributors. To add one more nail into this coffin: Well, then you will eliminate 99.999% of candidates. Or more likely, you will get none. Seriously, read that sentence again: "maintain a sophisticated [open source] project". How many of those exist in open source? A few thousand at most. And there are millions of developers in the world.

hiAndrewQuinn 3y ago

Alternatively, an external vetting company which provides on-site, locked down locations for doing LeetCode problems in and then publishes their results online could be very useful. The real trick of course would be trust.

BeFlatXIII 3y ago

> Knows how to code in X language in a large project Or split this into “knows how to play nice on a large project” and “can readily learn new languages” if your company is willing to invest in training.