6 ms·

I think this is the most plausible explanation too, but is this true in practice? I would have thought most talented programmers don't want to waste their time

I think this is the most plausible explanation too, but is this true in practice? I would have thought most talented programmers don't want to waste their time on leetcode, so it would primarily select for people who can't succeed without dedicating time to it. Have you seen this first-hand and compared leetcode grinders against say, prolific open source developers?

username90 5y ago
Most people in the world care about money, and practicing leetcode is worth way more money per time spent than basically anything else for a talented programmer, up until you hit the level where you can get into top paying companies. You can get into well paying jobs without it, but then you need skills that are way harder to practice and usually you also need a lot of personal connections as those jobs often selects friends.
- MillenialMan 5y ago
  I'm not really convinced by this argument. Consistently improving your overall skills will lead you to outpace the competition significantly more in the long run than min-maxing, unless your industry is extremely bad at differentiating ability. I suppose there's an argument to be gained that the marginal gain of Leetcode given thousands of hours of practice already is the better side of the tradeoff, but that's a circular argument. You're going to select your programmers based on whether they recently spent 30 hours doing Leetcode, rather than on what they got out of the 10,000 hours of regular coding practice they did beforehand? Doesn't seem like a good selection criteria to me. That's why I asked whether OP had actually seen the effect he was describing and compared results directly. It's easy to come up with hypothetical benefits.
  - username90 5y ago
    It is a bit tiring to explain the process to everyone, but here it goes: Companies don't select programmers based on leetcode performance, no, they care a great deal about those 10,000 hours people spent. However if you fail that leetcode test you are out no matter what other experience you have. And it so happens that the set of companies paying twice the average for any experience levels tend to do leetcoding, so no matter what level of experience you have leetcoding is great. Although sometimes your experience might impress low tier companies but not high tier ones, so they downlevel you and ultimately your total compensation didn't move much. If you haven't seen this process then you are either ignorant, don't live in an area with high tier companies, or don't have a lot of experience. Personally I did double my income by going through this process and I've seen many others do as well. Now, lots of low tier companies started seeing high tier companies do this process, so they started copying it for some reason. But it doesn't work if you don't combine it with also paying significantly more than most others.
    - MillenialMan 5y ago
      Ok, but companies do select based on Leetcode performance, in practice. It's the largest portion of the interview. It's useful as a filter, but it's treated as a benchmark. As evidenced by: > Personally I did double my income by going through this process and I've seen many others do as well. That's insane. That's a broken market. If that's happening, it's not a reliable signal. Ultimately, if the market is rational then your actual ability will matter more than anything else. The only time that isn't the case is if the market is irrational. You can argue the market is currently irrational (although I would say that's not quite what's going on), but placing a long-term bet on it staying that way... Bad idea, IMO. But I think the actual explanation for why high-tier (read: large) companies use Leetcode is a combination of the fact that they're monopolies, so they aren't punished for bad selection criteria (see Google's use of lateral thinking puzzles for years before admitting they were completely useless for predicting job performance), and that they need a replicable interview procedure that can be applied consistently by a workforce that fundamentally isn't very good at interviewing, thinks they are, and doesn't care if they get it wrong. It's the Big Mac of interviewing methods.
      - comp_throw7 5y ago
        > The only time that isn't the case is if the market is irrational. There are other situations, such as when measuring actual ability is sufficiently difficult or impossible (subject to the constraints you pointed out - via a process that's repeatable by a largely undifferentiated set of interviewers). In those situations, you have basically no choice but to look for proxy measures. Now, obviously, proxies can be gamed to various degrees. Leetcode is not actually such a terrible proxy, though. Here are some points in favor: 1) On average, better engineers will require less practice time to achieve similar results, 2) There is a certain minimum level of capability required to even "grind it out"; that minimum doesn't serve as a sufficient floor for being a net productive engineer, but it gets you a reasonable chunk of the way there, and that's why so much of the evaluation criteria for these interviews focuses on communication ability, since that's the rest of what they care about Taken together, you get a process that is reasonably good at eliminating false positives and still manages to tilt the field in favor of more skilled candidates. On the candidate's side, it has the benefits of being generalizable across multiple interviews and also getting easier each time you do it. While I'm somewhat sympathetic to the claim that the process favors candidates who have more free time, I think it's a largely overstated concern. First, this is true of any interviewing process wherein a candidate can improve their performance through practice. To a first approximation this is all interview processes. Second, it doesn't take _that_ much time. If you put in 100 hours (which is basically "made it a moderately important priority for 2-3 months") and you still can't clear any interviews, there are a few possible explanations. 1) You're failing at the communication side of things. Thankfully this is also something you can practice! 2) You fall into an unfortunate edge case, e.g. extreme performance anxiety, which isn't reflected in your day-to-day work. This sucks! I don't think there's a process that doesn't have unfortunate edge cases; all we can do is try to minimize them. 3) You're not able to learn the material well enough to generalize it to novel interview questions. This is the process working as intended. I've also heard a reason that goes something like "I can totally solve those problems, just not in 45 minutes". Problem-solving speed is going to be positively correlated with other traits of talented engineers; obviously some otherwise talented engineers will still fall on the right side of the curve. Again, this sucks, but please bring me a process that effectively rules out false positives without bringing in some false negatives. If you can manage it there's a huge market opportunity there.
jmchuster 5y ago
I don't consider myself particularly talented, but it's really not that much time to practice. At 10+ years of experience, I spent maybe up to 40 hours of practice the last couple of times I switched to interviewing mode. Pretty easy to spread out over 2 weeks to a month to get in shape. I don't consider that an unreasonable or unacceptable burden.
- MillenialMan 5y ago
  It's not that it's unreasonable, it's that it's a bad criteria because it's so easy. In 40 hours, a college graduate can get to the point where they're outdoing, say, one of the principal architects of AlphaGo? Then that selection criteria sucks. In any other situation we avoid metrics that are easy to game. The argument that it selects for industriousness strikes me as a cope. It might, but it's also so easy to game that it obscures actual ability (unless you're just using it as a filter). Is that trade-off really worth it?
  - jmchuster 5y ago
    So, i was responding to your comment: > I would have thought most talented programmers don't want to waste their time on leetcode, so it would primarily select for people who can't succeed without dedicating time to it. I was saying that it's not that much of a waste of time, because it's not that much of a time investment for a good outcome (doing well at interviews). If you want to change the argument from "it's a waste of time" to "it's trivially easy and easy to game", then why are there people who complain about leetcode and not wanting to do it?
    - MillenialMan 5y ago
      It is both a waste of time and easy to game. The reason it's a waste of time is that it's easy to game, so some of your rivals will be gaming it. Which means you have to do it too, so it just becomes a dumb tax you pay every time you interview for a new job. That's all from the individual candidate's perspective, of course. From the employer's perspective, you're still only going to have a minority of your interviewees having prepared their cheat codes. So you're evaluating based on ability to prepare a cheat code. Which IMO is retarded. If there wasn't a site dedicated to preparing cheat codes (which is how it used to be), then it would be sensible.

username90 5y ago

Most people in the world care about money, and practicing leetcode is worth way more money per time spent than basically anything else for a talented programmer, up until you hit the level where you can get into top paying companies. You can get into well paying jobs without it, but then you need skills that are way harder to practice and usually you also need a lot of personal connections as those jobs often selects friends.

MillenialMan 5y ago
I'm not really convinced by this argument. Consistently improving your overall skills will lead you to outpace the competition significantly more in the long run than min-maxing, unless your industry is extremely bad at differentiating ability. I suppose there's an argument to be gained that the marginal gain of Leetcode given thousands of hours of practice already is the better side of the tradeoff, but that's a circular argument. You're going to select your programmers based on whether they recently spent 30 hours doing Leetcode, rather than on what they got out of the 10,000 hours of regular coding practice they did beforehand? Doesn't seem like a good selection criteria to me. That's why I asked whether OP had actually seen the effect he was describing and compared results directly. It's easy to come up with hypothetical benefits.
- username90 5y ago
  It is a bit tiring to explain the process to everyone, but here it goes: Companies don't select programmers based on leetcode performance, no, they care a great deal about those 10,000 hours people spent. However if you fail that leetcode test you are out no matter what other experience you have. And it so happens that the set of companies paying twice the average for any experience levels tend to do leetcoding, so no matter what level of experience you have leetcoding is great. Although sometimes your experience might impress low tier companies but not high tier ones, so they downlevel you and ultimately your total compensation didn't move much. If you haven't seen this process then you are either ignorant, don't live in an area with high tier companies, or don't have a lot of experience. Personally I did double my income by going through this process and I've seen many others do as well. Now, lots of low tier companies started seeing high tier companies do this process, so they started copying it for some reason. But it doesn't work if you don't combine it with also paying significantly more than most others.
  - MillenialMan 5y ago
    Ok, but companies do select based on Leetcode performance, in practice. It's the largest portion of the interview. It's useful as a filter, but it's treated as a benchmark. As evidenced by: > Personally I did double my income by going through this process and I've seen many others do as well. That's insane. That's a broken market. If that's happening, it's not a reliable signal. Ultimately, if the market is rational then your actual ability will matter more than anything else. The only time that isn't the case is if the market is irrational. You can argue the market is currently irrational (although I would say that's not quite what's going on), but placing a long-term bet on it staying that way... Bad idea, IMO. But I think the actual explanation for why high-tier (read: large) companies use Leetcode is a combination of the fact that they're monopolies, so they aren't punished for bad selection criteria (see Google's use of lateral thinking puzzles for years before admitting they were completely useless for predicting job performance), and that they need a replicable interview procedure that can be applied consistently by a workforce that fundamentally isn't very good at interviewing, thinks they are, and doesn't care if they get it wrong. It's the Big Mac of interviewing methods.
    - comp_throw7 5y ago
      > The only time that isn't the case is if the market is irrational. There are other situations, such as when measuring actual ability is sufficiently difficult or impossible (subject to the constraints you pointed out - via a process that's repeatable by a largely undifferentiated set of interviewers). In those situations, you have basically no choice but to look for proxy measures. Now, obviously, proxies can be gamed to various degrees. Leetcode is not actually such a terrible proxy, though. Here are some points in favor: 1) On average, better engineers will require less practice time to achieve similar results, 2) There is a certain minimum level of capability required to even "grind it out"; that minimum doesn't serve as a sufficient floor for being a net productive engineer, but it gets you a reasonable chunk of the way there, and that's why so much of the evaluation criteria for these interviews focuses on communication ability, since that's the rest of what they care about Taken together, you get a process that is reasonably good at eliminating false positives and still manages to tilt the field in favor of more skilled candidates. On the candidate's side, it has the benefits of being generalizable across multiple interviews and also getting easier each time you do it. While I'm somewhat sympathetic to the claim that the process favors candidates who have more free time, I think it's a largely overstated concern. First, this is true of any interviewing process wherein a candidate can improve their performance through practice. To a first approximation this is all interview processes. Second, it doesn't take _that_ much time. If you put in 100 hours (which is basically "made it a moderately important priority for 2-3 months") and you still can't clear any interviews, there are a few possible explanations. 1) You're failing at the communication side of things. Thankfully this is also something you can practice! 2) You fall into an unfortunate edge case, e.g. extreme performance anxiety, which isn't reflected in your day-to-day work. This sucks! I don't think there's a process that doesn't have unfortunate edge cases; all we can do is try to minimize them. 3) You're not able to learn the material well enough to generalize it to novel interview questions. This is the process working as intended. I've also heard a reason that goes something like "I can totally solve those problems, just not in 45 minutes". Problem-solving speed is going to be positively correlated with other traits of talented engineers; obviously some otherwise talented engineers will still fall on the right side of the curve. Again, this sucks, but please bring me a process that effectively rules out false positives without bringing in some false negatives. If you can manage it there's a huge market opportunity there.

jmchuster 5y ago

I don't consider myself particularly talented, but it's really not that much time to practice. At 10+ years of experience, I spent maybe up to 40 hours of practice the last couple of times I switched to interviewing mode. Pretty easy to spread out over 2 weeks to a month to get in shape. I don't consider that an unreasonable or unacceptable burden.

MillenialMan 5y ago
It's not that it's unreasonable, it's that it's a bad criteria because it's so easy. In 40 hours, a college graduate can get to the point where they're outdoing, say, one of the principal architects of AlphaGo? Then that selection criteria sucks. In any other situation we avoid metrics that are easy to game. The argument that it selects for industriousness strikes me as a cope. It might, but it's also so easy to game that it obscures actual ability (unless you're just using it as a filter). Is that trade-off really worth it?
- jmchuster 5y ago
  So, i was responding to your comment: > I would have thought most talented programmers don't want to waste their time on leetcode, so it would primarily select for people who can't succeed without dedicating time to it. I was saying that it's not that much of a waste of time, because it's not that much of a time investment for a good outcome (doing well at interviews). If you want to change the argument from "it's a waste of time" to "it's trivially easy and easy to game", then why are there people who complain about leetcode and not wanting to do it?
  - MillenialMan 5y ago
    It is both a waste of time and easy to game. The reason it's a waste of time is that it's easy to game, so some of your rivals will be gaming it. Which means you have to do it too, so it just becomes a dumb tax you pay every time you interview for a new job. That's all from the individual candidate's perspective, of course. From the employer's perspective, you're still only going to have a minority of your interviewees having prepared their cheat codes. So you're evaluating based on ability to prepare a cheat code. Which IMO is retarded. If there wasn't a site dedicated to preparing cheat codes (which is how it used to be), then it would be sensible.