5 ms·

I hope this downturn gets rid of leetcode. I really have no desire to grind leetcode whenever I need a new job. I am more than happy to do system design and pra

I hope this downturn gets rid of leetcode. I really have no desire to grind leetcode whenever I need a new job. I am more than happy to do system design and practical coding questions (ex: integrate with some API like Stripe does), but the leetcode stuff is ridiculous for mid-senior+ level engineers.

jmole 3y ago
I don't think it will be the downturn that eliminates it, so much as it will be LLM-accelerated coding, and the normalization and incorporation of these tools into standard development workflows. If you think co-pilot is cool now, just wait and see what it does in 5 years. A team of 10 tomorrow can probably replace a team of 50 today. So maybe the interview process in 10 years is more like: "build an auction site like ebay in the next hour."
- arthurcolle 3y ago
  > So maybe the interview process in 10 years is more like: "build an auction site like ebay in the next hour." This is very "Monkey's Paw" and I can absolutely see something like that taking hold, as sloppy and stupid as such a task might be. I had an interview with a YC company who asked me how to do something, and then I did it, and then they asked "how could I make it a one-liner" - immediately thought was "I'm pretty sure I don't want to maintain a codebase full of fun little one-liners for everything" - managed to get it, but it definitely looked like crap. If I were to get asked to build eBay in an hour, I'm sure my auction model would be 1) shit, 2) very prone to social engineering type attacks (fake bids, etc)
- fnimick 3y ago
  Who needs LLM-accelerated coding for that? Leetcode was already comically unrealistic as an evaluation of actual engineering skill. When's the last time you had to solve an algorithm puzzle without any ability to do research or check references? Shoutout to the interview I had where I was asked a tricky graph question, reasoned my way to the optimal solution, then failed because I "should have recognized" that it was a variant of Djikstra's algorithm from the beginning. As we all know, being able to regurgitate an existing algorithm shows far more promise than being able to logically create it yourself!
  - bckr 3y ago
    I think that this shows how these interviews are really just a way to not get sued for discrimination when an interviewer just doesn’t like you.
- breck 3y ago
  > If you think co-pilot is cool now, just wait and see what it does in 5 years. What will it do? It can't write exponentially better code. Maybe 50% better, so that I would need to edit it's outputs less often. The only thing I could see it doing exponentially better would be writing _less_ code. Telling me "stop! close this file. close this project. you are working on the wrong thing!" Technology can jump sharply but then go relatively flat. We still haven't gone back to the moon, 50 years later. We still don't have at-home nuclear power plants.
rcme 3y ago
I detest leetcode-style questions, but system design questions are even worse. At least leetcode-style questions have some type of objective evaluation criteria. System design questions are entirely subjective and whether or not you're successful generally comes down to whether or not your first impression of a problem matches the expectations of your interviewer. At my last company, we stopped doing system design entirely because we found it was completely uncorrelated to whether or not a candidate did well on the job.
- jkaptur 3y ago
  Did you hire people who did poorly on the system design interview and then measure their job performance? Or did you just determine that "passing with flying colors" didn't really improve job performance vs. "just scraping by"?
  - rcme 3y ago
    We moved from leetcode + system design question to project-based interview + system design question to just project based. Our project-based interviews were the main hiring criteria, and we originally planned on using system design interviews for leveling. What we found is that doing well on the project-based interview was a much better signal than the system design interview. When we used the system design interview for leveling, we under leveled certain people and over leveled others. We discovered that it would have been more accurate to only use the project interview for both making a hiring decision and leveling. Ultimately, what we had trouble with was formalizing the evaluation criteria for the system design interview. With a project interview, we formalized the requirements of the project and everyone on the loop evaluates candidates against those criteria. With the system design interview, only one person ends up evaluating each candidate and a single person's judgement is highly variable. Unless the candidate did something egregiously wrong, it was very hard to say whether or not something was strictly "bad" or "good" and it was impossible to write down the list of "bad" and "good" qualities ahead of time.
    - MichaelNolan 3y ago
      How did you structure your project based interview? Was it live pair coding, or a take home project? Did you have any trouble with candidates declining the project? My first instinct as a candidate is to turn down take home project based interviews. I almost always say no to them (especially if it’s for the 1st round) for a few reasons: One: In general they seem like a poor use of my time. I would rather do 5 one hour coding or design interviews at 5 different companies than a single 5 hour project at one company. Two: I’m skeptical that anyone is actually reviewing the submitted project. And the evaluation criteria are usually unclear. For instance is it worth it to add linters during the build process? Or to introduce caching for api responses? Will the person evaluating the project notice details like that? Lastly: Every coding or design interview carries over to the next one. Preparing for one is preparing for all of them. In contrast I’ve seen projects (for a backend role) range from “build a simple weather api, any language” to “build a sudoku solver specifically in JavaScript”.
qbasic_forever 3y ago
Unfortunately it will likely get worse in the short term. With a huge pool of laid off people fighting for a small number of open positions (especially at the senior level and above) companies can be as demanding and stupid as they want with hiring. Leet code prowess is just another screening that whittles down the work they have to do to hire--only people that study and learn those interviews will pass them so the company has to worry less about actually figuring out if someone is a good fit for the position.
diehunde 3y ago
The problem is that you and the other hundreds of senior engineers would perform the same if it were all design and the other example you gave. There's just not a good solution for companies that must evaluate the number of candidates these big companies get daily.
- rendaw 3y ago
  You can already filter out 95% of candidates with fizz buzz.
time_to_smile 3y ago
Leetcode is also just an awful signal. It would be fine if it was only problems with false negatives, but the false positive rate is through the roof. I'm consistently shocked by coworkers I've had, who I knew had to pass challenging leetcode interviews, who seem to not only know nothing about building real world software, but don't really have a firm grasp on algorithms. Yes they can process a leetcode question requiring dynamic programming in record time, but when mapping a real world problem to a dp (or any other similar solution) are completely at a loss.
lumost 3y ago
It’s a balance, an LC medium isn’t to hard to do offhand. If your business is hard tech, it’s a reasonable filter. I see diminishing returns from LC hards and greater other than as a guardrail to block management from hiring senior people who are practically no better than the junior people.
lovecg 3y ago
If you think of Leetcode and similar filters as just veiled IQ tests it makes way more sense - of course those problems don’t have much to do with actual day to day engineering. If companies could just administer IQ tests they would happily do that. Since that’s illegal, they rely on Leetcode.
- greatpostman 3y ago
  Not all iq is created equal. Creativity matters
  - anthonypasq 3y ago
    testing for creativity is much harder than testing for iq, and a high iq person with no creativity is a safer bet than a dumb creative person.
- blululu 3y ago
  It's a modified IQ test filtered through the transfer function of having a CS degree. A big part of the shade that this practice gets is from people with backgrounds that are CS adjacent who are highly intelligent, can do math at a university level and perfectly capable of writing quality production ready code are dinged because they don't know some CS 201 puzzle that is never used in practice.
  - 93po 3y ago
    Interviewing isn't just a measure of how good you are at programming in real life. It's how good of an employee you will be. Conformity is a strong indicator of this. Comparing someone who completed a CS degree versus some guy that learned to program in a garage hacking together broken computer parts - the employer is going to choose the CS degree guy, even if the hacker is better at programming. CS degree means you follow direction, you can work within a rigid structure, you follow rules enough to make good grades and not get kicked out, and you're capable of doing monotonous work that sometimes has very little value.
    - 3y ago
myth_drannon 3y ago
From reading recent interview experiences on Reddit, the thing got way worse now that you have thousands of unemployed engineers competing for scarce positions. Now it's leetcode hards AND takeaway assignment.
cwilkes 3y ago
How would the downturn get rid of leetcode? If anything it will dramatically increase its use as now there’s 100k more programmers on the market all applying for the same reduced pool of jobs. Somehow companies are going to have to now filter through a lot more candidates to fill positions, and for right now leetcode-like tests are the quickest way to get a gauge of an engineer’s performance (on leetcode). Don’t get me wrong: I think leetcode hiring depending on how it is done doesn’t result in getting the best talent.

jmole 3y ago

I don't think it will be the downturn that eliminates it, so much as it will be LLM-accelerated coding, and the normalization and incorporation of these tools into standard development workflows. If you think co-pilot is cool now, just wait and see what it does in 5 years. A team of 10 tomorrow can probably replace a team of 50 today. So maybe the interview process in 10 years is more like: "build an auction site like ebay in the next hour."

arthurcolle 3y ago
> So maybe the interview process in 10 years is more like: "build an auction site like ebay in the next hour." This is very "Monkey's Paw" and I can absolutely see something like that taking hold, as sloppy and stupid as such a task might be. I had an interview with a YC company who asked me how to do something, and then I did it, and then they asked "how could I make it a one-liner" - immediately thought was "I'm pretty sure I don't want to maintain a codebase full of fun little one-liners for everything" - managed to get it, but it definitely looked like crap. If I were to get asked to build eBay in an hour, I'm sure my auction model would be 1) shit, 2) very prone to social engineering type attacks (fake bids, etc)
fnimick 3y ago
Who needs LLM-accelerated coding for that? Leetcode was already comically unrealistic as an evaluation of actual engineering skill. When's the last time you had to solve an algorithm puzzle without any ability to do research or check references? Shoutout to the interview I had where I was asked a tricky graph question, reasoned my way to the optimal solution, then failed because I "should have recognized" that it was a variant of Djikstra's algorithm from the beginning. As we all know, being able to regurgitate an existing algorithm shows far more promise than being able to logically create it yourself!
- bckr 3y ago
  I think that this shows how these interviews are really just a way to not get sued for discrimination when an interviewer just doesn’t like you.
breck 3y ago
> If you think co-pilot is cool now, just wait and see what it does in 5 years. What will it do? It can't write exponentially better code. Maybe 50% better, so that I would need to edit it's outputs less often. The only thing I could see it doing exponentially better would be writing _less_ code. Telling me "stop! close this file. close this project. you are working on the wrong thing!" Technology can jump sharply but then go relatively flat. We still haven't gone back to the moon, 50 years later. We still don't have at-home nuclear power plants.

rcme 3y ago

I detest leetcode-style questions, but system design questions are even worse. At least leetcode-style questions have some type of objective evaluation criteria. System design questions are entirely subjective and whether or not you're successful generally comes down to whether or not your first impression of a problem matches the expectations of your interviewer. At my last company, we stopped doing system design entirely because we found it was completely uncorrelated to whether or not a candidate did well on the job.

jkaptur 3y ago
Did you hire people who did poorly on the system design interview and then measure their job performance? Or did you just determine that "passing with flying colors" didn't really improve job performance vs. "just scraping by"?
- rcme 3y ago
  We moved from leetcode + system design question to project-based interview + system design question to just project based. Our project-based interviews were the main hiring criteria, and we originally planned on using system design interviews for leveling. What we found is that doing well on the project-based interview was a much better signal than the system design interview. When we used the system design interview for leveling, we under leveled certain people and over leveled others. We discovered that it would have been more accurate to only use the project interview for both making a hiring decision and leveling. Ultimately, what we had trouble with was formalizing the evaluation criteria for the system design interview. With a project interview, we formalized the requirements of the project and everyone on the loop evaluates candidates against those criteria. With the system design interview, only one person ends up evaluating each candidate and a single person's judgement is highly variable. Unless the candidate did something egregiously wrong, it was very hard to say whether or not something was strictly "bad" or "good" and it was impossible to write down the list of "bad" and "good" qualities ahead of time.
  - MichaelNolan 3y ago
    How did you structure your project based interview? Was it live pair coding, or a take home project? Did you have any trouble with candidates declining the project? My first instinct as a candidate is to turn down take home project based interviews. I almost always say no to them (especially if it’s for the 1st round) for a few reasons: One: In general they seem like a poor use of my time. I would rather do 5 one hour coding or design interviews at 5 different companies than a single 5 hour project at one company. Two: I’m skeptical that anyone is actually reviewing the submitted project. And the evaluation criteria are usually unclear. For instance is it worth it to add linters during the build process? Or to introduce caching for api responses? Will the person evaluating the project notice details like that? Lastly: Every coding or design interview carries over to the next one. Preparing for one is preparing for all of them. In contrast I’ve seen projects (for a backend role) range from “build a simple weather api, any language” to “build a sudoku solver specifically in JavaScript”.

qbasic_forever 3y ago

Unfortunately it will likely get worse in the short term. With a huge pool of laid off people fighting for a small number of open positions (especially at the senior level and above) companies can be as demanding and stupid as they want with hiring. Leet code prowess is just another screening that whittles down the work they have to do to hire--only people that study and learn those interviews will pass them so the company has to worry less about actually figuring out if someone is a good fit for the position.

diehunde 3y ago

The problem is that you and the other hundreds of senior engineers would perform the same if it were all design and the other example you gave. There's just not a good solution for companies that must evaluate the number of candidates these big companies get daily.

rendaw 3y ago
You can already filter out 95% of candidates with fizz buzz.

time_to_smile 3y ago

Leetcode is also just an awful signal. It would be fine if it was only problems with false negatives, but the false positive rate is through the roof. I'm consistently shocked by coworkers I've had, who I knew had to pass challenging leetcode interviews, who seem to not only know nothing about building real world software, but don't really have a firm grasp on algorithms. Yes they can process a leetcode question requiring dynamic programming in record time, but when mapping a real world problem to a dp (or any other similar solution) are completely at a loss.

lumost 3y ago

It’s a balance, an LC medium isn’t to hard to do offhand. If your business is hard tech, it’s a reasonable filter. I see diminishing returns from LC hards and greater other than as a guardrail to block management from hiring senior people who are practically no better than the junior people.

lovecg 3y ago

If you think of Leetcode and similar filters as just veiled IQ tests it makes way more sense - of course those problems don’t have much to do with actual day to day engineering. If companies could just administer IQ tests they would happily do that. Since that’s illegal, they rely on Leetcode.

greatpostman 3y ago
Not all iq is created equal. Creativity matters
- anthonypasq 3y ago
  testing for creativity is much harder than testing for iq, and a high iq person with no creativity is a safer bet than a dumb creative person.
blululu 3y ago
It's a modified IQ test filtered through the transfer function of having a CS degree. A big part of the shade that this practice gets is from people with backgrounds that are CS adjacent who are highly intelligent, can do math at a university level and perfectly capable of writing quality production ready code are dinged because they don't know some CS 201 puzzle that is never used in practice.
- 93po 3y ago
  Interviewing isn't just a measure of how good you are at programming in real life. It's how good of an employee you will be. Conformity is a strong indicator of this. Comparing someone who completed a CS degree versus some guy that learned to program in a garage hacking together broken computer parts - the employer is going to choose the CS degree guy, even if the hacker is better at programming. CS degree means you follow direction, you can work within a rigid structure, you follow rules enough to make good grades and not get kicked out, and you're capable of doing monotonous work that sometimes has very little value.
  - 3y ago

myth_drannon 3y ago

From reading recent interview experiences on Reddit, the thing got way worse now that you have thousands of unemployed engineers competing for scarce positions. Now it's leetcode hards AND takeaway assignment.

cwilkes 3y ago

How would the downturn get rid of leetcode? If anything it will dramatically increase its use as now there’s 100k more programmers on the market all applying for the same reduced pool of jobs. Somehow companies are going to have to now filter through a lot more candidates to fill positions, and for right now leetcode-like tests are the quickest way to get a gauge of an engineer’s performance (on leetcode). Don’t get me wrong: I think leetcode hiring depending on how it is done doesn’t result in getting the best talent.