9 ms·

In Defense of LeetCode Interviews

24 points by alexmolas 2y ago

trh0awayman 2y ago
If you hire using LeetCode, you will surround yourself with people who enjoy blogging about LeetCode in their free time. LeetCode was never about LeetCode, it was always a stand in for culture. It's now a signal for baseline compliance. That's generally good for companies that require mostly operationalists. The problem is that anyone can learn to leetcode. If you're interested in doing something new and not just warehousing CS lawyers, you're gonna have to ask better questions than that.
- beaconify 2y ago
  Leetcode is often one or two rounds. You also get tested for culture and design interviews. This is not ideal but hard to see how FAANGs are gonna do it. I guess FAANGs need operationalists though except for their research arms. Dynamodb works well. Your job is to make it work 0.01% better to save a few mill.
- drewcoo 2y ago
  > The problem is that anyone can learn to leetcode. If you're interested in doing something new and not just warehousing CS lawyers, you're gonna have to ask better questions than that. I think the problem is that everyone thinks they can ask better questions and almost none of those people are qualified to judge their interviewing competency or have data to back themselves up. Every company has its own special interview techniques that it's convinced will lead to the best possible outcomes when those only demonstrably lead to the current staffing. To question any of that means pointing fingers at pretty much everyone and that's a political non-starter.
cherryteastain 2y ago
I feel like the author missed the most critical point: Leetcode style interviews are great at weeding out terrible coders (95% of applicants) early on in the hiring process, but don't carry an as strong signal for separating excellent candidates from the good ones. Asking standard array/string manipulation/sorting etc questions in a 30min phone screen is very valuable to save your engineers 5hrs on a poor candidate. Conversely, throwing an NP hard leetcode hard at a senior dev with 20 years of experience and excellent culture fit with your organization in the 9th interview is basically meaningless.
- deleted 2y ago
  [deleted]
- chongli 2y ago
  Conversely, throwing an NP hard leetcode hard at a senior dev with 20 years of experience and excellent culture fit with your organization in the 9th interview is basically meaningless. Worse than meaningless, it's a great way to toss away a diamond in the rough. Leetcode problems skew towards competitive programming and grinders. They do absolutely nothing to show real-world programming skills which involve: 1) working within a large, existing codebase, 2) strong code documentation and commit message habits, 3) understanding of coding styles within the company culture so that you don't write in an unintelligible, bespoke style, and 4) communication skills and the ability to work within a team so that the candidate is not fighting strong headwinds.
argentum47 2y ago
This opinion might be a bit controversial, so correct me if I am wrong. Going by the Think Fast Think Slow, most of the people who were great at what they did, could be categorized overall, in terms of how they put in their effort, where they put in, and when they started from. (Apart from luck) I think of this in terms of traits. Big tech companies, they would want to avoid any form of engineering silos, giving so much power to a single/single set of engineers, that would halt their business. Which is why they have this process. They want a whole lot of easily replaceable engineers, who (might work on different projects), but overall have the same kind of mentality. This way they know, if the candidates have done what they need to do, and talked about things a certain way, they on a fundamental level would be the same. So even if the person leaves, the style of work for the next incoming engineer wouldn't change, and everyone can go along with minimum friction. Also, thinking a bit more, /^[FAANG]*/ companies would obviously try to cross-hire. Same as BIG4, same as /TCS,CTS,Infosys/ and likes. A good chunk of people who work in MNC's stay in MNC's. Infact sometimes the projects are transferred from one MNC to the other, and then the manager or TL or whatever leading the project is hired first, and then he/she brings his/her team from the previous org. Although end of the day, there will be a few hits and misses, but given the plethora of people working there, on a large scale of things, it doesn't really matter. I mean in a group of 20 people, 2 leaves. Its barely anything to make a dent.
habitue 2y ago
Leetcode is just a proxy for an IQ test. That's it. If you can study up and do well on leetcode, then you're smart. People getting hung up on how realistic the questions are, or whether you ever need to implement X data structure are confused. Fact is that no job can give a reasonable test for how it is to work there short of working there. There's team dynamics, developer / project fit, etc etc. All you can ever do is measure some proxies. Leetcode is just a much better proxy than the old "how many ping-pong balls fit in a school bus" questions.
- deleted 2y ago
  [deleted]
- dilyevsky 2y ago
  I agree. One could say that just administering proper iq test would be better but that would be seen as insulting or whatever and also potentially illegal so people resort to elaborate challenges like lc
  - novok 2y ago
    you can only administer most iq tests once, maybe twice if its been quite a while and you can rely on them not remembering the questions properly to get a practice effect
- graymatters 2y ago
  Actually - “how many golf balls fit into a VW Beatle” is a better proxy.
  - drewcoo 2y ago
    Those are called Fermi questions. https://en.wikipedia.org/wiki/Fermi_problem https://en.wikipedia.org/wiki/Fermi_problem
- willio58 2y ago
  I don’t know that people generally have a problem with testing their knowledge, I think most people that hate on Leetcode (including myself) just don’t like to code in front of strangers in a timed setting like that. I’ve never been good at tests in school. I probably averaged Cs on tests through college. Projects though? Aced them every time and sometimes got pulled aside and told I was highly exceeding the projects of others in the class. I just do well when I can think alone, or when I can actually work toward a solution with a coworker. I’ve made the companies I work for millions of dollars. But put me in front of a white board and ask me an algorithm question? You’d think I was fresh out of CS101.
geuis 2y ago
I did a lot of leetcode stuff this year. Dozens of interviews, no takers. What got me a job was that I was a solo founder running a business and learning how to make the most of limited resources. Aka I was someone that gets things done and spends the least amount possible to get there. Use the resources you have and show you can create value. Again, it all comes down to show don't tell. Leetcode is valuable as a way to practice and maybe reaffirm skills. It's not useful for hiring in a direct way.
Apocryphon 2y ago
Aren't there supposed AI tools out there for cheating on these standardized tests?
- sciencesama 2y ago
  For real ??
- slashdave 2y ago
  There are, but they are easy to detect
  - novok 2y ago
    In my experience, I just need the aha hint and I'm usually good. When I looked them up later I was slapping my hand on my head. How do you detect that?
waythenewsgoes 2y ago
I have always seen LeetCode problems as effectively hazing rituals in a job interview setting. A high pressure interview situation simply won’t bring out peak problem-solving capabilities in many people. At best you get some superficial insight into someone’s problem solving methodology, but at worst you filter out otherwise excellent fits for not being able to solve an ultimately inconsequential problem under pressure. LeetCode questions as interview questions are mostly theater. Most people who do well on these aren’t actually “solving them” on the fly from scratch. They just happen to have seen the exact same problem before and retake the steps they’ve memorized to get to the answer. Testing whether or not someone can regurgitate the solution they have memorized to a math problem doesn’t tell you much about how they will perform in a truly novel non-contrived constraint problem scenario, which is generally what most dev work entails. Perhaps if you are working at Bespoke Algorithms ‘R Us, benchmarking this would have more value to your org, but for most dev roles at most companies it is hard to see it as more than a compliance exercise, or maybe even as a tool to weed out those with families that can’t devote the hours/day to LC memorization.
- erik_seaberg 2y ago
  I don't try to add any pressure, but going on call will require some resilience from each team member.
  - waythenewsgoes 2y ago
    As long as you don’t wake me at 3AM to rotate a red-black tree, or find the median of two sorted arrays we should be good
    - ihumanable 2y ago
      Exactly this. As someone that’s served as an oncall engineer for years now, the skills you need to operate a cluster are completely different from the things leetcode tests for.
    - Ekaros 2y ago
      I must remember to ask how many times those problems happen in company's code base in next interview. Or whatever they ask. And if it is more than one, ask if they have libraries for it... You know good development practises and all.
neilv 2y ago
I'm so sick of LeetCode nonsense, I'm inspired to make a denylist of every company and blog and username that promotes it.
- dothack 2y ago
  Here is an allowlist: https://github.com/poteto/hiring-without-whiteboards https://github.com/poteto/hiring-without-whiteboards
eadwu 2y ago
If you are the type of person who writes a big brain log(n) solution instead of n solution when there is no real reason (compute wise) and the log(n) is a pain in the ass to maintain. I hope I never see you in my team.
- slashdave 2y ago
  There is a lot to say about this. Due to technical limitations, the Leetcode site uses odd corner cases in their test suite to guide practitioners to specific, bespoke solutions. It's kind of insidious.
- beaconify 2y ago
  Yes! Just say give me o(log(n)) dollars! Ah now you care about n and k.
deleted 2y ago
[deleted]
chrisbrandow 2y ago
There’s just so much guesswork with these as interview questions. The number of different problem spaces is non-trivial, and understanding the nature of the problem space for use in work is quite different from being able to work out a problem in 30 minutes in front of other people often in a code editor that is unfamiliar. We lack an agreed upon, specific way to evaluate someone’s talent in 30-90 minutes. LC problems are not a terribly efficient way to fix this.
joshdavham 2y ago
I would rather hire someone who has evidence of working on prior projects than someone who can solve brain teasers. There are many people who excel at LeetCode but struggle to build even simple software.
- coolThingsFirst 2y ago
  It's 2024, you can find tons of people with project experience AND who are good at LC. It's really saturated right now and it's very difficult to stand out.
valenterry 2y ago
> People that never done LC problems probably don’t even know how to implement some data structures. So when they face a difficult problem they will probably just use a brute force approach. You don't need leetcode for that. It's sufficient to talk about datastructures. In fact, that is a much better and actually reasonable thing to do in an in an interview. > I also like these questions because you can ask them to any kind of software developer, from frontend to DevOps. Only those that will apply at your place, which will exclude me if you require a leetcode interview. So, in fact, you will be biased by prefiltering candidates. It's like when companies say "we only hire the top 1%" and I ask "the top 1% of all engineers or the top 1% of the engineers that chose to apply at your company?"
hatthew 2y ago
Coding assessments are definitely valuable in general. What isn't valuable are the class leetcode problems, where you have to use a heap, or DP, or some random useless data structure to solve a problem in O(n) instead of O(n log n) time. How about instead of asking a simple problem that has an obscure solution, ask a complicated problem with an obvious solution, but one that tests their software engineering skills. Make them ask clarifying questions about which of the many edge cases they are expected to handle, and verify which can be ignored. Add additional requirements that make refactoring necessary. Add constraints about the environment and ask how the candidate would handle that.
MarkMarine 2y ago
I have a couple great companies on my resume because I ground these leetcode questions, but now that I’ve been working and in industry for 14 years… I don’t have the time to do the grind unless I’m explicitly looking for a job. The DS&A skills are still there, I use them and I’m glad I have them, but when someone reaches out to me to join their startup out of the blue and then hits me with the leetcode hard question that I haven’t seen in 8 years… well I’ve not passed those interviews. Usually I don’t say “ok, I’ll study for 4 weeks before the rounds” so I go in cold. I never give these type of questions in my interviews of senior/staff+, I build out topical problems for the space I’m actually hiring for, then simplify it down to the interesting bits. I give a ramp, a simple problem, a more complex tweak to the simple problem that needs an interesting data structure (maybe needs a heap or similar) and then another tweak that forces them to abandon that data structure and do something novel. You can also fail this and still get hired. With junior engineers, I’m sorry but I need something that looks like leetcode so I know you put the work in, and I can’t ask the topical questions because they have no frame of reference. These questions are like the common denominator for someone with no real world skills. I need to see that you’re driven and self motivated enough to teach yourself this (probably useless, when is the last time you had to implement merge sort) skill. I also don’t think there is a great alternative. I put an extraordinary amount of work into making my questions that aren’t Leetcode, when they leak I’m heartbroken. I don’t want to just let a fancy school be the decision point, so I need to find a fair way to test. Asking people to do 1-2 days work or pair program… it’s usually caused a lot of dropping out of the funnel. So I would love to hear alternatives that are working for others
- choppaface 2y ago
  Leetcode can work well for junior / new grad candidates if you help & hint them a bit. For example they probably forgot breadth-first search, but if you give a couple hints about queues and they show tremendous swiftness, then that can be a good sign they’ll learn very quickly on the job. Does not prevent false negatives but reduces the FN rate and gives them what they really want (learning experience).
ihumanable 2y ago
There are thousands, and I’m not exaggerating, literally thousands of leetcode problems. A lot of them require you to make, what to me at least, is some non-obvious clever observation about the problem. Sure the problem is talking about a guy robbing houses, but if you stand on your head and squint just right you’ll realize this is actually a graph cycle detection problem and you should use Floyd’s algorithm to solve it. Because there are thousands of these problems the amount of time it would take someone to become familiar with them is prohibitive. So you are at the mercy of the interviewer, have they picked a super clever one, are they going to be ok with removing duplicates from the answer by tossing stuff in a set or do they want you to pull some dynamic programming out of thin air. It’s the part where you have to divine the trick under pressure that measures nothing of value. I’ve been a professional software engineer for 2 decades. I’ve had times when I’ve been trying to solve some very tricky problem and done research and thought thoughts and come up with pretty clever solutions, or at least I think they are clever. Not once have I had to do this under pressure in a 45 minute time box with someone looking over my shoulder. That’s my objection to leetcode. Sure it’s great if a candidate can recognize that your riddle about topological map rain capture is actually just a restatement of Kolger’s postulate at first glance (a problem and postulate I’ve just made up because I’m not going to wade back into leetcode right now) but that’s an insane thing to optimize for. The vast majority of the problems programmers solve are actually just mapping business domains into code. The most common problems that need solving is taking squishy, incomplete, and contradictory requirements from multiple stakeholders and figuring out what needs to get done. People in the real world are rarely rolling their own data structures, because the red black tree you slap together is going to be infinitely worse than the battle tested highly optimized one you can pull out of the standard library or off your package manager. In my long career I’ve had a handful of occasions to actually build a data structure or solve a problem with some very clever algorithm. And in those cases you don’t really want people shooting from the hip anyways, you would want them to do research and see what prior art exists so they can discover something like Floyd’s algorithm for finding cycles in a unidirectional graph (ok this one is real). It is not clear to me what exactly leetcode tests. My best guess would be your ability to take a disguised questions and convert it into a handful of problem shapes and solve those. But if you grade leetcode like the website does during an interview, expect to lose a lot of perfectly fine candidates along the way.
ifiht 2y ago
Welcome to your leetcode interview in the age of ai: https://leetcodewizard.io/ https://leetcodewizard.io/
- ifiht 2y ago
  Also I love these comments, thank you all for vindicating my disdain for this practice.
000ooo000 2y ago
>However, I usually trust my intuition quite a lot, and before giving it up I decided to find some strong arguments in favor of Leetcode (LC) interviews, and here they are. Sounds like confirmation bias to me..
bearjaws 2y ago
IMO the worst problem with leetcode challenges are the fact they are 100% in the training set of LLMs. They solve them instantly, and making matters worse they are 100% valueless to a company. What universe do we live in that hiring managers want staff that are 100% as skilled as an LLM but probably not as strong in areas that actually matter? The reality everyone with a brain is going to cheat on these, they are typically pretty early in the interview process and it will hopefully get replaced with a real world test. At my last job we just used example problems that we had seen in the past, usually REST API focused with just enough nuance to make engineers think through it and potentially refactor their code. Then you can ask them specifics of their thought process and get insight to their experience.
danjl 2y ago
A good senior developer should be able to go out to lunch and know if the candidate has the skills and the right cultural match by the end. There's no need for coding tests or whiteboarding problems. I've seen some of the best developers I've worked with in 40 years of coding get turned away by Google and Apple because of their terrible interview processes. The only thing better than going out to lunch is working with the person for a couple of weeks. Ideally this happens on a paid contract basis. Of course our culture makes it more challenging to switch jobs if you want to work together first, but really no other interview process has ever provided good data.
- novok 2y ago
  At big companies HR and legal will say you need a reproducible, auditable, objective and defined interview process or you open yourself to a whole host of lawsuits, and you will get lawsuits because there is blood to squeeze from that lawsuit cow. Also at large scale you cannot trust the entire corpus of employees since your are well beyond dunbars number, so the process is there to prevent cheater and nepotism clusters forming where random employees hire their incompetent drinking buddies or cousins after a single "lunch". That process works for small companies, and it's an advantage they should leverage, it does not work for the large ones. IMO you need work sample tests as a minimum, which means a coding assignment. Some people are great bullshitters at lunch.
- alisey 2y ago
  In my experience, the results from those who claim they can judge skills based on a lunch conversation have not been good. I think people who can do it are the exception, not the norm.
jchw 2y ago
> People don’t like LC interviews because they are bad at them, I enjoy that this assertion is made with essentially no attempt to justify it, it's just said matter-of-factly as if it's just obvious truth. Well, let me speak at least for myself: Absolutely not.
dherls 2y ago
> These interviews can identify candidates with strong problem-solving skills and logical reasoning abilities. I would disagree with this premise. Leetcode identifies people who have just finished cramming for Leetcode questions. You don't need logical reasoning abilities to solve Leetcode, just encyclopedic knowledge of algorithms and data structures
MichaelNolan 2y ago
I think the article missed the biggest argument in favor leetcode style interviews - that they are quick and cheap to administer, and relatively unbiased/objective. The reason big companies use these interview methods, is because they have to interview tens of thousands of candidates per year. None of the common alternative interview methods that get tossed around can scale to thousands of applicants.
pajeets 2y ago
my retort to this article is this simple question: if you can look things up in an encyclopedia, why are you forced to recite it? if even low parameter LLM can solve most of these leetcode examples that cost nothing to run, why are we using it to discriminate applicants to measure "how badly do you want the $250k/year" It's almost like the people hiring are optimizing for masochists and there lies the devil, you want hyper-rationality because they are the easiest people to manipulate and brainwash. Highly sensitive and type B personality aren't going to sit around and let them trade their life to destroy or trap large number of the population with addictive algorithms or drone target selection. Matter of fact IQ was invented specifically to filter out the most programmable minds, you wouldn't be able to run a country's intelligence agency without being able to brainwash people or a corporation. The promoters at the helms are usually the exact opposite of the producers, unwavering, unforgiving, uncompromising.
- beaconify 2y ago
  L1 cache is useful (or maybe good processor design is the better analogy...) Being quick at the command line, using the right tools, can get around VS code quick, can solve the more trivial code problems quick (or quickly critique LLM output or a blog post). It adds up.
etse 2y ago
This article was kind of lazy. It’s a lot of conjecture, a couple anecdotal observations, and some rhetoric for good measure. Basically, the content was a let down from the title. On the other hand, I didn’t know Data Scientist candidates would also be tested with LeetCode questions.
mcslambley 2y ago
These are not strong or well structured counter arguments. The first argument, for instance, reads as premature optimization via hiring practices.
golly_ned 2y ago
None of the points made in this article were original or insightful. Disappointing for an unorthodox position.
coolThingsFirst 2y ago
They are good and I've had nothing but positive experiences through them. In all the tech phone interviews that I've done in my life I have NEVER failed a LC/algorithm related interview. My secret is to try to engage genuinely with the problem. And even when I haven't been able to solve a problem I still would get passed to next round bcs I'd be on the right track. I still don't know what ppl are complaining about. LC have a correct solution and a good interviewer will pass you if you are on a good track. They are also relatively easy to prepare for. Now compare to actual BS which is the requirement to have done work in a particular track that the company is hiring for. Now that's total BS, I can't get 5 years in python in like 2 weeks before interview no matter how motivated I may be for the job.

24 points by alexmolas 2y ago

If you hire using LeetCode, you will surround yourself with people who enjoy blogging about LeetCode in their free time. LeetCode was never about LeetCode, it was always a stand in for culture. It's now a signal for baseline compliance. That's generally good for companies that require mostly operationalists. The problem is that anyone can learn to leetcode. If you're interested in doing something new and not just warehousing CS lawyers, you're gonna have to ask better questions than that.

beaconify 2y ago
Leetcode is often one or two rounds. You also get tested for culture and design interviews. This is not ideal but hard to see how FAANGs are gonna do it. I guess FAANGs need operationalists though except for their research arms. Dynamodb works well. Your job is to make it work 0.01% better to save a few mill.
drewcoo 2y ago
> The problem is that anyone can learn to leetcode. If you're interested in doing something new and not just warehousing CS lawyers, you're gonna have to ask better questions than that. I think the problem is that everyone thinks they can ask better questions and almost none of those people are qualified to judge their interviewing competency or have data to back themselves up. Every company has its own special interview techniques that it's convinced will lead to the best possible outcomes when those only demonstrably lead to the current staffing. To question any of that means pointing fingers at pretty much everyone and that's a political non-starter.

cherryteastain 2y ago

I feel like the author missed the most critical point: Leetcode style interviews are great at weeding out terrible coders (95% of applicants) early on in the hiring process, but don't carry an as strong signal for separating excellent candidates from the good ones. Asking standard array/string manipulation/sorting etc questions in a 30min phone screen is very valuable to save your engineers 5hrs on a poor candidate. Conversely, throwing an NP hard leetcode hard at a senior dev with 20 years of experience and excellent culture fit with your organization in the 9th interview is basically meaningless.

deleted 2y ago
[deleted]
chongli 2y ago
Conversely, throwing an NP hard leetcode hard at a senior dev with 20 years of experience and excellent culture fit with your organization in the 9th interview is basically meaningless. Worse than meaningless, it's a great way to toss away a diamond in the rough. Leetcode problems skew towards competitive programming and grinders. They do absolutely nothing to show real-world programming skills which involve: 1) working within a large, existing codebase, 2) strong code documentation and commit message habits, 3) understanding of coding styles within the company culture so that you don't write in an unintelligible, bespoke style, and 4) communication skills and the ability to work within a team so that the candidate is not fighting strong headwinds.

argentum47 2y ago

This opinion might be a bit controversial, so correct me if I am wrong. Going by the Think Fast Think Slow, most of the people who were great at what they did, could be categorized overall, in terms of how they put in their effort, where they put in, and when they started from. (Apart from luck) I think of this in terms of traits. Big tech companies, they would want to avoid any form of engineering silos, giving so much power to a single/single set of engineers, that would halt their business. Which is why they have this process. They want a whole lot of easily replaceable engineers, who (might work on different projects), but overall have the same kind of mentality. This way they know, if the candidates have done what they need to do, and talked about things a certain way, they on a fundamental level would be the same. So even if the person leaves, the style of work for the next incoming engineer wouldn't change, and everyone can go along with minimum friction. Also, thinking a bit more, /^[FAANG]*/ companies would obviously try to cross-hire. Same as BIG4, same as /TCS,CTS,Infosys/ and likes. A good chunk of people who work in MNC's stay in MNC's. Infact sometimes the projects are transferred from one MNC to the other, and then the manager or TL or whatever leading the project is hired first, and then he/she brings his/her team from the previous org. Although end of the day, there will be a few hits and misses, but given the plethora of people working there, on a large scale of things, it doesn't really matter. I mean in a group of 20 people, 2 leaves. Its barely anything to make a dent.

habitue 2y ago

Leetcode is just a proxy for an IQ test. That's it. If you can study up and do well on leetcode, then you're smart. People getting hung up on how realistic the questions are, or whether you ever need to implement X data structure are confused. Fact is that no job can give a reasonable test for how it is to work there short of working there. There's team dynamics, developer / project fit, etc etc. All you can ever do is measure some proxies. Leetcode is just a much better proxy than the old "how many ping-pong balls fit in a school bus" questions.

deleted 2y ago
[deleted]
dilyevsky 2y ago
I agree. One could say that just administering proper iq test would be better but that would be seen as insulting or whatever and also potentially illegal so people resort to elaborate challenges like lc
- novok 2y ago
  you can only administer most iq tests once, maybe twice if its been quite a while and you can rely on them not remembering the questions properly to get a practice effect
graymatters 2y ago
Actually - “how many golf balls fit into a VW Beatle” is a better proxy.
- drewcoo 2y ago
  Those are called Fermi questions. https://en.wikipedia.org/wiki/Fermi_problem https://en.wikipedia.org/wiki/Fermi_problem
willio58 2y ago
I don’t know that people generally have a problem with testing their knowledge, I think most people that hate on Leetcode (including myself) just don’t like to code in front of strangers in a timed setting like that. I’ve never been good at tests in school. I probably averaged Cs on tests through college. Projects though? Aced them every time and sometimes got pulled aside and told I was highly exceeding the projects of others in the class. I just do well when I can think alone, or when I can actually work toward a solution with a coworker. I’ve made the companies I work for millions of dollars. But put me in front of a white board and ask me an algorithm question? You’d think I was fresh out of CS101.

geuis 2y ago

I did a lot of leetcode stuff this year. Dozens of interviews, no takers. What got me a job was that I was a solo founder running a business and learning how to make the most of limited resources. Aka I was someone that gets things done and spends the least amount possible to get there. Use the resources you have and show you can create value. Again, it all comes down to show don't tell. Leetcode is valuable as a way to practice and maybe reaffirm skills. It's not useful for hiring in a direct way.

Apocryphon 2y ago

Aren't there supposed AI tools out there for cheating on these standardized tests?

sciencesama 2y ago
For real ??
slashdave 2y ago
There are, but they are easy to detect
- novok 2y ago
  In my experience, I just need the aha hint and I'm usually good. When I looked them up later I was slapping my hand on my head. How do you detect that?

waythenewsgoes 2y ago

I have always seen LeetCode problems as effectively hazing rituals in a job interview setting. A high pressure interview situation simply won’t bring out peak problem-solving capabilities in many people. At best you get some superficial insight into someone’s problem solving methodology, but at worst you filter out otherwise excellent fits for not being able to solve an ultimately inconsequential problem under pressure. LeetCode questions as interview questions are mostly theater. Most people who do well on these aren’t actually “solving them” on the fly from scratch. They just happen to have seen the exact same problem before and retake the steps they’ve memorized to get to the answer. Testing whether or not someone can regurgitate the solution they have memorized to a math problem doesn’t tell you much about how they will perform in a truly novel non-contrived constraint problem scenario, which is generally what most dev work entails. Perhaps if you are working at Bespoke Algorithms ‘R Us, benchmarking this would have more value to your org, but for most dev roles at most companies it is hard to see it as more than a compliance exercise, or maybe even as a tool to weed out those with families that can’t devote the hours/day to LC memorization.

erik_seaberg 2y ago
I don't try to add any pressure, but going on call will require some resilience from each team member.
- waythenewsgoes 2y ago
  As long as you don’t wake me at 3AM to rotate a red-black tree, or find the median of two sorted arrays we should be good
  - ihumanable 2y ago
    Exactly this. As someone that’s served as an oncall engineer for years now, the skills you need to operate a cluster are completely different from the things leetcode tests for.
  - Ekaros 2y ago
    I must remember to ask how many times those problems happen in company's code base in next interview. Or whatever they ask. And if it is more than one, ask if they have libraries for it... You know good development practises and all.

neilv 2y ago

I'm so sick of LeetCode nonsense, I'm inspired to make a denylist of every company and blog and username that promotes it.

dothack 2y ago
Here is an allowlist: https://github.com/poteto/hiring-without-whiteboards https://github.com/poteto/hiring-without-whiteboards

eadwu 2y ago

If you are the type of person who writes a big brain log(n) solution instead of n solution when there is no real reason (compute wise) and the log(n) is a pain in the ass to maintain. I hope I never see you in my team.

slashdave 2y ago
There is a lot to say about this. Due to technical limitations, the Leetcode site uses odd corner cases in their test suite to guide practitioners to specific, bespoke solutions. It's kind of insidious.
beaconify 2y ago
Yes! Just say give me o(log(n)) dollars! Ah now you care about n and k.

deleted 2y ago

[deleted]

chrisbrandow 2y ago

There’s just so much guesswork with these as interview questions. The number of different problem spaces is non-trivial, and understanding the nature of the problem space for use in work is quite different from being able to work out a problem in 30 minutes in front of other people often in a code editor that is unfamiliar. We lack an agreed upon, specific way to evaluate someone’s talent in 30-90 minutes. LC problems are not a terribly efficient way to fix this.

joshdavham 2y ago

I would rather hire someone who has evidence of working on prior projects than someone who can solve brain teasers. There are many people who excel at LeetCode but struggle to build even simple software.

coolThingsFirst 2y ago
It's 2024, you can find tons of people with project experience AND who are good at LC. It's really saturated right now and it's very difficult to stand out.

valenterry 2y ago

> People that never done LC problems probably don’t even know how to implement some data structures. So when they face a difficult problem they will probably just use a brute force approach. You don't need leetcode for that. It's sufficient to talk about datastructures. In fact, that is a much better and actually reasonable thing to do in an in an interview. > I also like these questions because you can ask them to any kind of software developer, from frontend to DevOps. Only those that will apply at your place, which will exclude me if you require a leetcode interview. So, in fact, you will be biased by prefiltering candidates. It's like when companies say "we only hire the top 1%" and I ask "the top 1% of all engineers or the top 1% of the engineers that chose to apply at your company?"

hatthew 2y ago

Coding assessments are definitely valuable in general. What isn't valuable are the class leetcode problems, where you have to use a heap, or DP, or some random useless data structure to solve a problem in O(n) instead of O(n log n) time. How about instead of asking a simple problem that has an obscure solution, ask a complicated problem with an obvious solution, but one that tests their software engineering skills. Make them ask clarifying questions about which of the many edge cases they are expected to handle, and verify which can be ignored. Add additional requirements that make refactoring necessary. Add constraints about the environment and ask how the candidate would handle that.

MarkMarine 2y ago

I have a couple great companies on my resume because I ground these leetcode questions, but now that I’ve been working and in industry for 14 years… I don’t have the time to do the grind unless I’m explicitly looking for a job. The DS&A skills are still there, I use them and I’m glad I have them, but when someone reaches out to me to join their startup out of the blue and then hits me with the leetcode hard question that I haven’t seen in 8 years… well I’ve not passed those interviews. Usually I don’t say “ok, I’ll study for 4 weeks before the rounds” so I go in cold. I never give these type of questions in my interviews of senior/staff+, I build out topical problems for the space I’m actually hiring for, then simplify it down to the interesting bits. I give a ramp, a simple problem, a more complex tweak to the simple problem that needs an interesting data structure (maybe needs a heap or similar) and then another tweak that forces them to abandon that data structure and do something novel. You can also fail this and still get hired. With junior engineers, I’m sorry but I need something that looks like leetcode so I know you put the work in, and I can’t ask the topical questions because they have no frame of reference. These questions are like the common denominator for someone with no real world skills. I need to see that you’re driven and self motivated enough to teach yourself this (probably useless, when is the last time you had to implement merge sort) skill. I also don’t think there is a great alternative. I put an extraordinary amount of work into making my questions that aren’t Leetcode, when they leak I’m heartbroken. I don’t want to just let a fancy school be the decision point, so I need to find a fair way to test. Asking people to do 1-2 days work or pair program… it’s usually caused a lot of dropping out of the funnel. So I would love to hear alternatives that are working for others

choppaface 2y ago
Leetcode can work well for junior / new grad candidates if you help & hint them a bit. For example they probably forgot breadth-first search, but if you give a couple hints about queues and they show tremendous swiftness, then that can be a good sign they’ll learn very quickly on the job. Does not prevent false negatives but reduces the FN rate and gives them what they really want (learning experience).

ihumanable 2y ago

There are thousands, and I’m not exaggerating, literally thousands of leetcode problems. A lot of them require you to make, what to me at least, is some non-obvious clever observation about the problem. Sure the problem is talking about a guy robbing houses, but if you stand on your head and squint just right you’ll realize this is actually a graph cycle detection problem and you should use Floyd’s algorithm to solve it. Because there are thousands of these problems the amount of time it would take someone to become familiar with them is prohibitive. So you are at the mercy of the interviewer, have they picked a super clever one, are they going to be ok with removing duplicates from the answer by tossing stuff in a set or do they want you to pull some dynamic programming out of thin air. It’s the part where you have to divine the trick under pressure that measures nothing of value. I’ve been a professional software engineer for 2 decades. I’ve had times when I’ve been trying to solve some very tricky problem and done research and thought thoughts and come up with pretty clever solutions, or at least I think they are clever. Not once have I had to do this under pressure in a 45 minute time box with someone looking over my shoulder. That’s my objection to leetcode. Sure it’s great if a candidate can recognize that your riddle about topological map rain capture is actually just a restatement of Kolger’s postulate at first glance (a problem and postulate I’ve just made up because I’m not going to wade back into leetcode right now) but that’s an insane thing to optimize for. The vast majority of the problems programmers solve are actually just mapping business domains into code. The most common problems that need solving is taking squishy, incomplete, and contradictory requirements from multiple stakeholders and figuring out what needs to get done. People in the real world are rarely rolling their own data structures, because the red black tree you slap together is going to be infinitely worse than the battle tested highly optimized one you can pull out of the standard library or off your package manager. In my long career I’ve had a handful of occasions to actually build a data structure or solve a problem with some very clever algorithm. And in those cases you don’t really want people shooting from the hip anyways, you would want them to do research and see what prior art exists so they can discover something like Floyd’s algorithm for finding cycles in a unidirectional graph (ok this one is real). It is not clear to me what exactly leetcode tests. My best guess would be your ability to take a disguised questions and convert it into a handful of problem shapes and solve those. But if you grade leetcode like the website does during an interview, expect to lose a lot of perfectly fine candidates along the way.

ifiht 2y ago

Welcome to your leetcode interview in the age of ai: https://leetcodewizard.io/ https://leetcodewizard.io/

ifiht 2y ago
Also I love these comments, thank you all for vindicating my disdain for this practice.

000ooo000 2y ago

>However, I usually trust my intuition quite a lot, and before giving it up I decided to find some strong arguments in favor of Leetcode (LC) interviews, and here they are. Sounds like confirmation bias to me..

bearjaws 2y ago

IMO the worst problem with leetcode challenges are the fact they are 100% in the training set of LLMs. They solve them instantly, and making matters worse they are 100% valueless to a company. What universe do we live in that hiring managers want staff that are 100% as skilled as an LLM but probably not as strong in areas that actually matter? The reality everyone with a brain is going to cheat on these, they are typically pretty early in the interview process and it will hopefully get replaced with a real world test. At my last job we just used example problems that we had seen in the past, usually REST API focused with just enough nuance to make engineers think through it and potentially refactor their code. Then you can ask them specifics of their thought process and get insight to their experience.

danjl 2y ago

A good senior developer should be able to go out to lunch and know if the candidate has the skills and the right cultural match by the end. There's no need for coding tests or whiteboarding problems. I've seen some of the best developers I've worked with in 40 years of coding get turned away by Google and Apple because of their terrible interview processes. The only thing better than going out to lunch is working with the person for a couple of weeks. Ideally this happens on a paid contract basis. Of course our culture makes it more challenging to switch jobs if you want to work together first, but really no other interview process has ever provided good data.

novok 2y ago
At big companies HR and legal will say you need a reproducible, auditable, objective and defined interview process or you open yourself to a whole host of lawsuits, and you will get lawsuits because there is blood to squeeze from that lawsuit cow. Also at large scale you cannot trust the entire corpus of employees since your are well beyond dunbars number, so the process is there to prevent cheater and nepotism clusters forming where random employees hire their incompetent drinking buddies or cousins after a single "lunch". That process works for small companies, and it's an advantage they should leverage, it does not work for the large ones. IMO you need work sample tests as a minimum, which means a coding assignment. Some people are great bullshitters at lunch.
alisey 2y ago
In my experience, the results from those who claim they can judge skills based on a lunch conversation have not been good. I think people who can do it are the exception, not the norm.

jchw 2y ago

> People don’t like LC interviews because they are bad at them, I enjoy that this assertion is made with essentially no attempt to justify it, it's just said matter-of-factly as if it's just obvious truth. Well, let me speak at least for myself: Absolutely not.

dherls 2y ago

> These interviews can identify candidates with strong problem-solving skills and logical reasoning abilities. I would disagree with this premise. Leetcode identifies people who have just finished cramming for Leetcode questions. You don't need logical reasoning abilities to solve Leetcode, just encyclopedic knowledge of algorithms and data structures

MichaelNolan 2y ago

I think the article missed the biggest argument in favor leetcode style interviews - that they are quick and cheap to administer, and relatively unbiased/objective. The reason big companies use these interview methods, is because they have to interview tens of thousands of candidates per year. None of the common alternative interview methods that get tossed around can scale to thousands of applicants.

pajeets 2y ago

my retort to this article is this simple question: if you can look things up in an encyclopedia, why are you forced to recite it? if even low parameter LLM can solve most of these leetcode examples that cost nothing to run, why are we using it to discriminate applicants to measure "how badly do you want the $250k/year" It's almost like the people hiring are optimizing for masochists and there lies the devil, you want hyper-rationality because they are the easiest people to manipulate and brainwash. Highly sensitive and type B personality aren't going to sit around and let them trade their life to destroy or trap large number of the population with addictive algorithms or drone target selection. Matter of fact IQ was invented specifically to filter out the most programmable minds, you wouldn't be able to run a country's intelligence agency without being able to brainwash people or a corporation. The promoters at the helms are usually the exact opposite of the producers, unwavering, unforgiving, uncompromising.

beaconify 2y ago
L1 cache is useful (or maybe good processor design is the better analogy...) Being quick at the command line, using the right tools, can get around VS code quick, can solve the more trivial code problems quick (or quickly critique LLM output or a blog post). It adds up.

etse 2y ago

This article was kind of lazy. It’s a lot of conjecture, a couple anecdotal observations, and some rhetoric for good measure. Basically, the content was a let down from the title. On the other hand, I didn’t know Data Scientist candidates would also be tested with LeetCode questions.

mcslambley 2y ago

These are not strong or well structured counter arguments. The first argument, for instance, reads as premature optimization via hiring practices.

golly_ned 2y ago

None of the points made in this article were original or insightful. Disappointing for an unorthodox position.

coolThingsFirst 2y ago

They are good and I've had nothing but positive experiences through them. In all the tech phone interviews that I've done in my life I have NEVER failed a LC/algorithm related interview. My secret is to try to engage genuinely with the problem. And even when I haven't been able to solve a problem I still would get passed to next round bcs I'd be on the right track. I still don't know what ppl are complaining about. LC have a correct solution and a good interviewer will pass you if you are on a good track. They are also relatively easy to prepare for. Now compare to actual BS which is the requirement to have done work in a particular track that the company is hiring for. Now that's total BS, I can't get 5 years in python in like 2 weeks before interview no matter how motivated I may be for the job.