4 ms·

> They're not great at that purpose, but they're good enough and they generally produce false negatives (smart people fail) rather than false positives (you hir

by NhanH 2y ago

> They're not great at that purpose, but they're good enough and they generally produce false negatives (smart people fail) rather than false positives (you hire an idiot). I've seen this reasoning, but I'm wondering if this is actually true, especially if you also believe the other myth in hiring: most people applying are not qualified for the job. Since most hiring actually requires you to fill the position in the end (in other words, you can't leave it open indefinitely if no one qualifies as eventually the pressure to hire mount up), every false negatives does increase the odd of you hiring a false positives as well. Just take the example at the extreme, if there is only ever one person on Earth qualified for your job and you got a false negatives on him, there is no other way but getting an idiot in the job.

breckenedge 2y ago
In my years of hiring experience, it doesn't hold up. I've hired folks who could leet code all day long, but couldn't develop a feature from scratch, or figure things out without enormous hand holding. I want to hire problem solvers who also know how to code. I stopped doing leet-code interviews about 7 years ago. By all means have people code in interviews, but don’t do leet code.
- FabHK 2y ago
  How do you test whether they're problem solvers that also know how to code, without leet-code interviews?
  - breckenedge 2y ago
    Giving people real world problems and pairing with them on finding a solution.
    - rented_mule 2y ago
      I'm retired now, but I used a variant of this that worked well for me. In my experience, much of what we do as engineers is learn new tools and application domains and apply what we learn to solve problems. So I wanted to test how good a candidate is at leaning and whether they appear to enjoy it (good and enjoy often correlate). We would work together at the whiteboard, with me teaching them the basics of our application domain and our internal distributed computation framework. Then there were a series of 5 increasingly difficult iterations we'd work through. First, do a simple calculation. Now there's a need for a more difficult computation. And an even harder one. Then explain that it needs to run over billions of log lines, and it's I/O bound, so let's make it faster. Finally, a problem that requires them to probe me deeply about the nature of the data and only needs a hand-wavy answer (calculate a median in a single pass with limited memory - not too bad once you extract from me that the samples are all integers between 0 and 500). To the candidate, this looked like iteration driven by evolving requirements, not 5 interview questions. I mostly let them drive, but I jump in now and then to keep things on pace. And I would try to answer any question the same way I would as a teammate / mentor. They don't need to get every problem right. The increasing difficulty is designed so that almost everyone gets something right and something wrong (< 1% gets all the way through the last one). That's an opportunity for me to give supportive feedback to see how they respond. The best responses to feedback yielded something like "oh cool, I can also use that over here to simplify the code". The worst was someone insisting that you can flip the order of addition and division without changing the answer (i.e., order of operations doesn't matter), me saying "I think it does matter, try doing 1/2 + 1/4 in different orders", and them saying "no I have a math degree, I know what I'm talking about." What they do need to do is be able to solve problems collaboratively, extract learning from an eager mentor, and extrapolate from those learnings to solve problems. The best candidates start finding trade-offs across the application domain / tech tooling boundary that I haven't even brought up. One of the downsides is that it takes a while to calibrate as an interviewer, and that calibration is not entirely transferrable across interviewers, as candidates will respond differently to different interviewers. I did this interview thousands of times (my record was 14 times in one day, and frequently 20+ times a week - we were scaling an org from ~50 engineers to over 1,000). I never stopped learning about signals I was seeing, but I was mostly calibrated after a few dozen candidates. Leet code calibration is much more transferrable, but what use is that if it's not measuring what matters? One thing that I think leet code interviews get very wrong is that interviews involve two sides trying to evaluate each other. Wearing people down with leet code is not a good pitch for your work environment. Many candidates told me at the end of my interview that they were shocked at how much fun/educational an interview could be, and many hires said that this interview style was the reason they accepted our offer over more lucrative offers, because they felt they would learn more and have more fun with us.
      - throwaway2037 2y ago
        > One thing that I think leet code interviews get very wrong is that interviews involve two sides trying to evaluate each other. Wearing people down with leet code is not a good pitch for your work environment. This is huge and is underrated. Even when I have (_rarely_) passed the most bruising interview process, my emotional state was so negative towards the company. Part of the OP's emotional state (exhausted / burned-out) is the result of an adversarial interviewing style. With your style, you can gently push people to the edge and see how they perform. Your style is more dynamic, which more closely matches real world working conditions.
- Jensson 2y ago
  Were those juniors with no experience developing features? Leetcode doesn't test if they have experience, you need to check both if you want someone that can do everything. Someone who does well on leetcode is usually easy to teach so they will become good/great in a year or two, but if you don't have that time then go for leetcode+experience just like everyone else is.
  - breckenedge 2y ago
    Leetcode deselected too many good experienced candidates, which is really who I prefer to hire. I have hired good leetcode juniors that did not improve and are no longer working in software.
    - throwaway2037 2y ago
      Did you select candidates with easy / medium / or hard LeetCode problems? Honestly, without studying, people need to be top 5% of candidates to solve hard problems. You can find real algorithms stars with those problems, but you will also pass on a large percentage of "good enough" candidates who can do a medium problem with a bit of hinting.
Aurornis 2y ago
> if you also believe the other myth in hiring: most people applying are not qualified for the job. Depends where you’re looking in the hiring pipeline. If you’re looking at raw applications submitted to a publicly listed job posting, I can say without reservation that the majority of candidates are not qualified. The pre-screening candidate pool is extremely bad on average, especially for remote job listings. Some people spend all day clicking the apply button on every job listing they see. Your post constrains another misunderstanding about the hiring process: The goal isn’t to hire the first person through the system who is good enough to pass some arbitrary threshold. The goal is collect a number of candidates and hire the best one. People hate this reality, but it’s true.
- FabHK 2y ago
  > and hire the best one Or rather: the best fit. You don't need a Turing Award winner to clean up your website.
- NhanH 2y ago
  > Your post constrains another misunderstanding about the hiring process: The goal isn’t to hire the first person through the system who is good enough to pass some arbitrary threshold. The goal is collect a number of candidates and hire the best one. People hate this reality, but it’s true. I definitely do not have any misunderstanding here, and my description of hiring is the same as what you wrote, just in different phrasing: after a period of time, the company has to pick a candidate, presumably the best one by their interview signals. I am disputing the standard claim of "it's better to have false negatives than false positives" by pointing out how that mindset does not work: since you have to choose someone anyway, the whole "avoiding false positives" might not be a thing at all
zeven7 2y ago
> especially if you also believe the other myth in hiring: most people applying are not qualified for the job As someone who’s done a lot of interviews, this is very true for SWE roles.

by NhanH 2y ago

breckenedge 2y ago

In my years of hiring experience, it doesn't hold up. I've hired folks who could leet code all day long, but couldn't develop a feature from scratch, or figure things out without enormous hand holding. I want to hire problem solvers who also know how to code. I stopped doing leet-code interviews about 7 years ago. By all means have people code in interviews, but don’t do leet code.

FabHK 2y ago
How do you test whether they're problem solvers that also know how to code, without leet-code interviews?
- breckenedge 2y ago
  Giving people real world problems and pairing with them on finding a solution.
  - rented_mule 2y ago
    I'm retired now, but I used a variant of this that worked well for me. In my experience, much of what we do as engineers is learn new tools and application domains and apply what we learn to solve problems. So I wanted to test how good a candidate is at leaning and whether they appear to enjoy it (good and enjoy often correlate). We would work together at the whiteboard, with me teaching them the basics of our application domain and our internal distributed computation framework. Then there were a series of 5 increasingly difficult iterations we'd work through. First, do a simple calculation. Now there's a need for a more difficult computation. And an even harder one. Then explain that it needs to run over billions of log lines, and it's I/O bound, so let's make it faster. Finally, a problem that requires them to probe me deeply about the nature of the data and only needs a hand-wavy answer (calculate a median in a single pass with limited memory - not too bad once you extract from me that the samples are all integers between 0 and 500). To the candidate, this looked like iteration driven by evolving requirements, not 5 interview questions. I mostly let them drive, but I jump in now and then to keep things on pace. And I would try to answer any question the same way I would as a teammate / mentor. They don't need to get every problem right. The increasing difficulty is designed so that almost everyone gets something right and something wrong (< 1% gets all the way through the last one). That's an opportunity for me to give supportive feedback to see how they respond. The best responses to feedback yielded something like "oh cool, I can also use that over here to simplify the code". The worst was someone insisting that you can flip the order of addition and division without changing the answer (i.e., order of operations doesn't matter), me saying "I think it does matter, try doing 1/2 + 1/4 in different orders", and them saying "no I have a math degree, I know what I'm talking about." What they do need to do is be able to solve problems collaboratively, extract learning from an eager mentor, and extrapolate from those learnings to solve problems. The best candidates start finding trade-offs across the application domain / tech tooling boundary that I haven't even brought up. One of the downsides is that it takes a while to calibrate as an interviewer, and that calibration is not entirely transferrable across interviewers, as candidates will respond differently to different interviewers. I did this interview thousands of times (my record was 14 times in one day, and frequently 20+ times a week - we were scaling an org from ~50 engineers to over 1,000). I never stopped learning about signals I was seeing, but I was mostly calibrated after a few dozen candidates. Leet code calibration is much more transferrable, but what use is that if it's not measuring what matters? One thing that I think leet code interviews get very wrong is that interviews involve two sides trying to evaluate each other. Wearing people down with leet code is not a good pitch for your work environment. Many candidates told me at the end of my interview that they were shocked at how much fun/educational an interview could be, and many hires said that this interview style was the reason they accepted our offer over more lucrative offers, because they felt they would learn more and have more fun with us.
    - throwaway2037 2y ago
      > One thing that I think leet code interviews get very wrong is that interviews involve two sides trying to evaluate each other. Wearing people down with leet code is not a good pitch for your work environment. This is huge and is underrated. Even when I have (_rarely_) passed the most bruising interview process, my emotional state was so negative towards the company. Part of the OP's emotional state (exhausted / burned-out) is the result of an adversarial interviewing style. With your style, you can gently push people to the edge and see how they perform. Your style is more dynamic, which more closely matches real world working conditions.
Jensson 2y ago
Were those juniors with no experience developing features? Leetcode doesn't test if they have experience, you need to check both if you want someone that can do everything. Someone who does well on leetcode is usually easy to teach so they will become good/great in a year or two, but if you don't have that time then go for leetcode+experience just like everyone else is.
- breckenedge 2y ago
  Leetcode deselected too many good experienced candidates, which is really who I prefer to hire. I have hired good leetcode juniors that did not improve and are no longer working in software.
  - throwaway2037 2y ago
    Did you select candidates with easy / medium / or hard LeetCode problems? Honestly, without studying, people need to be top 5% of candidates to solve hard problems. You can find real algorithms stars with those problems, but you will also pass on a large percentage of "good enough" candidates who can do a medium problem with a bit of hinting.

Aurornis 2y ago

> if you also believe the other myth in hiring: most people applying are not qualified for the job. Depends where you’re looking in the hiring pipeline. If you’re looking at raw applications submitted to a publicly listed job posting, I can say without reservation that the majority of candidates are not qualified. The pre-screening candidate pool is extremely bad on average, especially for remote job listings. Some people spend all day clicking the apply button on every job listing they see. Your post constrains another misunderstanding about the hiring process: The goal isn’t to hire the first person through the system who is good enough to pass some arbitrary threshold. The goal is collect a number of candidates and hire the best one. People hate this reality, but it’s true.

FabHK 2y ago
> and hire the best one Or rather: the best fit. You don't need a Turing Award winner to clean up your website.
NhanH 2y ago
> Your post constrains another misunderstanding about the hiring process: The goal isn’t to hire the first person through the system who is good enough to pass some arbitrary threshold. The goal is collect a number of candidates and hire the best one. People hate this reality, but it’s true. I definitely do not have any misunderstanding here, and my description of hiring is the same as what you wrote, just in different phrasing: after a period of time, the company has to pick a candidate, presumably the best one by their interview signals. I am disputing the standard claim of "it's better to have false negatives than false positives" by pointing out how that mindset does not work: since you have to choose someone anyway, the whole "avoiding false positives" might not be a thing at all

zeven7 2y ago

> especially if you also believe the other myth in hiring: most people applying are not qualified for the job As someone who’s done a lot of interviews, this is very true for SWE roles.