9 ms·

> Do coding interviews work in the actual world in which we live in?" No, fundamentally not. I look forward to seeing you disrupt every major tech company in t

by dlkf 4y ago

> Do coding interviews work in the actual world in which we live in?" No, fundamentally not. I look forward to seeing you disrupt every major tech company in the world via your superior hiring strategy. > EVEN IF THE CANDIDATE CAN IN FACT PROGRAM JUST FINE. You don’t seem to have thought very hard about what employers are optimising for. Everyone realises that programming interviews produce lots of false negatives. Even the strongest programmers have bad days. From the employer’s point of view, that’s okay - minimizing false negatives is not the goal. The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.

chrisweekly 4y ago
But the adversarial puzzles and whiteboard programming are not "the job". Hire for strengths, not to avoid weaknesses.
- Ntrails 4y ago
  So how do I effectively test for the strengths I want with a low likelihood of false positives? Ideally without an increase of time/cost requirements.
- dlkf 4y ago
  > But the adversarial puzzles and whiteboard programming are not "the job". This is true. They are closer to ”the rare, most intellectually difficult parts of the job.” A hockey player typically spends 70% of each game sitting on the bench; yet this isn’t what is emphasized in tryouts. > Hire for strengths, not to avoid weaknesses. I guess you didn’t read the part of my comment where I wrote: 1. ”Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.” This is the default position of large tech companies who do coding interviews, and especially those like Google and Microsoft who interview with very difficult discrete math problems. If we assume 1, the hiring practices of Google/Microsoft make perfect sense. They are not trying to hire all the extremely talented candidates. They are trying to ensure that all the hires are extremely talented. Put differently, they are optimising for precision rather than recall. It seems like you disagree with 1. And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing. But my default position is scepticism, because if it was possible it would probably already be happening.
  - lapcat 4y ago
    > And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing. Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage. We can get into a discussion/argument about what is the competitive advantage of Google/Microsoft/Apple (which collectively own near 100% of the market for mobile OS, desktop OS, web browser, search, etc.), but that's kind of a tangent IMO. I will say that Google/Microsoft/Apple have a competitive advantage specifically in hiring (that is, getting many of the top candidates to apply) because of their prestige (a product of the aforementioned market dominance) and vast wealth. The latter in particular allows them to offer more compensation, especially stock compensation, than other companies. It's important to mention though that the formalized hiring practices of these companies are largely ex post facto and could only be put in place after they were already successful. It's unlikely they hired exactly in the same way during the early years. Steve Jobs was introduced to Steve Wozniack by a friend in high school! Nobody can plausibly claim that the success of the Apple II was due to "coding interviews". Larry Page and Sergey Brin met in college, where they were already working on the ideas that led to Google. Hiring doesn't just turn into success, that's not a reliable formula. You can set the bar as high as you like, but you won't be able to hire anyone if candidates just walk away from your obstacle course. Talented people put themselves through a lot of crap to work at BigCos because there's a big payoff working for a BigCo. That's what tends to make the BigCo hiring pool better than for other companies, not the hiring bar itself. You can afford to be extremely picky if you have a wealth of options to pick from.
    - robertlagrant 4y ago
      > Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage. You're conflating some things here. Without unpicking it all, the main thing I'd say is that you've only demonstrated that amazing hiring is not sufficient for competitive advantage. You have not demonstrated that it's not necessary.
BlargMcLarg 4y ago
The question is whether they are actually decreasing the number of false positives. So far I've not really seen evidence of this beyond some mere anecdotes. For such a bold claim, one would expect evidence to be readily available. Which becomes even worse when realizing "false positive/true positive" is too binary of a definition for a spectrum such as job performance. And I say this as someone who's in favor of giving people a quick check on whether they can actually do FizzBuzz.
- dlkf 4y ago
  > For such a bold claim, one would expect evidence to be readily available. The only way to provide this evidence would be to to conduct an RCT in which you hire some percentage of people who fail the test as a control group. The nascent psychologist in me loves this idea, but it seems unethical, and presumably this is why nobody does it. But I disagree that we need ”evidence.” It’s flat out obvious. If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job.
  - BlargMcLarg 4y ago
    >If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job. You're also maximizing the odds they will get bored and feel baited. With all due respect, you could have picked a better example to illustrate your point. This is exactly why you can't make such bold claims. You're looking at this from one perspective without considering all the variables which go into the workplace equation. So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place. For various reasons, this isn't really happening in software development, making people all to eager to grasp at proxies which they swear will do the same thing. Maybe whiteboarding is in line with the day-to-day at FAANGs, but tons of companies ask questions which are not in line with the day-to-day at all. Both vertically and horizontally. It creates dissatisfaction, burnout, bad cultures, sometimes even animosity, as if hiring managers are playing pranks on candidates. If you're only looking at this from the perspective of "get someone who can do the job", you're going to miss the forest for the trees. As others say, the company is also being interviewed.
fumeux_fume 4y ago
Maybe you're the enlightened one here or maybe you're just rationalizing an awful and lazy part of the hiring process for many companies.
- dlkf 4y ago
  If anyone actually believed this they wouldn’t complain about it on HackerNews. If it were true that Google, Microsoft, Apple, etc don’t know how to hire programmers effectively, this would be an incredibly powerful observation which you would want to keep top secret while silently beating the hiring market and building the most successful tech company of all time. We’re all waiting. The reason people are upset by this issue is that it highlights a misalignment in the incentives of employers and job seekers. And with this I fully empathize: job hunting is hell in any industry, ours included. But we can’t begin to have a conversation about improving the situation until we have a rudimentary understanding of what is happening today. And all these ”Google doesn’t know how to hire smart people” takes are brutally handicapping the discourse.
  - lapcat 4y ago
    > If it were true that Google, Microsoft, Apple, etc don’t know how to hire programmers effectively, this would be an incredibly powerful observation which you would want to keep top secret while silently beating the hiring market and building the most successful tech company of all time. Well, I'd like to have a word with the Apple Mac Mail app team. And the Disk Utility team. And some others...
tony 4y ago
> You don’t seem to have thought very hard about what employers are optimising for. Have you ever seen a hiring process that's simply not effective? The undercurrent of this irks me a bit. It assumes a kind of omniscience on part of the employer, and enough time for them to develop a good interview process. I think this is really hard to do. I think you're generally right. And a lot of the time, employers are empathetic, pragmatic, self-aware and really are doing us a favor when they deny us. Some take into account I-O psychology stuff, too. I (and others probably) are eager to hear what companies you feel do a good job at hiring. > Everyone realises that programming interviews produce lots of false negatives. Let's assume a bad scenario. A company can disregard evidence of competency, like having a portfolio, to basically have a rolling, year round tech Olympiad. To them, it's about fronting as a strong candidate in events, even at the expense of being good at the role. You can be a pro at pole vault or discus throwing, but a poor carpenter. But there's a mentality out there - with some - that a strong, fit candidate can handle any technical challenge. Can a chemist be a great chef? I bet, I also bet some could overthink and/or overengineer preparing a simple meal. Does the interview process do anything to check for soft skills, like curiosity, simplicity, etc? Hopefully. > The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job. I think some companies - not all - can create a "squid game" contest out of candidates. Worse, I feel candidates can unduly suffer and be humiliated in these cases. The company gets a superb interviewer, who is no doubt a very intelligent person, but not necessarily one that can adapt to technical reality, contingencies, ambiguities, etc. Some interview processes end up focusing on theoretical and rote knowledge, not integration and synthesis, when they deeply need that. Turnover happens. But we're back to the beginning, did the company: 1. really know what they needed for that position, 2. interview process check for that? My guess is when you wrote that you were assuming situations where both were true?

by dlkf 4y ago

chrisweekly 4y ago

But the adversarial puzzles and whiteboard programming are not "the job". Hire for strengths, not to avoid weaknesses.

Ntrails 4y ago
So how do I effectively test for the strengths I want with a low likelihood of false positives? Ideally without an increase of time/cost requirements.
dlkf 4y ago
> But the adversarial puzzles and whiteboard programming are not "the job". This is true. They are closer to ”the rare, most intellectually difficult parts of the job.” A hockey player typically spends 70% of each game sitting on the bench; yet this isn’t what is emphasized in tryouts. > Hire for strengths, not to avoid weaknesses. I guess you didn’t read the part of my comment where I wrote: 1. ”Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job.” This is the default position of large tech companies who do coding interviews, and especially those like Google and Microsoft who interview with very difficult discrete math problems. If we assume 1, the hiring practices of Google/Microsoft make perfect sense. They are not trying to hire all the extremely talented candidates. They are trying to ensure that all the hires are extremely talented. Put differently, they are optimising for precision rather than recall. It seems like you disagree with 1. And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing. But my default position is scepticism, because if it was possible it would probably already be happening.
- lapcat 4y ago
  > And if you’re right, then you have a huge opportunity, because this suggests you can disrupt any vertical where the incumbent does the LeetCode thing. Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage. We can get into a discussion/argument about what is the competitive advantage of Google/Microsoft/Apple (which collectively own near 100% of the market for mobile OS, desktop OS, web browser, search, etc.), but that's kind of a tangent IMO. I will say that Google/Microsoft/Apple have a competitive advantage specifically in hiring (that is, getting many of the top candidates to apply) because of their prestige (a product of the aforementioned market dominance) and vast wealth. The latter in particular allows them to offer more compensation, especially stock compensation, than other companies. It's important to mention though that the formalized hiring practices of these companies are largely ex post facto and could only be put in place after they were already successful. It's unlikely they hired exactly in the same way during the early years. Steve Jobs was introduced to Steve Wozniack by a friend in high school! Nobody can plausibly claim that the success of the Apple II was due to "coding interviews". Larry Page and Sergey Brin met in college, where they were already working on the ideas that led to Google. Hiring doesn't just turn into success, that's not a reliable formula. You can set the bar as high as you like, but you won't be able to hire anyone if candidates just walk away from your obstacle course. Talented people put themselves through a lot of crap to work at BigCos because there's a big payoff working for a BigCo. That's what tends to make the BigCo hiring pool better than for other companies, not the hiring bar itself. You can afford to be extremely picky if you have a wealth of options to pick from.
  - robertlagrant 4y ago
    > Only if you view hiring as the competitive advantage of Google/Microsoft/Apple. But everyone cargo cults the hiring practices of Google/Microsoft/Apple without becoming Google/Microsoft/Apple, which shows that it's not actually some magical unique competitive advantage. You're conflating some things here. Without unpicking it all, the main thing I'd say is that you've only demonstrated that amazing hiring is not sufficient for competitive advantage. You have not demonstrated that it's not necessary.

BlargMcLarg 4y ago

The question is whether they are actually decreasing the number of false positives. So far I've not really seen evidence of this beyond some mere anecdotes. For such a bold claim, one would expect evidence to be readily available. Which becomes even worse when realizing "false positive/true positive" is too binary of a definition for a spectrum such as job performance. And I say this as someone who's in favor of giving people a quick check on whether they can actually do FizzBuzz.

dlkf 4y ago
> For such a bold claim, one would expect evidence to be readily available. The only way to provide this evidence would be to to conduct an RCT in which you hire some percentage of people who fail the test as a control group. The nascent psychologist in me loves this idea, but it seems unethical, and presumably this is why nobody does it. But I disagree that we need ”evidence.” It’s flat out obvious. If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job.
- BlargMcLarg 4y ago
  >If you give your job candidates a discrete math problem orders more difficult than the usual ticket, you minimize the risk of hiring someone who can’t do the job. You're also maximizing the odds they will get bored and feel baited. With all due respect, you could have picked a better example to illustrate your point. This is exactly why you can't make such bold claims. You're looking at this from one perspective without considering all the variables which go into the workplace equation. So far, the best correlator to job performance is giving people work corresponding to what they will be doing at the work place. For various reasons, this isn't really happening in software development, making people all to eager to grasp at proxies which they swear will do the same thing. Maybe whiteboarding is in line with the day-to-day at FAANGs, but tons of companies ask questions which are not in line with the day-to-day at all. Both vertically and horizontally. It creates dissatisfaction, burnout, bad cultures, sometimes even animosity, as if hiring managers are playing pranks on candidates. If you're only looking at this from the perspective of "get someone who can do the job", you're going to miss the forest for the trees. As others say, the company is also being interviewed.

fumeux_fume 4y ago

Maybe you're the enlightened one here or maybe you're just rationalizing an awful and lazy part of the hiring process for many companies.

dlkf 4y ago
If anyone actually believed this they wouldn’t complain about it on HackerNews. If it were true that Google, Microsoft, Apple, etc don’t know how to hire programmers effectively, this would be an incredibly powerful observation which you would want to keep top secret while silently beating the hiring market and building the most successful tech company of all time. We’re all waiting. The reason people are upset by this issue is that it highlights a misalignment in the incentives of employers and job seekers. And with this I fully empathize: job hunting is hell in any industry, ours included. But we can’t begin to have a conversation about improving the situation until we have a rudimentary understanding of what is happening today. And all these ”Google doesn’t know how to hire smart people” takes are brutally handicapping the discourse.
- lapcat 4y ago
  > If it were true that Google, Microsoft, Apple, etc don’t know how to hire programmers effectively, this would be an incredibly powerful observation which you would want to keep top secret while silently beating the hiring market and building the most successful tech company of all time. Well, I'd like to have a word with the Apple Mac Mail app team. And the Disk Utility team. And some others...

tony 4y ago

> You don’t seem to have thought very hard about what employers are optimising for. Have you ever seen a hiring process that's simply not effective? The undercurrent of this irks me a bit. It assumes a kind of omniscience on part of the employer, and enough time for them to develop a good interview process. I think this is really hard to do. I think you're generally right. And a lot of the time, employers are empathetic, pragmatic, self-aware and really are doing us a favor when they deny us. Some take into account I-O psychology stuff, too. I (and others probably) are eager to hear what companies you feel do a good job at hiring. > Everyone realises that programming interviews produce lots of false negatives. Let's assume a bad scenario. A company can disregard evidence of competency, like having a portfolio, to basically have a rolling, year round tech Olympiad. To them, it's about fronting as a strong candidate in events, even at the expense of being good at the role. You can be a pro at pole vault or discus throwing, but a poor carpenter. But there's a mentality out there - with some - that a strong, fit candidate can handle any technical challenge. Can a chemist be a great chef? I bet, I also bet some could overthink and/or overengineer preparing a simple meal. Does the interview process do anything to check for soft skills, like curiosity, simplicity, etc? Hopefully. > The goal is minimizing false positives. Failing to hire a strong candidate carries a much lower cost than hiring someone who can’t do the job. I think some companies - not all - can create a "squid game" contest out of candidates. Worse, I feel candidates can unduly suffer and be humiliated in these cases. The company gets a superb interviewer, who is no doubt a very intelligent person, but not necessarily one that can adapt to technical reality, contingencies, ambiguities, etc. Some interview processes end up focusing on theoretical and rote knowledge, not integration and synthesis, when they deeply need that. Turnover happens. But we're back to the beginning, did the company: 1. really know what they needed for that position, 2. interview process check for that? My guess is when you wrote that you were assuming situations where both were true?