6 ms·

Depends what you’re hiring for. Firstly the interview should be structured with an aim to maximize correlation between success on the interview and success on

Depends what you’re hiring for. Firstly the interview should be structured with an aim to maximize correlation between success on the interview and success on the job. If you’re not thinking along these lines you aren’t even playing the game. For most startups, the right criteria to screen for is pace and quality of code, plus work ethic. You can ask a relatively easy coding problem, whether it be algorithmic/data structure, or more real world, that 99% of people will be able to solve, but still apply a high bar for what’s considered passing. Can they code the obvious solution quickly and effortlessly? I dont care at all if you’re able to produce a solution, I care about the path to that solution. Screening for the straight arbitrary algorithmic/design diagram problems has hordes of people who have trained to the test. At scale hiring these people may work through selecting for people with decent work ethic to grind problems, but you’re not going to get consistently great people this way. Lots of false negatives and positives. Having hired over 100 engineers in the bay area, judging by coding competency first was always more effective than judging based on quick shot leetcode/algorithmic aptitude. You can easily grind canned algorithms to pass a leetcode style interview, but you can’t fake coding proficiency. If you’re hiring for DB or ML developers, obviously theory is more important. But 90+% of you are not

actuator 4y ago
For startups I agree, they should definitely customize their interview rather than copying big tech pattern. The kind of interview you describe is something Stripe also does, but I have seen even this resulting in non objective selection criterias. Like one Stripe employee told me that they prefer if someone uses a map rather than iterating on a list, as it is cleaner. While I agree with the sentiment; I don't think it is a good filter to add in the interview.
slantedview 4y ago
Can you elaborate on the distinction between coding competency and leetcoding? I agree with you that leetcode is terrible but I don’t quite follow how your suggestion is very different in practice.
- mFixman 4y ago
  LeetCode problems are much simpler than the general coding contest, and they are only used as a basic competency tests. It's like asking for the difference between an Olympic race and a blue-collar job requiring you to be fit enough to run a mile.
- adam_arthur 4y ago
  Coding competency: - Can they code quickly? - Do they know common APIs offhand? (Not have to look up) - Do they write code in a readable fashion? - Do they consider tests? - Do they think more deeply about code structure? Immutability, single source of truth, DRY, not overengineering You can understand all of this by asking a very easy problem that is written in 50-100 LOC. I tend to ask them to implement a certain data structure, where the naive solution is obvious, but still requires ~50 LOC. Then there’s a better runtime solution so I can assess theory as well, but I dont judge primarily on that, or care too much if they figure it out. You may be surprised how much signal you can get out of asking somebody to implement a class with a few APIs and basic loops/data structures. The simple fact that people who don’t train specifically for leetcode, don’t do well at it is effectively proof that it’s weakly correlated with real world work results. Otherwise people who are effective in the real world would automatically do well with leetcode. Speaking more to the harder problems though, the easier problems are much more correlated
isaacfung 4y ago
Recently read a blog post of Erik Bernhardsson on the interviewing process(he has several other interesting posts that approaches hiring as a machine learning problem). https://erikbern.com/2018/05/02/interviewing-is-a-noisy-prediction-problem.html https://erikbern.com/2018/05/02/interviewing-is-a-noisy-pred... He suggested asking many short questions instead of doing long leetcode problem(as the solution often relies on one single insight, it gives you very little information relative to the work involved). He also suggested using code reading >I print out 10-15 code snippets (not just code, but also UNIX commands, regular expressions, SQL queries and many other things). I then go through and ask: what does this snippet do, how does it work, are there any bugs, etc.? Reading code (as opposed to writing) means I can cover a lot of ground extremely quickly, spending no more than a minute or two on each problem.
- adam_arthur 4y ago
  Yeah, asking questions with multiple stages/progression, where the early stages are obvious but still provide signal through coding competency are good. Similarly, many small questions is going to give you a more accurate view than a single hard algorithm problem. There tends to be a trick or specific process to solve leetcode style problems, thus sessions are more black/white on whether the person makes that observation. Very unnatural to usual dev work. Interviews should never feel “hard” even if your threshold for success is high. There’s nothing that requires you to ask hard questions to get good candidates. By definition if your problems are perceived as “hard” then they aren’t assessing people on relevant skills, because a problem that is correlated with real world work shouldn’t be hard for anybody who is coding all day/every day. (Though somebody who is unskilled may find pretty much any problem hard, speaking more broadly here)
- Yoric 4y ago
  I've had an interview during which I was asked to review code. I loved it.
geraldwhen 4y ago
I agree, and I’ve competed at the highest level in programming competitions. Most of my peers there would not be good hires. Top level programming competitions are like top level spelling bees. You need to have as many algorithms memorized as possible in code. That’s right. You need to be able to recite code by memory to pump out algorithm functions as quickly as possible since libs aren’t allowed generally. This isn’t a useful skill.
- FiberBundle 4y ago
  So you've competed at the highest level of programming competitions, but you also seem to be doing work on Upwork [1]? That seems pretty unusual... [1] https://news.ycombinator.com/item?id=34935440#34935783 https://news.ycombinator.com/item?id=34935440#34935783
  - NotOscarWilde 4y ago
    I do not know the OP and I cannot prove or disprove any claims they make about their life, but I can indirectly attest to the following: I was competing around 2008-2010 at the regional level of the ICPC in Central Europe and indeed, our team's approach at the time had some memorization aspects as well. (Our university had a significant amount of support for the competitions, with some coaching as well as a course that consisted of weekly practice contests.) We never won anything, so I would not dare claim we competed at a highest level. As far as I remember, most of our preparation was about "recognition" -- how to tell if a greedy approach is optimal, or how to recognize if a dynamic approach fits. And of course, how to write a program quickly and not forget any corner cases. I remember having daydreams back then of memorizing a max-flow algorithm or potentially even a linear programming solver and then quickly retyping it at a competition. Flows and LPs indeed solve a lot of stuff (LPs are P-complete). I admit I never did that, and it wouldn't be a winning strategy there anyway. PS: Oh, and contrary to the poster above, most of my friends from the university days would be and indeed were great hires, judging by their jobs at Google, Microsoft and elsewhere. Some others, such as the actual ICPC winners from our university, ended up pursuing academic careers -- but I dare not say they would have a bad time in the industry.

actuator 4y ago

For startups I agree, they should definitely customize their interview rather than copying big tech pattern. The kind of interview you describe is something Stripe also does, but I have seen even this resulting in non objective selection criterias. Like one Stripe employee told me that they prefer if someone uses a map rather than iterating on a list, as it is cleaner. While I agree with the sentiment; I don't think it is a good filter to add in the interview.

slantedview 4y ago

Can you elaborate on the distinction between coding competency and leetcoding? I agree with you that leetcode is terrible but I don’t quite follow how your suggestion is very different in practice.

mFixman 4y ago
LeetCode problems are much simpler than the general coding contest, and they are only used as a basic competency tests. It's like asking for the difference between an Olympic race and a blue-collar job requiring you to be fit enough to run a mile.
adam_arthur 4y ago
Coding competency: - Can they code quickly? - Do they know common APIs offhand? (Not have to look up) - Do they write code in a readable fashion? - Do they consider tests? - Do they think more deeply about code structure? Immutability, single source of truth, DRY, not overengineering You can understand all of this by asking a very easy problem that is written in 50-100 LOC. I tend to ask them to implement a certain data structure, where the naive solution is obvious, but still requires ~50 LOC. Then there’s a better runtime solution so I can assess theory as well, but I dont judge primarily on that, or care too much if they figure it out. You may be surprised how much signal you can get out of asking somebody to implement a class with a few APIs and basic loops/data structures. The simple fact that people who don’t train specifically for leetcode, don’t do well at it is effectively proof that it’s weakly correlated with real world work results. Otherwise people who are effective in the real world would automatically do well with leetcode. Speaking more to the harder problems though, the easier problems are much more correlated

isaacfung 4y ago

Recently read a blog post of Erik Bernhardsson on the interviewing process(he has several other interesting posts that approaches hiring as a machine learning problem). https://erikbern.com/2018/05/02/interviewing-is-a-noisy-prediction-problem.html https://erikbern.com/2018/05/02/interviewing-is-a-noisy-pred... He suggested asking many short questions instead of doing long leetcode problem(as the solution often relies on one single insight, it gives you very little information relative to the work involved). He also suggested using code reading >I print out 10-15 code snippets (not just code, but also UNIX commands, regular expressions, SQL queries and many other things). I then go through and ask: what does this snippet do, how does it work, are there any bugs, etc.? Reading code (as opposed to writing) means I can cover a lot of ground extremely quickly, spending no more than a minute or two on each problem.

adam_arthur 4y ago
Yeah, asking questions with multiple stages/progression, where the early stages are obvious but still provide signal through coding competency are good. Similarly, many small questions is going to give you a more accurate view than a single hard algorithm problem. There tends to be a trick or specific process to solve leetcode style problems, thus sessions are more black/white on whether the person makes that observation. Very unnatural to usual dev work. Interviews should never feel “hard” even if your threshold for success is high. There’s nothing that requires you to ask hard questions to get good candidates. By definition if your problems are perceived as “hard” then they aren’t assessing people on relevant skills, because a problem that is correlated with real world work shouldn’t be hard for anybody who is coding all day/every day. (Though somebody who is unskilled may find pretty much any problem hard, speaking more broadly here)
Yoric 4y ago
I've had an interview during which I was asked to review code. I loved it.

geraldwhen 4y ago

I agree, and I’ve competed at the highest level in programming competitions. Most of my peers there would not be good hires. Top level programming competitions are like top level spelling bees. You need to have as many algorithms memorized as possible in code. That’s right. You need to be able to recite code by memory to pump out algorithm functions as quickly as possible since libs aren’t allowed generally. This isn’t a useful skill.

FiberBundle 4y ago
So you've competed at the highest level of programming competitions, but you also seem to be doing work on Upwork [1]? That seems pretty unusual... [1] https://news.ycombinator.com/item?id=34935440#34935783 https://news.ycombinator.com/item?id=34935440#34935783
- NotOscarWilde 4y ago
  I do not know the OP and I cannot prove or disprove any claims they make about their life, but I can indirectly attest to the following: I was competing around 2008-2010 at the regional level of the ICPC in Central Europe and indeed, our team's approach at the time had some memorization aspects as well. (Our university had a significant amount of support for the competitions, with some coaching as well as a course that consisted of weekly practice contests.) We never won anything, so I would not dare claim we competed at a highest level. As far as I remember, most of our preparation was about "recognition" -- how to tell if a greedy approach is optimal, or how to recognize if a dynamic approach fits. And of course, how to write a program quickly and not forget any corner cases. I remember having daydreams back then of memorizing a max-flow algorithm or potentially even a linear programming solver and then quickly retyping it at a competition. Flows and LPs indeed solve a lot of stuff (LPs are P-complete). I admit I never did that, and it wouldn't be a winning strategy there anyway. PS: Oh, and contrary to the poster above, most of my friends from the university days would be and indeed were great hires, judging by their jobs at Google, Microsoft and elsewhere. Some others, such as the actual ICPC winners from our university, ended up pursuing academic careers -- but I dare not say they would have a bad time in the industry.