3 ms·

Generally you don’t get all the resumes at once, this is literally the secretary problem. Unfortunately you need some objective standard of measurement across c

Generally you don’t get all the resumes at once, this is literally the secretary problem. Unfortunately you need some objective standard of measurement across candidates that is static over at least months, and ideally takes a minimum of you and your candidates’ time. Though imperfect, real-time technical questions are a pretty good solution to this problem, especially since a lot of the best engineers prefer them to take-homes. Also, anecdotally after interviewing hundreds of candidates, I’ve found resumes to have a relatively high false positive rate. Surprising numbers of engineers look great on paper but can’t write working code for semi-complex problems when tested, even accounting for interview anxiety.

_qulr 6y ago
> Surprising numbers of engineers look great on paper but can’t write working code when tested, even accounting for interview anxiety. How have you accounted for interview anxiety?
- salt-licker 6y ago
  It’s definitely not an exact science. 1. I try to make the parameters of the interview as friendly as possible — language of their choice, don’t have to worry about code style or computational complexity, or talk while they’re coding, just write working code that solves the problem. They can use google freely. I’m there as a resource to talk through their solution, answer questions, and help them get unstuck if needed. 2. The algorithm to solve the question doesn’t require a trick and there are many viable solutions so most candidates can verbally explain to me a working algorithm quickly, +- a couple edge cases. 3. Candidates often get visibly flustered or hung up on irrelevant details, or unsure what exactly I want for them, and I focus on noticing this. I try to guide them down a correct path if they’re close and tell them not to over-architect if they start doing that. Despite all this, too manly candidates do not have a dependable metaprocess for writing working code - a common scenario is they write code, it gives the wrong result, they drop some print statements to debug, but can’t effectively trace the program to figure out why they’re getting the debug output and so they guess and try to fix the wrong thing, causing further problems. I know that some of this is nerves but it can’t account for all of it.
  - _qulr 6y ago
    > I know that some of this is nerves but it can’t account for all of it. Why not? If you've never experienced severe situational anxiety, you may not know how debilitating it can be. And then, maddeningly, perhaps just minutes after the interview is over, you return to your "normal" self, and start to think about what you would have done in that situation if you weren't nervous. It's basically stage fright, which can be terrifying if you're not used to the stage.
    - salt-licker 6y ago
      I guess I'm generally going off of an interviewee's body language, which isn't entirely fair since I'm not a mind reader. And I'm sure I'm missing some cases of stage fright, which is why again I mentioned all the techniques above that I try to mitigate this, but I agree is a really unfortunate side-effect of the interview process that I don't have a fool-proof solution for. Still, I do have some EQ. A lot of candidates exhibit visible anxiety and then get into a rhythm and calm down when they start coding. Or they visibly calm down once I start asking them questions about how they're thinking and coax them towards viable implementation. On the flip side, a lot of candidates actually exhibit overconfidence running in the wrong direction, which I try to lightly mitigate but can't prevent in all cases without giving an unfair advantage.
wolco 6y ago
Are you sure? I mean the 10 years of success spent over various companies projects on resumes have a high false rate because after they took your coding test they didn't do as well as you thought they should? Did you hire any of them and did they out perform your initial assessment. If you didn't hire them and compare them to new hires who have bad resumes but did well on your test how would you know? You can't objectivity judge a candidate by your test results only. Otherwise your test is just randomly filtering people. Until you do that you don't know how effective your test is. You could be flipping a coin and getting the same results.
- salt-licker 6y ago
  I’ve hired people who have not passed the coding screen (but did pass subsequent interviews) and they’ve been some of my best hires. But the type of failure really matters. Did they write generally correct code but ran out of time because the algorithm they came up with under time pressure was too complicated? Or did they fail to debug a list indexing off-by-one error, even after printing the list repeatedly? It’s sometimes a judgement call but I believe there’s a lot of signal in watching people write code in real-time

_qulr 6y ago

> Surprising numbers of engineers look great on paper but can’t write working code when tested, even accounting for interview anxiety. How have you accounted for interview anxiety?

salt-licker 6y ago
It’s definitely not an exact science. 1. I try to make the parameters of the interview as friendly as possible — language of their choice, don’t have to worry about code style or computational complexity, or talk while they’re coding, just write working code that solves the problem. They can use google freely. I’m there as a resource to talk through their solution, answer questions, and help them get unstuck if needed. 2. The algorithm to solve the question doesn’t require a trick and there are many viable solutions so most candidates can verbally explain to me a working algorithm quickly, +- a couple edge cases. 3. Candidates often get visibly flustered or hung up on irrelevant details, or unsure what exactly I want for them, and I focus on noticing this. I try to guide them down a correct path if they’re close and tell them not to over-architect if they start doing that. Despite all this, too manly candidates do not have a dependable metaprocess for writing working code - a common scenario is they write code, it gives the wrong result, they drop some print statements to debug, but can’t effectively trace the program to figure out why they’re getting the debug output and so they guess and try to fix the wrong thing, causing further problems. I know that some of this is nerves but it can’t account for all of it.
- _qulr 6y ago
  > I know that some of this is nerves but it can’t account for all of it. Why not? If you've never experienced severe situational anxiety, you may not know how debilitating it can be. And then, maddeningly, perhaps just minutes after the interview is over, you return to your "normal" self, and start to think about what you would have done in that situation if you weren't nervous. It's basically stage fright, which can be terrifying if you're not used to the stage.
  - salt-licker 6y ago
    I guess I'm generally going off of an interviewee's body language, which isn't entirely fair since I'm not a mind reader. And I'm sure I'm missing some cases of stage fright, which is why again I mentioned all the techniques above that I try to mitigate this, but I agree is a really unfortunate side-effect of the interview process that I don't have a fool-proof solution for. Still, I do have some EQ. A lot of candidates exhibit visible anxiety and then get into a rhythm and calm down when they start coding. Or they visibly calm down once I start asking them questions about how they're thinking and coax them towards viable implementation. On the flip side, a lot of candidates actually exhibit overconfidence running in the wrong direction, which I try to lightly mitigate but can't prevent in all cases without giving an unfair advantage.

wolco 6y ago

Are you sure? I mean the 10 years of success spent over various companies projects on resumes have a high false rate because after they took your coding test they didn't do as well as you thought they should? Did you hire any of them and did they out perform your initial assessment. If you didn't hire them and compare them to new hires who have bad resumes but did well on your test how would you know? You can't objectivity judge a candidate by your test results only. Otherwise your test is just randomly filtering people. Until you do that you don't know how effective your test is. You could be flipping a coin and getting the same results.

salt-licker 6y ago
I’ve hired people who have not passed the coding screen (but did pass subsequent interviews) and they’ve been some of my best hires. But the type of failure really matters. Did they write generally correct code but ran out of time because the algorithm they came up with under time pressure was too complicated? Or did they fail to debug a list indexing off-by-one error, even after printing the list repeatedly? It’s sometimes a judgement call but I believe there’s a lot of signal in watching people write code in real-time