4 ms·

> I talk to the candidate and you can easily ask the right questions to tell who can code and who can't. You can even get a judge of experience level (junior, i

> I talk to the candidate and you can easily ask the right questions to tell who can code and who can't. You can even get a judge of experience level (junior, intermediate, senior) which you won't get from a code test. I also think I can do this, but I believe most people (myself included) introduce a lot of bias when doing this. I've spoken to developers who in 15 minutes I might have written off as not very qualified, but after hours/days have realised are great at what they do, and very experienced in types of software engineering that I have had less exposure to. I think we have to be careful when judging candidates with what amounts to "gut feeling" like this, as it's typically biased, and not aligned within the hiring team.

sgift 7y ago
The thing is: The method described in the post has the same problem. It just doesn't look like it. Most of these types of interview templates are designed by one or two persons and then blindly applied. Sometimes you have a view modifications, but for the most part it's fixed. Since every interviewee gets the same set people then start to think this is more scientific than "gut feeling" - it is not, but instead of your own biases you have the biases of the template designer. There are probably a few companies which extensively study if their interview process for the people hired (the only subset they have access to, which already is a problem) actually matches their job performance and then try to readjust the process, but it doesn't seem to happen very often. This also gets us back to the old problem that we have no idea what a good programmer is or how to evaluate job performance.
codeduck 7y ago
I find that asking "What is the worst bug you've ever encountered" is a great sounding board.
- foldr 7y ago
  I think that question can be great, but it also puts a lot of pressure on the candidate to execute a rather complicated query on the spot. I've definitely encountered lots of interesting bugs in my time, but my initial reaction to this question is a total mind blank. Especially since "worst" could mean a lot of things, and what I really want is "worst bug I've encountered that I can say something interesting and impressive about". I'd say it's not a bad idea to forward questions like that to the candidate in advance of the interview. That way, you avoid false negatives from people who just couldn't think of a good answer under interview pressure.
  - karmelapple 7y ago
    Would “what is one of the most interesting bugs you’ve ever encountered?” allay your concerns? I think that’s the main intent of the question above.
    - foldr 7y ago
      I think that's a better way of phrasing the question. However, I think my real issue is just that it's difficult to rummage through your mental database of past bugs in the space of a few seconds. (Realistically, it's awkward to pause for more than a few seconds before answering a question in a face-to-face interview.) What would be the downside to giving the candidate a heads-up that you're going to ask this question?
    - vageli 7y ago
      > Would “what is one of the most interesting bugs you’ve ever encountered?” allay your concerns? I think that’s the main intent of the question above. I prefer the even more casual, "What's an interesting bug you've been bitten by?"
      - pmiller2 7y ago
        I think I like this variation the best, because it opens things up to bugs in tools that one has used, and other kinds of bugs that aren’t necessarily ones the candidate would have had to attempt to fix.
    - mike_hearn 7y ago
      (I wrote the article) I never ask this sort of question because: 1. I'd struggle to answer it myself despite nearly three decades of fixing bugs. I could probably come up with some interesting ones after a few minutes of thinking, but most of the bugs that'd immediately pop to mind would just be recent bugs that were hard to explain without knowledge of the codebase. 2. There's no way to guide an interviewer on how to mark this question beyond gut feel. What exactly does a good answer to this look like? More importantly: what does a bad answer look like? Asking questions which are impossible to actually fail at, or for which the outcome is basically random and unjustifiable, is a very common failure mode for interviewers. 3. It tells you nothing about the candidate because they can easily just repeat a bug they read about on a blog last week. You can't check they actually debugged it themselves. For instance I just searched for "most interesting software bug" and clicked the first relevant link, the second story on this Quora post could easily just be repeated verbatim to an interviewer over the phone: https://www.quora.com/What-is-the-most-interesting-software-bug-you-have-ever-encountered-or-heard-of https://www.quora.com/What-is-the-most-interesting-software-... If the interviewer wasn't alert they might be fooled by it. Candidates will quietly Google for answers during talking questions whilst trying to avoid the interviewer noticing, and many other things. That's why I emphasise that designing good interview questions is quite hard.
- skybrian 7y ago
  Maybe, but this is something I'm very bad at. I forget about problems once they're solved.

sgift 7y ago

The thing is: The method described in the post has the same problem. It just doesn't look like it. Most of these types of interview templates are designed by one or two persons and then blindly applied. Sometimes you have a view modifications, but for the most part it's fixed. Since every interviewee gets the same set people then start to think this is more scientific than "gut feeling" - it is not, but instead of your own biases you have the biases of the template designer. There are probably a few companies which extensively study if their interview process for the people hired (the only subset they have access to, which already is a problem) actually matches their job performance and then try to readjust the process, but it doesn't seem to happen very often. This also gets us back to the old problem that we have no idea what a good programmer is or how to evaluate job performance.

codeduck 7y ago

I find that asking "What is the worst bug you've ever encountered" is a great sounding board.

foldr 7y ago
I think that question can be great, but it also puts a lot of pressure on the candidate to execute a rather complicated query on the spot. I've definitely encountered lots of interesting bugs in my time, but my initial reaction to this question is a total mind blank. Especially since "worst" could mean a lot of things, and what I really want is "worst bug I've encountered that I can say something interesting and impressive about". I'd say it's not a bad idea to forward questions like that to the candidate in advance of the interview. That way, you avoid false negatives from people who just couldn't think of a good answer under interview pressure.
- karmelapple 7y ago
  Would “what is one of the most interesting bugs you’ve ever encountered?” allay your concerns? I think that’s the main intent of the question above.
  - foldr 7y ago
    I think that's a better way of phrasing the question. However, I think my real issue is just that it's difficult to rummage through your mental database of past bugs in the space of a few seconds. (Realistically, it's awkward to pause for more than a few seconds before answering a question in a face-to-face interview.) What would be the downside to giving the candidate a heads-up that you're going to ask this question?
  - vageli 7y ago
    > Would “what is one of the most interesting bugs you’ve ever encountered?” allay your concerns? I think that’s the main intent of the question above. I prefer the even more casual, "What's an interesting bug you've been bitten by?"
    - pmiller2 7y ago
      I think I like this variation the best, because it opens things up to bugs in tools that one has used, and other kinds of bugs that aren’t necessarily ones the candidate would have had to attempt to fix.
  - mike_hearn 7y ago
    (I wrote the article) I never ask this sort of question because: 1. I'd struggle to answer it myself despite nearly three decades of fixing bugs. I could probably come up with some interesting ones after a few minutes of thinking, but most of the bugs that'd immediately pop to mind would just be recent bugs that were hard to explain without knowledge of the codebase. 2. There's no way to guide an interviewer on how to mark this question beyond gut feel. What exactly does a good answer to this look like? More importantly: what does a bad answer look like? Asking questions which are impossible to actually fail at, or for which the outcome is basically random and unjustifiable, is a very common failure mode for interviewers. 3. It tells you nothing about the candidate because they can easily just repeat a bug they read about on a blog last week. You can't check they actually debugged it themselves. For instance I just searched for "most interesting software bug" and clicked the first relevant link, the second story on this Quora post could easily just be repeated verbatim to an interviewer over the phone: https://www.quora.com/What-is-the-most-interesting-software-bug-you-have-ever-encountered-or-heard-of https://www.quora.com/What-is-the-most-interesting-software-... If the interviewer wasn't alert they might be fooled by it. Candidates will quietly Google for answers during talking questions whilst trying to avoid the interviewer noticing, and many other things. That's why I emphasise that designing good interview questions is quite hard.
skybrian 7y ago
Maybe, but this is something I'm very bad at. I forget about problems once they're solved.