3 ms·

At the company I work at when we interview we tend to ask tougher and tougher questions until the candidate either starts spouting BS or replies that they don't

by Cerium 11y ago

At the company I work at when we interview we tend to ask tougher and tougher questions until the candidate either starts spouting BS or replies that they don't know. BS is a no-hire.

pklausler 11y ago
An ability to state "I don't know that" without shame is a strong positive signal of intellectual maturity, but I'm not sure that I'd want to press it out of a candidate by asking an impossible question, if it had to get to that point. The process that you describe isn't one that I would feel comfortable using, but I like the objective.
- solipsism 11y ago
  No one said "impossible". Just harder and harder. I think this is okay, especially if the interviewee knows that's the plan. Make sure they shed the notion that they're expected to know everything that is asked of them, let them know that your plan is to challenge them. A developer who can't accept a challenge that stretches his/her boundaries a little bit is probably not a developer you want.
  - pklausler 11y ago
    I completely agree that I want to hire programmers who want to constantly learn and improve, but I'm not sure that I should expect them to do so during a stressful high-stakes interview.
    - timv 11y ago
      I've had good success when we introduce the conversation with "We're going to ask a bunch of questions across a variety of problem domains. We don't expect anyone to be strong in all these areas, but we want to get a feel for where your strengths are, and then we'll drill down into those areas" But, once you do that setup, everyone is eventually going to say that they don't know what they don't know. It's exceptionally rare for someone to be able to delve into the implementation details of a couple of different programming languages (of our choosing), cryptography, distributed systems, relational modelling, CS theory, and a variety of algorithms and data structures. So "I don't know" ceases to be much of a signal, since everyone says it a few times. In fact we prefer to see the opposite, where people are willing to tell us the bits of info that they do know. Even if you don't know how two phase commit is implemented, knowing what problem it's trying to solve will allow you to research it when the need arises. Because an interview situation is so unlike a normal workplace environment, I don't know any reliable way to translate their willingness (or otherwise) to say "I don't know" in the interview, to on the job behaviours. Either you'll coach everyone into it, or you'll be letting the interview pressure (potentiality) distort their behaviour so much that it's too hard to derive a signal from.
- lsiebert 11y ago
  The thing is, at the edges of people's knowledge there are two issues. One is, people want to demonstrate what they know about the topic, even if they can't answer the specific question. If you ask me to explain elliptic curve cryptography, I can tell you that it's useful because it uses smaller keys then prime based RSA crypto, and that's something I'd want to mention to indicate that, despite the fact that I'd need to look at a book or writeup to actually implement elliptic curve cryptography and I'm not sure I can verbally explain the math behind it, I at least know something about it. The second is, at the limits of my domain knowledge, I may not know what I know and don't know. Limits are fuzzy, and I may feel like I have a pretty good understanding of a linux file system from a user perspective (permissions, inodes, etc) but I couldn't implement one off the top of my head and I'm probably missing some important components. As an interviewee, I may have incorrect or incomplete knowledge, but not know that. That doesn't mean I'm bullshitting intentionally, I'm just trying my best to answer the question I think you are asking. Also I have to think that other developer cultures may have different values around these ideas then my own. Maybe the cultural expectation is to show confidence in my ability; I know that confident individuals are more likely to get hired or even apply for jobs, and that often job requirements are a screen for confidence. Sure, I'd love if I knew every gap in my knowledge and every mistaken understanding I have, but I don't know how realistic it is to suggest a deliberate deception on the edge of a person's knowledge. And that's setting aside whether the interview process is a good predictor of workplace behavior... I think we all know people who are excellent devs that can't write on a white board for shit, or who sit down for a good think for an hour and then come up with a perfect solution but aren't going to be able to write and simultaneously talk you through a solution to a 15 minute data algorithm question.
  - pklausler 11y ago
    Knowing what you don't know is pretty important, because then you know when you have to go look something up rather than rely on your memory. When I have to call mmap(2), I know that I have to read "man mmap" first, because I know that I just can't remember the order of the five arguments, or if that's even the right number. I've been working with UNIX type systems for 30 years or more, and have worked on the guts of three different kernels on many different architectures, but when I say that I know UNIX, I think that I mean that I really know where the stuff I need to know is described in the man pages.
whitegrape 11y ago
If you're going to do this level of hazing, save yourself some time and just have the candidates compete in a big Jeopardy match. Those that don't know will stay silent, those that know will answer correctly, those that don't know but answer anyway will be punished more than if they stayed silent. Personally I'm pretty tolerant of a lot of BS, depending on the person. Sometimes it's just a cognitive quirk that someone uses BS to search around their brain see if they can trigger some association that lets them remember, or discover that they really don't know. Saying "I don't know" immediately is often the wrong thing to say (especially in interview settings) when the real answer is "I don't remember, let me rediscover / look that up", but not many interviewers let you look things up or will mark you down if you say you'd need a reference for X though you studied X some years ago in college like everyone else and might work it out (after wading through a bunch of BS and false-starts in general) if you had unlimited time and no pressure...
Obi_Juan_Kenobi 11y ago
This is basically how PhD qualifying exams work, as well as the defense to an extent. You're welcome to guess or describe how you'd think about something, but you should be willing to clearly state when you're really out of your depth. It's important to note, though, that this isn't done adversarially; it's usually just a natural conversation that goes down a particular rabbit hole.

by Cerium 11y ago

At the company I work at when we interview we tend to ask tougher and tougher questions until the candidate either starts spouting BS or replies that they don't know. BS is a no-hire.

pklausler 11y ago

An ability to state "I don't know that" without shame is a strong positive signal of intellectual maturity, but I'm not sure that I'd want to press it out of a candidate by asking an impossible question, if it had to get to that point. The process that you describe isn't one that I would feel comfortable using, but I like the objective.

solipsism 11y ago
No one said "impossible". Just harder and harder. I think this is okay, especially if the interviewee knows that's the plan. Make sure they shed the notion that they're expected to know everything that is asked of them, let them know that your plan is to challenge them. A developer who can't accept a challenge that stretches his/her boundaries a little bit is probably not a developer you want.
- pklausler 11y ago
  I completely agree that I want to hire programmers who want to constantly learn and improve, but I'm not sure that I should expect them to do so during a stressful high-stakes interview.
  - timv 11y ago
    I've had good success when we introduce the conversation with "We're going to ask a bunch of questions across a variety of problem domains. We don't expect anyone to be strong in all these areas, but we want to get a feel for where your strengths are, and then we'll drill down into those areas" But, once you do that setup, everyone is eventually going to say that they don't know what they don't know. It's exceptionally rare for someone to be able to delve into the implementation details of a couple of different programming languages (of our choosing), cryptography, distributed systems, relational modelling, CS theory, and a variety of algorithms and data structures. So "I don't know" ceases to be much of a signal, since everyone says it a few times. In fact we prefer to see the opposite, where people are willing to tell us the bits of info that they do know. Even if you don't know how two phase commit is implemented, knowing what problem it's trying to solve will allow you to research it when the need arises. Because an interview situation is so unlike a normal workplace environment, I don't know any reliable way to translate their willingness (or otherwise) to say "I don't know" in the interview, to on the job behaviours. Either you'll coach everyone into it, or you'll be letting the interview pressure (potentiality) distort their behaviour so much that it's too hard to derive a signal from.
lsiebert 11y ago
The thing is, at the edges of people's knowledge there are two issues. One is, people want to demonstrate what they know about the topic, even if they can't answer the specific question. If you ask me to explain elliptic curve cryptography, I can tell you that it's useful because it uses smaller keys then prime based RSA crypto, and that's something I'd want to mention to indicate that, despite the fact that I'd need to look at a book or writeup to actually implement elliptic curve cryptography and I'm not sure I can verbally explain the math behind it, I at least know something about it. The second is, at the limits of my domain knowledge, I may not know what I know and don't know. Limits are fuzzy, and I may feel like I have a pretty good understanding of a linux file system from a user perspective (permissions, inodes, etc) but I couldn't implement one off the top of my head and I'm probably missing some important components. As an interviewee, I may have incorrect or incomplete knowledge, but not know that. That doesn't mean I'm bullshitting intentionally, I'm just trying my best to answer the question I think you are asking. Also I have to think that other developer cultures may have different values around these ideas then my own. Maybe the cultural expectation is to show confidence in my ability; I know that confident individuals are more likely to get hired or even apply for jobs, and that often job requirements are a screen for confidence. Sure, I'd love if I knew every gap in my knowledge and every mistaken understanding I have, but I don't know how realistic it is to suggest a deliberate deception on the edge of a person's knowledge. And that's setting aside whether the interview process is a good predictor of workplace behavior... I think we all know people who are excellent devs that can't write on a white board for shit, or who sit down for a good think for an hour and then come up with a perfect solution but aren't going to be able to write and simultaneously talk you through a solution to a 15 minute data algorithm question.
- pklausler 11y ago
  Knowing what you don't know is pretty important, because then you know when you have to go look something up rather than rely on your memory. When I have to call mmap(2), I know that I have to read "man mmap" first, because I know that I just can't remember the order of the five arguments, or if that's even the right number. I've been working with UNIX type systems for 30 years or more, and have worked on the guts of three different kernels on many different architectures, but when I say that I know UNIX, I think that I mean that I really know where the stuff I need to know is described in the man pages.

whitegrape 11y ago

If you're going to do this level of hazing, save yourself some time and just have the candidates compete in a big Jeopardy match. Those that don't know will stay silent, those that know will answer correctly, those that don't know but answer anyway will be punished more than if they stayed silent. Personally I'm pretty tolerant of a lot of BS, depending on the person. Sometimes it's just a cognitive quirk that someone uses BS to search around their brain see if they can trigger some association that lets them remember, or discover that they really don't know. Saying "I don't know" immediately is often the wrong thing to say (especially in interview settings) when the real answer is "I don't remember, let me rediscover / look that up", but not many interviewers let you look things up or will mark you down if you say you'd need a reference for X though you studied X some years ago in college like everyone else and might work it out (after wading through a bunch of BS and false-starts in general) if you had unlimited time and no pressure...

Obi_Juan_Kenobi 11y ago

This is basically how PhD qualifying exams work, as well as the defense to an extent. You're welcome to guess or describe how you'd think about something, but you should be willing to clearly state when you're really out of your depth. It's important to note, though, that this isn't done adversarially; it's usually just a natural conversation that goes down a particular rabbit hole.