3 ms·

Why 5 years? Why not 1? Why not 10? 2? 4? How might those numbers change the answers you receive? Are you even testing to see how people's responses change

Why 5 years? Why not 1? Why not 10? 2? 4? How might those numbers change the answers you receive? Are you even testing to see how people's responses change as a function of that number? Are you even coding the responses in a way that would be able to detect how their responses change in response to that number? Is the variance in the responses small enough that you can even come up with a way to code them? For example, if the interviewee is startup savvy and you ask the 5-year question then they understand the vesting cliffs are at play. Suddenly you're taking into account how savvy the interviewee is about the way vesting works. Was that the intention? If not then you're measuring something without even realizing it Either way: more variance! Is it clear what the question is trying to get at? Do all candidates understand what it's trying to get at? Could candidate misinterpret the intent of the question given the fact that they're in an interview? Could different interviewers interpret the same response differently? It's a "bad" question because it controls for very little and therefore inherently produces non-diagnostic information (such is the nature of humans). The level of variance in all the components of this question is exactly the surface area on which cognitive biases become the determining factor. And the nature of cognitive biases is that you become more confident when they're operational, not less. You can ask questions about career goals without going at it sideways. Here's some off the top of my head. I haven't thought them through from the above perspective, but as a first pass they control for more, are likely to get higher-signal responses that are easier to evaluate in a consistent, interpretable way. - At your last job, what career goals did you have when you joined? Did you achieve them? How? - Imagine you've been working here for 18 months. What would need to happen between now and then for you to not feel disappointed you joined? - Think back to a manager who helped you define and achieve a professional goal (big or small). What was that goal? How did they help you achieve it? Why did that work?

kenhwang 8y ago
I usually don't say '5 years' specifically. "In a couple of years" is closer to the phrasing I use. The candidate can pick whatever time period they're comfortable with answering. Most answer with equal fuzziness in their timeline. Every now and then we'll get a candidate who'll have a super rigid timeline, "raise in 6 months, promotion to in 12, second promotion at the 2 year mark, able to buy at house after 3 years, student loans paid off in 47months, after 401k/stocks vest I'll do something else". That's more information than I need and not the answer I'm looking for. Most candidates get what we're trying to ask right away. Sure, sometimes we get a response like "well my current apartment doesn't have enough power outlets and smells like bread, so I would like to move and get a cat"; then we'll follow up with a surgically specific question. It's a conversation, not an interrogation. The vagueness usually helps candidates open up and talk. Interviewing is stressful enough without the multi-part hyper specific word problem about something they probably haven't thought about. We're interviewing people, and people are different with high variance. If we wanted consistency with bias removed, we can just craft a long multiple choice exam and have HR proctor it. Interviewing will always have noise, variance, and bias.
- jfarmer 8y ago
  Haha, well, hopefully nobody's opening with "Where do you see yourself in 5 years?" or any of the questions I listed! Doing either of those things is a great way to signal to the candidate that INTERVIEW MODE has been engaged. The goal isn't to remove noise, variance, and bias, but to control it in a way that produces a better assessment and a better experience for the candidate. This also allows the interview question + overall process to improve over time. Variance is present in every scientific experiment, after all, but the whole process of designing an experiment + evaluating the results is about controlling for known sources of variance and minimizing the ways unknown sources can get in. For example, one might frame the structured process thus: "We ask an identical series of questions to every candidate so we know everyone's getting a fair shake. It's important to us that we're evaluating candidates based on their actual experience and potential, not on their ability to do well in interviews. We hate tricky or ambiguous questions because they make the people we interview anxious and produce answers that are hard to evaluate fairly." You would want to do that before they came into the room, too, to control for the possibility that receiving that information in the interview room itself produced surprising interactions (more variance again). In general, the candidate should have a clear picture of what will be happening in an interview regardless, to control for the above as best as possible. (I can say from experience that if the above is done with a friendly demeanor and good bedside manner, the sense of relief in the candidate is palpable.) Imagine asking a candidate a question about a positive interaction with a previous manager that gets at the thing you're trying to evaluate. The candidate is naturally going to be happy talking about a positive experience! The framing of the question is intended to get them thinking about the moments you want them to be thinking about. "A couple of years" is not great because now it's a function of how the candidate perceives the phrase "a couple of years." More variance!What if one candidate specifies their impression of "a couple of years" and another doesn't? Asking the candidate to pick a time period explicitly also not great because now your impression is a function of the (time period, response) they chose and not just the response. More variance, again! It's better to ask questions structured in a way that gets at what you're really looking for. I don't know exactly what you're looking for, specifically — ambition, reasonable expectations around opportunities for and speed of promotion, alignment with company priorities, something else? — but there will almost certainly be questions that produce better diagnostic information and feel more natural to the candidate. Inherent in the interview process is the feeling that you, the candidate, want to give the answer that the person is looking for. "Where do you see yourself in (5/a couple/choose-your-time-period) years?" hits right at that anxiety (which is why a blog post like "what are they really asking?" is even necessary). Are you evaluating their ability to understand the question in the way you intended? If not, structure it in a way that reduces that possibility.

kenhwang 8y ago

I usually don't say '5 years' specifically. "In a couple of years" is closer to the phrasing I use. The candidate can pick whatever time period they're comfortable with answering. Most answer with equal fuzziness in their timeline. Every now and then we'll get a candidate who'll have a super rigid timeline, "raise in 6 months, promotion to in 12, second promotion at the 2 year mark, able to buy at house after 3 years, student loans paid off in 47months, after 401k/stocks vest I'll do something else". That's more information than I need and not the answer I'm looking for. Most candidates get what we're trying to ask right away. Sure, sometimes we get a response like "well my current apartment doesn't have enough power outlets and smells like bread, so I would like to move and get a cat"; then we'll follow up with a surgically specific question. It's a conversation, not an interrogation. The vagueness usually helps candidates open up and talk. Interviewing is stressful enough without the multi-part hyper specific word problem about something they probably haven't thought about. We're interviewing people, and people are different with high variance. If we wanted consistency with bias removed, we can just craft a long multiple choice exam and have HR proctor it. Interviewing will always have noise, variance, and bias.

jfarmer 8y ago
Haha, well, hopefully nobody's opening with "Where do you see yourself in 5 years?" or any of the questions I listed! Doing either of those things is a great way to signal to the candidate that INTERVIEW MODE has been engaged. The goal isn't to remove noise, variance, and bias, but to control it in a way that produces a better assessment and a better experience for the candidate. This also allows the interview question + overall process to improve over time. Variance is present in every scientific experiment, after all, but the whole process of designing an experiment + evaluating the results is about controlling for known sources of variance and minimizing the ways unknown sources can get in. For example, one might frame the structured process thus: "We ask an identical series of questions to every candidate so we know everyone's getting a fair shake. It's important to us that we're evaluating candidates based on their actual experience and potential, not on their ability to do well in interviews. We hate tricky or ambiguous questions because they make the people we interview anxious and produce answers that are hard to evaluate fairly." You would want to do that before they came into the room, too, to control for the possibility that receiving that information in the interview room itself produced surprising interactions (more variance again). In general, the candidate should have a clear picture of what will be happening in an interview regardless, to control for the above as best as possible. (I can say from experience that if the above is done with a friendly demeanor and good bedside manner, the sense of relief in the candidate is palpable.) Imagine asking a candidate a question about a positive interaction with a previous manager that gets at the thing you're trying to evaluate. The candidate is naturally going to be happy talking about a positive experience! The framing of the question is intended to get them thinking about the moments you want them to be thinking about. "A couple of years" is not great because now it's a function of how the candidate perceives the phrase "a couple of years." More variance!What if one candidate specifies their impression of "a couple of years" and another doesn't? Asking the candidate to pick a time period explicitly also not great because now your impression is a function of the (time period, response) they chose and not just the response. More variance, again! It's better to ask questions structured in a way that gets at what you're really looking for. I don't know exactly what you're looking for, specifically — ambition, reasonable expectations around opportunities for and speed of promotion, alignment with company priorities, something else? — but there will almost certainly be questions that produce better diagnostic information and feel more natural to the candidate. Inherent in the interview process is the feeling that you, the candidate, want to give the answer that the person is looking for. "Where do you see yourself in (5/a couple/choose-your-time-period) years?" hits right at that anxiety (which is why a blog post like "what are they really asking?" is even necessary). Are you evaluating their ability to understand the question in the way you intended? If not, structure it in a way that reduces that possibility.