5 ms·

I am perplexed why anyone would think that interview performances has any interesting statistical relevance. Much more interesting would be how successful the c

I am perplexed why anyone would think that interview performances has any interesting statistical relevance. Much more interesting would be how successful the candidate was after receiving a job at the company.

munchbunny 10y ago
For two reasons: 1. From the perspective of a job seeker, the interview is what gets you the job, so it's in your interests as a job seeker to learn how to look better as a candidate based on data about how interviewers judge candidates. 2. The purpose of an interview is to be a predictor for job performance, so while this article doesn't address this part of the question, you want interviews to predict job performance, and how to make interviews do that is interesting.
throw_away_777 10y ago
To get a job at a company you have to be given the opportunity to interview, and to succeed at the interview. If you only evaluate performance at the job level, you don't evaluate performance of anybody who failed the interview. This is one of the fundamental challenges in hiring - you have a biased sample when trying to determine how effective the hiring process is at screening candidates.
crispyambulance 10y ago
You're right. It is weird that so much effort is spent on interviewing and so much effort is spent on performance reviews. But AFAIK, hardly any organization tries to use performance evaluation of current employees to inform their hiring decisions.
inopinatus 10y ago
Most hiring managers would hope that interview performance is at least partially correlated to subsequent job performance. And once experience is gained as an interviewer and team manager, it is not an unreasonable hope.
- JamesBarney 10y ago
  The problem is that experience isn't very useful without feedback. And managers and interviewers get so little feedback about how well they do as interviewers I would imagine they don't improve much.
stale2002 10y ago
Because interviewing skill is how you get a job. If I am looking for a job, I care a whole lot about what would make me better at getting jobs.
- AndyNemmity 10y ago
  Interviewing skill is one way you get a job. After you've been doing this for long enough, and have a reputation, you tend to just get jobs by referrals. I decided in my last change of positions to try interviewing for jobs like most people without referrals, since I had several other referral positions open. And I was horrible. I was ridiculously bad, and couldn't get a job at jobs that paid less, and were for less skill. It was quite hilarious. Thankfully, all of the referral jobs were begging for me, and that was fine. There was no real loss, but it was a bit concerning that if I didn't have an extensive connection network that I would be in a significantly worse role.
  - alex_c 10y ago
    Which parts of the interviews were you really bad at? How would you change the interviews you had so they don't reject qualified candidates? I've been interviewing a lot of developers lately, and while I'm fairly happy with my process (I've managed to avoid bad hires so far), I always have to wonder if I'm turning down people I shouldn't.
    - chrshawkes 10y ago
      "I hate anything that asks me to design on the spot. That's asking to demonstrate a skill rarely required on the job in a high-stress environment, where it is difficult for a candidate to accurately prove their abilities. I think it's fundamentally an unfair thing to request of a candidate." Scott Meyers
    - AndyNemmity 10y ago
      The technical question part. Because I work on so many different things at the edge of innovation, I have a wide array of experience for a short period of time, and then I move on. When I interview for jobs, I open up to any of the possible things in my background, of which it will take me perhaps 2 weeks to get back up to speed. But I am not going to take that 2 weeks for each job I interview for. I have trained my brain to learn things, and discard them keeping only the current tasks in memory. The rest I know how to obtain again. This means that I get asked questions I know that I've done repeatedly in the past, but can only vaguely explain without a refresher. To the interviewer, this means I am underqualified at best, and at worst, lying about my past skills. A lovely example of this during the process was that I was asked a fairly basic SQL question that I failed to explain when I had just landed from my second flight of the day. I didn't get the presales role for working with the database. (non referral job.) However, I did get offered a job working on development for that database, as well as product management for the database. So I couldn't get a presales role, that I've done in my past, because I couldn't answer a fairly basic sql question. I knew exactly how to answer it, the venn diagram of sql joins, a half second google, but in an interview, I blanked on the answer. That is indicative of my experience with non-referral positions. I've given training on sql joins a good 30 times in my life, and couldn't explain the join in an interview. Just the way it goes for me, thankfully it's not actually impactful in my life.
joshvm 10y ago
The conclusion is a bit obvious: if you ask people algorithms questions, the people who do the best will be people who spent a lot of time rote learning algorithms material. This means schools which place a lot of emphasis on the theoretical core or people who've taken online algorithms courses. I'd be much more interested to see performance on other questions e.g. Google's typical curveball for new grads is an architecture question like "How would you implement YouTube?".
- drd93 10y ago
  These are not curveballs though. Nowadays it's just another category of interview questions - System Design. There are tons of resources to prepare for this type of questions just like with the Algorithms & Data Structures problems.
eanzenberg 10y ago
That, I'm afraid, would be too rigorous for this blog.
edblarney 10y ago
" interview performances has any interesting statistical relevance" I'm perplexed as to how people don't think it has any relevance at all. The entirety of every industry is based on 'interview performance' as being one fairly important factor of future success. Interviews are obviously not perfect predictors, but they are pretty strongly correlated with outcomes. Someone who does very well in a technical interview is more likely to be a 'good engineer' than someone who fails miserably - though I totally accept that is not always the case. There's no doubt we could use a little more science to our approach, however.
kristopolous 10y ago
I've honestly stopped working for places that interview based on whiteboards and brain teasers. I think it leads to a rather arbitrary selection of candidates and pretty terrible teams.
- chrshawkes 10y ago
  That is great to hear. I have personally turned down opportunities that ask for these high stress ridiculous tests in front of a bunch of strangers.

munchbunny 10y ago

For two reasons: 1. From the perspective of a job seeker, the interview is what gets you the job, so it's in your interests as a job seeker to learn how to look better as a candidate based on data about how interviewers judge candidates. 2. The purpose of an interview is to be a predictor for job performance, so while this article doesn't address this part of the question, you want interviews to predict job performance, and how to make interviews do that is interesting.

throw_away_777 10y ago

To get a job at a company you have to be given the opportunity to interview, and to succeed at the interview. If you only evaluate performance at the job level, you don't evaluate performance of anybody who failed the interview. This is one of the fundamental challenges in hiring - you have a biased sample when trying to determine how effective the hiring process is at screening candidates.

crispyambulance 10y ago

You're right. It is weird that so much effort is spent on interviewing and so much effort is spent on performance reviews. But AFAIK, hardly any organization tries to use performance evaluation of current employees to inform their hiring decisions.

inopinatus 10y ago

Most hiring managers would hope that interview performance is at least partially correlated to subsequent job performance. And once experience is gained as an interviewer and team manager, it is not an unreasonable hope.

JamesBarney 10y ago
The problem is that experience isn't very useful without feedback. And managers and interviewers get so little feedback about how well they do as interviewers I would imagine they don't improve much.

stale2002 10y ago

Because interviewing skill is how you get a job. If I am looking for a job, I care a whole lot about what would make me better at getting jobs.

AndyNemmity 10y ago
Interviewing skill is one way you get a job. After you've been doing this for long enough, and have a reputation, you tend to just get jobs by referrals. I decided in my last change of positions to try interviewing for jobs like most people without referrals, since I had several other referral positions open. And I was horrible. I was ridiculously bad, and couldn't get a job at jobs that paid less, and were for less skill. It was quite hilarious. Thankfully, all of the referral jobs were begging for me, and that was fine. There was no real loss, but it was a bit concerning that if I didn't have an extensive connection network that I would be in a significantly worse role.
- alex_c 10y ago
  Which parts of the interviews were you really bad at? How would you change the interviews you had so they don't reject qualified candidates? I've been interviewing a lot of developers lately, and while I'm fairly happy with my process (I've managed to avoid bad hires so far), I always have to wonder if I'm turning down people I shouldn't.
  - chrshawkes 10y ago
    "I hate anything that asks me to design on the spot. That's asking to demonstrate a skill rarely required on the job in a high-stress environment, where it is difficult for a candidate to accurately prove their abilities. I think it's fundamentally an unfair thing to request of a candidate." Scott Meyers
  - AndyNemmity 10y ago
    The technical question part. Because I work on so many different things at the edge of innovation, I have a wide array of experience for a short period of time, and then I move on. When I interview for jobs, I open up to any of the possible things in my background, of which it will take me perhaps 2 weeks to get back up to speed. But I am not going to take that 2 weeks for each job I interview for. I have trained my brain to learn things, and discard them keeping only the current tasks in memory. The rest I know how to obtain again. This means that I get asked questions I know that I've done repeatedly in the past, but can only vaguely explain without a refresher. To the interviewer, this means I am underqualified at best, and at worst, lying about my past skills. A lovely example of this during the process was that I was asked a fairly basic SQL question that I failed to explain when I had just landed from my second flight of the day. I didn't get the presales role for working with the database. (non referral job.) However, I did get offered a job working on development for that database, as well as product management for the database. So I couldn't get a presales role, that I've done in my past, because I couldn't answer a fairly basic sql question. I knew exactly how to answer it, the venn diagram of sql joins, a half second google, but in an interview, I blanked on the answer. That is indicative of my experience with non-referral positions. I've given training on sql joins a good 30 times in my life, and couldn't explain the join in an interview. Just the way it goes for me, thankfully it's not actually impactful in my life.

joshvm 10y ago

The conclusion is a bit obvious: if you ask people algorithms questions, the people who do the best will be people who spent a lot of time rote learning algorithms material. This means schools which place a lot of emphasis on the theoretical core or people who've taken online algorithms courses. I'd be much more interested to see performance on other questions e.g. Google's typical curveball for new grads is an architecture question like "How would you implement YouTube?".

drd93 10y ago
These are not curveballs though. Nowadays it's just another category of interview questions - System Design. There are tons of resources to prepare for this type of questions just like with the Algorithms & Data Structures problems.

eanzenberg 10y ago

That, I'm afraid, would be too rigorous for this blog.

edblarney 10y ago

" interview performances has any interesting statistical relevance" I'm perplexed as to how people don't think it has any relevance at all. The entirety of every industry is based on 'interview performance' as being one fairly important factor of future success. Interviews are obviously not perfect predictors, but they are pretty strongly correlated with outcomes. Someone who does very well in a technical interview is more likely to be a 'good engineer' than someone who fails miserably - though I totally accept that is not always the case. There's no doubt we could use a little more science to our approach, however.

kristopolous 10y ago

I've honestly stopped working for places that interview based on whiteboards and brain teasers. I think it leads to a rather arbitrary selection of candidates and pretty terrible teams.

chrshawkes 10y ago
That is great to hear. I have personally turned down opportunities that ask for these high stress ridiculous tests in front of a bunch of strangers.