4 ms·

It seems to me that it should be straightforward to objectively measure hiring practices by correlating different interview questions and techniques with subseq

It seems to me that it should be straightforward to objectively measure hiring practices by correlating different interview questions and techniques with subsequent job performance (compared to a control.) Has this been done? If so, I'd love to see the studies, if not, then it would be hard to argue that current techniques aren't voodoo.

username90 7y ago
It has been done, I've seen the numbers and they work. Other companies have probably done the same. Of course this isn't popular, instead people up-vote anecdotes from people who say that they once had an algorithm wiz who performed badly, or how some stellar performers were rejected by this kind of interview.
- tastroder 7y ago
  > Of course this isn't popular, instead people up-vote anecdotes from people who say that they once had an algorithm wiz who performed badly, or how some stellar performers were rejected by this kind of interview. That's kind of reductionist. I upvote answers that at least attempt to propose alternatives to the current status quo. The employer side isn't the only relevant one, which is likely a stronger reason why a lot of current hiring practices aren't popular. Some only work at FAANG scale with a large enough pool of applicants, some seem optimized to hire people that waste time on code puzzles (that doesn't make them an "algorithm wiz") and others generally put applicants in a position resembling begging for a job, especially for junior roles. Just because something works well on some quantitative metric does neither imply that it is the only, nor the optimal solution, especially in two player games like hiring.
  - username90 7y ago
    Right, if you pay significantly less than top companies you probably don't want to filter using their metrics since then you will just get their leftovers. Instead you should try to find the good candidates they miss. However there is really no reason for top companies to change their interviews.
    - tastroder 7y ago
      > Right, if you pay significantly less than top companies you probably don't want to filter using their metrics since then you will just get their leftovers. Not sure where you read "less pay" into my comment but fine, let's go with that and just ignore that quite a few companies immitate FAANG style hiring these days. > Instead you should try to find the good candidates they miss. However there is really no reason for top companies to change their interviews. Yeah, I feel like we're arguing two completely different things here, there's plenty of reasons for companies to change processes even if they work on some metric. When a large number of experienced participants in the fields you are hiring for are offering critique on your process it should be at least reason enough to talk about it, which is the whole point of discussions like these.
      - username90 7y ago
        > quite a few companies immitate FAANG style hiring these days. My point was that they shouldn't unless their offer can compete with FAANG offers. > When a large number of experienced participants in the fields you are hiring for are offering critique on your process it should be at least reason enough to talk about it FAANG are spending billions a year to keep this process running, there are huge amounts of work put into trying to change or improve it. And some things do change, but the signals from doing algorithms is just too salient to throw away.
deanCommie 7y ago
The short answer is yes, but those studies are kept internal to the companies. The longer answer is we can only study the people that joined the company and failed to perform rather than those that COULD have been outstanding but our interview process selected them out. So there are flaws, of course. I'm not sure how you could do the latter unless the biggest companies shared their data and compared scenarios of "Candidate X failed an Company Y interview, but then later got a job at Company Z, and thrived". Also you'd be working against the bias of "Well, that just means that the Company Z is easier to work at". There are just too many variables, team to team. Part of the problem is even within these massive companies there is a huge diversity of types of jobs, and an engineer who thrives on working on Google AdWords might fail on the SpannerDB team (to pick two random examples from not my company)

username90 7y ago

It has been done, I've seen the numbers and they work. Other companies have probably done the same. Of course this isn't popular, instead people up-vote anecdotes from people who say that they once had an algorithm wiz who performed badly, or how some stellar performers were rejected by this kind of interview.

tastroder 7y ago
> Of course this isn't popular, instead people up-vote anecdotes from people who say that they once had an algorithm wiz who performed badly, or how some stellar performers were rejected by this kind of interview. That's kind of reductionist. I upvote answers that at least attempt to propose alternatives to the current status quo. The employer side isn't the only relevant one, which is likely a stronger reason why a lot of current hiring practices aren't popular. Some only work at FAANG scale with a large enough pool of applicants, some seem optimized to hire people that waste time on code puzzles (that doesn't make them an "algorithm wiz") and others generally put applicants in a position resembling begging for a job, especially for junior roles. Just because something works well on some quantitative metric does neither imply that it is the only, nor the optimal solution, especially in two player games like hiring.
- username90 7y ago
  Right, if you pay significantly less than top companies you probably don't want to filter using their metrics since then you will just get their leftovers. Instead you should try to find the good candidates they miss. However there is really no reason for top companies to change their interviews.
  - tastroder 7y ago
    > Right, if you pay significantly less than top companies you probably don't want to filter using their metrics since then you will just get their leftovers. Not sure where you read "less pay" into my comment but fine, let's go with that and just ignore that quite a few companies immitate FAANG style hiring these days. > Instead you should try to find the good candidates they miss. However there is really no reason for top companies to change their interviews. Yeah, I feel like we're arguing two completely different things here, there's plenty of reasons for companies to change processes even if they work on some metric. When a large number of experienced participants in the fields you are hiring for are offering critique on your process it should be at least reason enough to talk about it, which is the whole point of discussions like these.
    - username90 7y ago
      > quite a few companies immitate FAANG style hiring these days. My point was that they shouldn't unless their offer can compete with FAANG offers. > When a large number of experienced participants in the fields you are hiring for are offering critique on your process it should be at least reason enough to talk about it FAANG are spending billions a year to keep this process running, there are huge amounts of work put into trying to change or improve it. And some things do change, but the signals from doing algorithms is just too salient to throw away.

deanCommie 7y ago

The short answer is yes, but those studies are kept internal to the companies. The longer answer is we can only study the people that joined the company and failed to perform rather than those that COULD have been outstanding but our interview process selected them out. So there are flaws, of course. I'm not sure how you could do the latter unless the biggest companies shared their data and compared scenarios of "Candidate X failed an Company Y interview, but then later got a job at Company Z, and thrived". Also you'd be working against the bias of "Well, that just means that the Company Z is easier to work at". There are just too many variables, team to team. Part of the problem is even within these massive companies there is a huge diversity of types of jobs, and an engineer who thrives on working on Google AdWords might fail on the SpannerDB team (to pick two random examples from not my company)