6 ms·

Asking a question well takes skill. It is possible that an interviewer screwed up asking the question rather than that the candidate screwed up. Most interview

by babesh 8y ago

Asking a question well takes skill. It is possible that an interviewer screwed up asking the question rather than that the candidate screwed up. Most interview systems don't have a systematic, ongoing way for accounting for this especially given that interviewers are self reporting. In my opinion, most interviewers are undertrained/underskilled at asking specific questions and at interviewing in general. They don't invest the resources to systematically improve but rather treat interviewing as a burden to be minimized. There is still a lot play in systems that allow for borderline smart ass behavior. Do you think an interviewer will write up that they sneered at the candidate? The interview process is a factory. The company is mostly concerned with reaching outputs. The candidate is just a happenstance casualty.

seanmcdirmid 8y ago
It’s almost as if interviewers need to be tested as well. An interviewing contest where a faux candidate rates interviewers and suggests area for improvement. Of course, that would be rediculously expensive, so at best some canned training is used instead. Still, it might make sense for higher value teams.
- UncleMeat 8y ago
  At Google, responses are calibrated against other interviews that interviewer has done. If they have some consistent skew then this can be adjusted for. If the interviewer sucks and the committee can tell then they can give that score less weight.
- tytso 8y ago
  After interviewers are trained, they will perform at least two "shadow" interviews where they tag along an experienced interviewer and they want the interviewer conduct the interview and then are asked to write up an interview report. After they finish writing the report, they can see what the experienced interviewer wrote up, so they can understand how the writeup should be done. Afterwards, the new interviewer has to do a non-trivial number of interviews before they are considered "calibrated". When an interviewer is uncalibrated, their score won't be given much weight, and the hiring committee can see how their interview reports and scores compare against more experienced interviewers. This also gives an opportunity for the hiring committee to send interview feedback (e.g., you're asking a banned question; the coding question is too simple, so it's not providing enough of a useful signal; don't ask "trick questions" which again don't provide much useful signal whether or not the candidate can answer it correctly, etc.) So there is certainly ways in which interviewers do get suggestions for improvement. And it doesn't have to be _that_ expensive. It's just a matter of making sure you don't have more than one uncalibrated interviewer per panel.
  - zzzcpan 8y ago
    Sounds very ritualistic and cult-ish. It won't be possible to extract any useful information from such interviews. But I guess it's possible to convince employees to haze candidates this way. You should feel bad for doing this to people though.
    - joshuamorton 8y ago
      >It won't be possible to extract any useful information from such interviews. But I guess it's possible to convince employees to haze candidates this way. You should feel bad for doing this to people though. Empirically this isn't the case. It would be much more interesting if you took the time to elaborate on what about this process is cultish, why it won't provide useful signal. Without that, it just comes across as a mean-spirited complaint.

by babesh 8y ago

seanmcdirmid 8y ago

It’s almost as if interviewers need to be tested as well. An interviewing contest where a faux candidate rates interviewers and suggests area for improvement. Of course, that would be rediculously expensive, so at best some canned training is used instead. Still, it might make sense for higher value teams.

UncleMeat 8y ago
At Google, responses are calibrated against other interviews that interviewer has done. If they have some consistent skew then this can be adjusted for. If the interviewer sucks and the committee can tell then they can give that score less weight.
tytso 8y ago
After interviewers are trained, they will perform at least two "shadow" interviews where they tag along an experienced interviewer and they want the interviewer conduct the interview and then are asked to write up an interview report. After they finish writing the report, they can see what the experienced interviewer wrote up, so they can understand how the writeup should be done. Afterwards, the new interviewer has to do a non-trivial number of interviews before they are considered "calibrated". When an interviewer is uncalibrated, their score won't be given much weight, and the hiring committee can see how their interview reports and scores compare against more experienced interviewers. This also gives an opportunity for the hiring committee to send interview feedback (e.g., you're asking a banned question; the coding question is too simple, so it's not providing enough of a useful signal; don't ask "trick questions" which again don't provide much useful signal whether or not the candidate can answer it correctly, etc.) So there is certainly ways in which interviewers do get suggestions for improvement. And it doesn't have to be _that_ expensive. It's just a matter of making sure you don't have more than one uncalibrated interviewer per panel.
- zzzcpan 8y ago
  Sounds very ritualistic and cult-ish. It won't be possible to extract any useful information from such interviews. But I guess it's possible to convince employees to haze candidates this way. You should feel bad for doing this to people though.
  - joshuamorton 8y ago
    >It won't be possible to extract any useful information from such interviews. But I guess it's possible to convince employees to haze candidates this way. You should feel bad for doing this to people though. Empirically this isn't the case. It would be much more interesting if you took the time to elaborate on what about this process is cultish, why it won't provide useful signal. Without that, it just comes across as a mean-spirited complaint.