4 ms·

I'm curious if the people giving the interviews have an incentive to be conservative with scoring. Maybe it looks bad when your score is the highest? As if you

I'm curious if the people giving the interviews have an incentive to be conservative with scoring. Maybe it looks bad when your score is the highest? As if you're lowering the bar? If true, I could see that contributing to how these interview stories play out (candidates thinking they did better than their no-hire outcome).

gil_vegliach 9y ago
Scores are normalized so it doesn't matter. If you always give a low score, that score will count more after normalization. It would be interesting to know how they treat outliers: for example, if you give always scores 1-2.5 and then suddenly a 4.
EmployedRussian 9y ago
There are no such incentives. When writing feedback, you simply look back at 20 other interviews where you asked the same question, and write "candidate X's performance on this question was was in {top,bottom} {5,25,50}% of ... at this level". You also have no idea how other interviewers will score, and if your score is 1.5 when everyone else's is a 3.9, it's likely that the HC will ignore you. The HCs do see multiple feedback from you and your peers, so they build a model of your feedback for themselves: P is a soft interviewer, so we'll adjust his score down but Q is a very hard interviewer so we'll adjust her score up ;-)
- steelframe 9y ago
  My hires are uniformly distributed across the score spectrum. Not sure how the HCs "adjust" me. I have gotten props for the quality of my written feedback though.
  - EmployedRussian 9y ago
    It doesn't matter how your candidates are distributed across the spectrum (and I do hope that your hires are not uniformly distributed ;-) What matters is how your scores compare to other interviewers scores on the given slate. > Not sure how the HCs "adjust" me. Maybe they don't. Maybe you are the most precise and best-calibrated interviewer on every slate, and never make a bad call. Good for you ;-)

gil_vegliach 9y ago

Scores are normalized so it doesn't matter. If you always give a low score, that score will count more after normalization. It would be interesting to know how they treat outliers: for example, if you give always scores 1-2.5 and then suddenly a 4.

EmployedRussian 9y ago

There are no such incentives. When writing feedback, you simply look back at 20 other interviews where you asked the same question, and write "candidate X's performance on this question was was in {top,bottom} {5,25,50}% of ... at this level". You also have no idea how other interviewers will score, and if your score is 1.5 when everyone else's is a 3.9, it's likely that the HC will ignore you. The HCs do see multiple feedback from you and your peers, so they build a model of your feedback for themselves: P is a soft interviewer, so we'll adjust his score down but Q is a very hard interviewer so we'll adjust her score up ;-)

steelframe 9y ago
My hires are uniformly distributed across the score spectrum. Not sure how the HCs "adjust" me. I have gotten props for the quality of my written feedback though.
- EmployedRussian 9y ago
  It doesn't matter how your candidates are distributed across the spectrum (and I do hope that your hires are not uniformly distributed ;-) What matters is how your scores compare to other interviewers scores on the given slate. > Not sure how the HCs "adjust" me. Maybe they don't. Maybe you are the most precise and best-calibrated interviewer on every slate, and never make a bad call. Good for you ;-)