4 ms·

In my experience, Google didn't have a forced curve ranking then, either. I've been a manager at Google since 2007 or so (with a 3-year gap when I went walkabou

In my experience, Google didn't have a forced curve ranking then, either. I've been a manager at Google since 2007 or so (with a 3-year gap when I went walkabout 2011-2014), I've probably participated in calibration of many thousands of engineers (and other ladders), and I've never once heard of a calibration score being adjusted to fit some expectation that a certain number of employees needed to be ranked as underperformers.

evgen 11y ago
While scores are almost never directly manipulated "as we are not hitting the curve" my experience is that you know what the curve is supposed to look like and people on the bubble one way or another are pushed in a particular direction during calibration to make the curve line up. Then at the end you all look at the curve and pat yourselves on the back for another round of matching what you expected to see rather than asking if expecting to see a particular curve influenced the results.
- timdierks 11y ago
  I've never seen people pushed like that to make the curve line up, that I can recall. There's a belief that if you have a large enough sample, that the distribution will have a certain shape (that is, it's unlikely that if you have 500 people in a calibration that there won't be a few underperformers). I've never seen that used to change a score, either in an retrospective or anticipatory fashion, but only to assess whether the process is being internally honest, and only at scales that are large enough that the concern isn't particularly affected by sampling error. (That is, if past experience is that a certain outlying part of the curve has roughly 2% of employees in it, I've never seen anyone object because that bucket is empty with a N of 150 or less.)
  - evgen 11y ago
    What I am saying is that you probably do not realize you are doing it, but if you have an expectation of hitting a particular curve (and please don't try to tell me that there is no page somewhere outlining what are the expected percentages at various performance levels...) then you will bias yourself towards hitting that curve. You will not realize it, but you will do it anyway. This usually happens at calibration and not initial assessment, but I am quite certain that it _does_ happen because the numbers do not lie.
    - timdierks 11y ago
      I guess at some point, I agree, but this is so far from the problems of forced ranking and application of curves to small numbers of staff that I think it's incommensurate.

evgen 11y ago

While scores are almost never directly manipulated "as we are not hitting the curve" my experience is that you know what the curve is supposed to look like and people on the bubble one way or another are pushed in a particular direction during calibration to make the curve line up. Then at the end you all look at the curve and pat yourselves on the back for another round of matching what you expected to see rather than asking if expecting to see a particular curve influenced the results.

timdierks 11y ago
I've never seen people pushed like that to make the curve line up, that I can recall. There's a belief that if you have a large enough sample, that the distribution will have a certain shape (that is, it's unlikely that if you have 500 people in a calibration that there won't be a few underperformers). I've never seen that used to change a score, either in an retrospective or anticipatory fashion, but only to assess whether the process is being internally honest, and only at scales that are large enough that the concern isn't particularly affected by sampling error. (That is, if past experience is that a certain outlying part of the curve has roughly 2% of employees in it, I've never seen anyone object because that bucket is empty with a N of 150 or less.)
- evgen 11y ago
  What I am saying is that you probably do not realize you are doing it, but if you have an expectation of hitting a particular curve (and please don't try to tell me that there is no page somewhere outlining what are the expected percentages at various performance levels...) then you will bias yourself towards hitting that curve. You will not realize it, but you will do it anyway. This usually happens at calibration and not initial assessment, but I am quite certain that it _does_ happen because the numbers do not lie.
  - timdierks 11y ago
    I guess at some point, I agree, but this is so far from the problems of forced ranking and application of curves to small numbers of staff that I think it's incommensurate.