7 ms·
All stories about how bad stack ranking goes like this: "I was a manager at X and all my subordinates were above average/great and the system forced me to give
by bjourne 13y ago
All stories about how bad stack ranking goes like this: "I was a manager at X and all my subordinates were above average/great and the system forced me to give them lower grades."
But it stands to reason that since the system measures performance relative to the average exactly half of the managers should have had the opposite problem: "all my subordinates were worse than average but I had to give them higher grades than they deserved." I think the former problem is much more common because people tend to overvalue the people they are directly in contact with.
I suspect that if the managers could set the grade freely from 1-5, the an unreasonable amount of 4's and 5's would be given. In a large organisation like Microsoft's the skill of the engineers and the ratings should follow a normal distribution.
- vwinsyee 13y ago> In a large organisation like Microsoft's the skill of the engineers and the ratings should follow a normal distribution. My understanding is that stack ranking is effective when first implemented. Subsequent implementations are progressively less effective and eventually become toxic if done too frequently. This is because at the beginning, the distribution of skill in the company's population of engineers does mirror the distribution in the overall, global population of engineers. So the initial stack rank eliminates the bottom, say 10%, performing engineers. The problem is when this happens again the following year, which seems to be what Microsoft has been doing. Barring a horrible incoming recruiting class, the company's distribution has already been shifting to the right. So each yearly cut is cutting out increasingly better engineers.
- marcosdumay 13y agoWell, by that rationale, the worst your hiring process, the more you gain by stack ranking once. But, even if you hiring people by tossing coins, if you don't have huge teams the results won't be fair. Yet, just defining a minimum performance and firing the people that don't reach it outperforms that first application of stack ranking. And you can apply that filter as many times you want, with no loss of quality.
- craigyk 13y agoThat's ignoring the actual "accuracy" of most employee "ranking" algorithms. And continual, though diminishing improvements is only true if your method of ranking employees is significantly more accurate on existing employees than for job candidates.
- RougeFemme 13y agoAnd if they were all ranked within one company-wide group, that might not be so bad. But in the organizations I've worked in that used stack ranking, that wasn't the case. You were ranked within much smaller groups. So group A, which, company-wide, would have had all average to above employees now has to drop some person(s) down to below average, within the group. That "below average" person - who would have been average company-wide - now gets a "performance improvement plan" or worse - shown the door. And group B, which, company-wide, would have had all average and below-average employees, will now rank some person(s) above average. That above-average person - who would've been only average company-wide - now gets a bonus. Should it work like that? No. Does it work like that? More often that it should; I've seen it - in mid-size and large companies.
- hkmurakami 13y agoNow that I think about it, Princeton University now has this sort of problem (since 2004) with its Grade Inflation policy, where there are quotas for the number of A's and A-'s that can be given out.
- seanmcdirmid 13y agoAcademia is one of the few places where the bell curve should be applied diligently...because you want the students to see some sort of top to aspire to.
- hga 13y agoNot at top schools where most of a class may well deserve an A or B. Outside of a period in the '60s (the draft) MIT never suffered from serious grade inflation (I've looked at the numbers), and the general rule is that if you've picked a suitable major you'll make As and Bs unless you're having personal problems. It would be stark raving mad to grade its students on the curve ... and I've heard of serious pathology in schools that mechanically do, e.g. premeds sabotaging their peers taking organic chemistry (one tale had one deactivating many of his peers' alarm clocks). You just don't want to set up the wrong incentives, and I can attest that at least in the '80s MIT students were generally happy to help each other. On the other hand there are state schools that by law have to accept every applicant who meets a random external threshold ... they don't have much choice but to flunk out large fractions of their freshman. But after the courses that do that, and often do that for a major, they shouldn't obsess on the curve.
- Florin_Andrei 13y agoThe article states it plainly: the problem with the stack ranking system is that it's a zero-sum game. (BTW, I tend to read it as "stank raking" for some weird reason.)
- dragonwriter 13y ago> The article states it plainly: the problem with the stack ranking system is that it's a zero-sum game. Done properly, and used for the right purposes, that actually makes sense. Particularly, in the case of compensation, if overall raises for a group were set (possibly by stack ranking of groups with a supergroup where the overall level was set), and stack ranking was used to determine the distribution within the group.
- Florin_Andrei 13y agoThe thing is, you don't zero-sum it if your goal is to become or remain a leader of innovation and a paragon of creativity.
- dragonwriter 13y agoThat's an interesting conclusion, but I don't see the argument supporting it. Zero-sum rankings within each group, in nested groups, seem to make perfect sense for most purposes -- what's wrong to me would seem to be the assumptions like, e.g., that punitive measures need to be assigned to the bottom rank within groups every time, regardless of the overall performance of the group.
- jleader 13y agoIf you choose your employees at random from the pool of applicants, and the applicants' abilities are normally distributed, then you're right that the employees' performance will be normally distributed. But I would expect any tech firm's hiring criteria to have at least some correlation with employees' performance, which means their performance will be biased towards the high end of the curve. So no, the skill of the engineers probably shouldn't follow a normal distribution.
- tiziano88 13y agoBy definition, average is in the middle of whatever distribution they follow. Maybe median would be a more appropriate point to measure against though.
- lutusp 13y ago> By definition, average is in the middle of whatever distribution they follow. Not necessarily the middle. In a skewed distribution, the average is still the sum of the samples divided by the number of samples, but it's not located at the middle (centerpoint) of the sample set. > Maybe median would be a more appropriate point to measure against though. You managed to get that exactly backwards. It's the median that's located at the middle of the distribution, not the mean. For a balanced distribution, they're the same, but for a skewed distribution, it's the mean that moves away from the "middle", not the median.
- craigyk 13y agoI think you are wrong. I think randomly drawing any significant number of people from a pool that isn't normally distributed is still likely to result in a normal distribution. http://en.wikipedia.org/wiki/Central_limit_theorem http://en.wikipedia.org/wiki/Central_limit_theorem
- zosima 13y agoNo, that's not quite right. Randomly picking people from a distribution should eventually end up giving the original distribution. Taking n people from a pool m times, then the means of those m samplings will be normally distributed however (given the constraints of the CLT of course).