8 ms·

The evil, destructive, part of stack ranking is when you need to apply the 20%/70%/10% breakdowns to all teams equally, meaning that even a team of 5 people nee

The evil, destructive, part of stack ranking is when you need to apply the 20%/70%/10% breakdowns to all teams equally, meaning that even a team of 5 people needs to have one and only one "exceeds expectation" person and one person who "needs improvement", regardless of overall team level. It's not clear from this writeup that Amazon and Facebook are applying curves at such a micro level.

fizx 13y ago
I've wondered if recursively propagating down "bonus points" would be a fair way of solving this. For example, Acme, Inc ranks on a 1-5 scale, and all things being equal, a manager will get (direct reports)x3 points to allocate for his performance reviews. But Acme has a good quarter due to a strong engineering effort, so the CEO decides that Bob, VP of Eng, gets a 4, and he gets to allocate (direct reports)x4 points among his directors, who each get to allocate (their score)x(their direct reports), and so on. I think there needs to be some adjustment to the equations, because it doesn't make sense that everyone under a superstar also should be rated a 5. But I like the general idea of having groups be accountable as a group, and having that propagate downwards into the organization.
- kybernetikos 13y ago
  I always wanted to experiment with a system where all employees have a currency that they can pay other developers who help them or that they think are doing really well. As well as the credits that all employees have, there are also large numbers of credits distributed to the product owners of products in proportion to how much money they make. Come bonus time, the bonuses are paid in proportion to how many credits you have received from others.
  - fizx 13y ago
    c.f. http://blog.bonus.ly/post/33011276149/google-peer-bonus http://blog.bonus.ly/post/33011276149/google-peer-bonus
  - kansface 13y ago
    I could easily imagine gaming this system just like the others via kickbacks or other quid pro quos.
    - kybernetikos 13y ago
      There are things you could do to make it harder and potentially more dangerous to game, but to be honest, all systems can be gamed, but honest people are less likely to do so if they are in an environment where they are trusted and treated like adults.
CamperBob2 13y ago
I wonder if you could fix that by exempting employees from individual ranking if they work in groups smaller than n, and then rank those groups as if they were individual contributors. That way, a mid-level manager with five groups of five people would have to cut the least promising group loose. ... Nah, still sounds like a stupid idea. Any ranking system where an individual's outcome depends on their number of colleagues is bafflingly broken. This process can't build a cathedral or a bazaar, just a bureaucracy.
- duaneb 13y ago
  > Any ranking system where an individual's outcome depends on their number of colleagues is bafflingly broken. From the perspective of the company, it's not as clear how this approach is broken. Assumably promotions, hirings, and firings should be based on the current needs of the company, not what's fair to the employees or some ideological vision. I'm pretty sure attempting to apply any one method for evaluating so many different employees in so many different roles will fail in some manner.
  - tolmasky 13y ago
    The critique is not in relation to fairness to employees but rather the reality that there doesn't always have to be such a distribution of performance. Sometimes most people are excelling, similarly, sometimes you may need to fire FIFTY percent of the workforce. You should judge people on a measurement that's not purely comparative to other employees. If every single employee steals from the company, that doesn't mean that the one who stole the least amount of things is "exceptional".
  - cwp 13y ago
    The problem with stack ranking isn't that it's unfair to individual employees, it's that it inhibits cooperation. If "success" is defined as performing better than your peers, it creates an incentive to prevent your peers from performing well. If everybody is actively sabotaging each other's efforts, or even just not helping each other, that hurts the company's overall performance.
ethomson 13y ago
My understanding of my group (which may not apply to all parts of Microsoft, of course) is that our 6-person groups are too small to stack rank, so stack ranking rolls up to the next level. So Microsoft doesn't apply stack ranking at such a micro level either.
- DerpDerpDerp 13y ago
  My understanding is that they don't /officially/ apply them at such a micro level, but it requires an unusually persuasive manager for the "low ranks" not to be shared around evenly between the people having to rank their workers.
  - paulyg 13y ago
    My wife worked at a company that did stack ranking and that is exactly the way it worked out for her 5 person team. You also need an unusually persuasive manager to get more than one high rank.
kvb 13y ago
What leads you to believe that Microsoft applies the curve at that level?
- tedunangst 13y ago
  There's an evil way to do stack ranking and a not evil way to do stack ranking. Obviously, Microsoft does it the evil way because Microsoft.
varunsrin 13y ago
Stack ranking at Microsoft has a min populaton size (x) - you generally walk up the org chart until the number of employees >= x, and then apply stack ranking to that group.
- Metrop0218 13y ago
  I think opponents of stack ranking generally overlook this fact. Yes, it's terrible if you have a 5 person team, but the data becomes more normal as you add more people.
  - JoeAltmaier 13y ago
    You don't get anything like 'normal' without scores of samples. Any small (<50?) group is going to deviate significantly. Which means stacking is abusive to somebody.
  - crazcarl 13y ago
    How do you know the data becomes more normal as your add more people? Especially if you are performing the rankings independently on each individual. If you feel like your company is hiring against a certain set of high standards, are your employees at the low end of the 'normalized' curve holding you back as much as the people at the high end are pushing you forward?
    - berntb 13y ago
      If we put it this way, you answer your own question: :-) >> How do you know the data becomes more normal as your add [data points]? That said, yeah this is crazy. The worst problem with stack ratings is not what it do (firing people which don't work out will happen, it is often good even for those fired). The worst problem is that it destroys team culture as it is described at Microsoft: "if you're in a too good team, switch or you'll be fired." It gets hard for managers to build good teams when the individuals even have an incentive to sabotage each others. And so on. Edit: The first part had a ":-)" on it. I think that is not interesting, compared to the idea of being in a team with people that have motivation to screw their team mates over. Like being in prison or playing the Paranoia RPG (an orc in Sauron's army?)
deleted 13y ago
[deleted]
yajoe 13y ago
Completely agree, and I didn't know this practice had spread so widely. While doing the HR-sanctioned stack ranking requires X > MIN_POP_SIZE, in practice each line manager (i.e. lead) provides a pre-sorted list up the chain for merge-sorting. That way the line managers minimize the number of arguments during the official org-wide stack ranking. Many managers have gotten good at balancing their teams with high and low performers so they come "pre-sorted" as the lingo goes. If you don't believe me, ask about "pre-sorting." The fact your manager recognizes the term should be telling... Remember, managers get graded on how well they appear to their peers in stack ranking. It's ok as a manager to have low performers so long as you are "managing them." Managers have a disincentive to overrepresent their team to their peers, and managers are perfectly fine pre-sorting their lists. Again, it's easier and better for them to do so. The unofficial pre-sorting when X < MIN_POP_SIZE is what most individual workers notice (managers play favorites, careers are differential functions anyway), and HR has prepared the logical response (MIN_POP_SIZE is larger than your team) for managers to say when an employee asks. It's clear that the primary effect of stack ranking is to induce churn in the org at the expense of loyalty and morale. The people making the decisions are smart, so I presume the effect is part of the strategy. My theory? Software development isn't as skilled as we like to think it is, and it's often cheaper to hire a young kid and pay a "high" entry-level salary with vested bonuses than invest in long-term employees. Think about all the unvested money that the company saves by pushing employees out before 4 and 5 year maturity cycles. Plus, actual senior engineers (not the title!) tend to say "no" more often to clueless middle management and create unnecessary headaches.
- eitally 13y ago
  I think you're exactly right. The vast majority of enterprise software development is commodity work that can be done acceptably well by junior staff. The senior folks and leaders will bitch about it, but not to the point of getting themselves fired for insubordination.
jrochkind1 13y ago
In fact, with Amazon it seems they are not, that's probably what Molly Graham meant when she wrote: > Facebook has seven performance assessments as well as a guideline for what % of employees should be at each level, however it is explicitly not a forced curve, particularly for small teams. I think the author completely missed the point of the original criticism they were responding to -- the criticism was very specifically about the specifically required distributions in each category, within each team. The author of OP, on the other hand, seems to just be making the case that there are reasons to rate some people high and others low, and all the tech giants do it! That's the proverbial straw man.
deleted 13y ago
[deleted]
rdxm 13y ago
too true. i'm being forced to do this to my team at another very large tech company. the large data-set curve approach is completely toxic in small teams that are destroying themselves to meet crazy goals.
walshemj 13y ago
The even more evil part is where the stack ranking figures are adjusted to suit hr's desired outcome with no direct link between an individuals performance and your final score.
xtracto 13y ago
I think the following anecdote may illustrate the silliness of the ranking systems: My wife works in a big manufacturing company (which is just slightly related to IT). She started to work in a just-opened position for some department. As the position is new, they are not sure about her obligations. After one year, she had a end-of-year review, where her immediate boss rates her performance. Because she completed everything she was asked to (which was not a lot... given that they still are defining the role), her boss gave mainly 9s or 10s (out of 10) for her rating. Well... apparently, nobody in that company is supposed to get 9s or 10s, as that would mean that their work is perfect, and their motto is that everyone can improve. So my wife's boss was scolded by her boss, and had to decrease my wife's ratings. The worst thing is that my wife is the only person in her actual position, so she is not competing against anyone. ...
- cpeterso 13y ago
  When I bought a car recently, the sales person said the company would send me a survey about his service. On a scale of 1 to 10, he claimed (and I did believe him) that the company treats a 10 as a 10 and a 1-9 as a 0 when determining employee bonuses.
  - yesiamyourdad 13y ago
    I read the same assertion in a couple car buying guides. It was given as advice going into negotiation: let the salesman and his(her) manager know that you know how the rating system works and you're happy to give perfect scores for a good buying experience.

fizx 13y ago

I've wondered if recursively propagating down "bonus points" would be a fair way of solving this. For example, Acme, Inc ranks on a 1-5 scale, and all things being equal, a manager will get (direct reports)x3 points to allocate for his performance reviews. But Acme has a good quarter due to a strong engineering effort, so the CEO decides that Bob, VP of Eng, gets a 4, and he gets to allocate (direct reports)x4 points among his directors, who each get to allocate (their score)x(their direct reports), and so on. I think there needs to be some adjustment to the equations, because it doesn't make sense that everyone under a superstar also should be rated a 5. But I like the general idea of having groups be accountable as a group, and having that propagate downwards into the organization.

kybernetikos 13y ago
I always wanted to experiment with a system where all employees have a currency that they can pay other developers who help them or that they think are doing really well. As well as the credits that all employees have, there are also large numbers of credits distributed to the product owners of products in proportion to how much money they make. Come bonus time, the bonuses are paid in proportion to how many credits you have received from others.
- fizx 13y ago
  c.f. http://blog.bonus.ly/post/33011276149/google-peer-bonus http://blog.bonus.ly/post/33011276149/google-peer-bonus
- kansface 13y ago
  I could easily imagine gaming this system just like the others via kickbacks or other quid pro quos.
  - kybernetikos 13y ago
    There are things you could do to make it harder and potentially more dangerous to game, but to be honest, all systems can be gamed, but honest people are less likely to do so if they are in an environment where they are trusted and treated like adults.

CamperBob2 13y ago

I wonder if you could fix that by exempting employees from individual ranking if they work in groups smaller than n, and then rank those groups as if they were individual contributors. That way, a mid-level manager with five groups of five people would have to cut the least promising group loose. ... Nah, still sounds like a stupid idea. Any ranking system where an individual's outcome depends on their number of colleagues is bafflingly broken. This process can't build a cathedral or a bazaar, just a bureaucracy.

duaneb 13y ago
> Any ranking system where an individual's outcome depends on their number of colleagues is bafflingly broken. From the perspective of the company, it's not as clear how this approach is broken. Assumably promotions, hirings, and firings should be based on the current needs of the company, not what's fair to the employees or some ideological vision. I'm pretty sure attempting to apply any one method for evaluating so many different employees in so many different roles will fail in some manner.
- tolmasky 13y ago
  The critique is not in relation to fairness to employees but rather the reality that there doesn't always have to be such a distribution of performance. Sometimes most people are excelling, similarly, sometimes you may need to fire FIFTY percent of the workforce. You should judge people on a measurement that's not purely comparative to other employees. If every single employee steals from the company, that doesn't mean that the one who stole the least amount of things is "exceptional".
- cwp 13y ago
  The problem with stack ranking isn't that it's unfair to individual employees, it's that it inhibits cooperation. If "success" is defined as performing better than your peers, it creates an incentive to prevent your peers from performing well. If everybody is actively sabotaging each other's efforts, or even just not helping each other, that hurts the company's overall performance.

ethomson 13y ago

My understanding of my group (which may not apply to all parts of Microsoft, of course) is that our 6-person groups are too small to stack rank, so stack ranking rolls up to the next level. So Microsoft doesn't apply stack ranking at such a micro level either.

DerpDerpDerp 13y ago
My understanding is that they don't /officially/ apply them at such a micro level, but it requires an unusually persuasive manager for the "low ranks" not to be shared around evenly between the people having to rank their workers.
- paulyg 13y ago
  My wife worked at a company that did stack ranking and that is exactly the way it worked out for her 5 person team. You also need an unusually persuasive manager to get more than one high rank.

kvb 13y ago

What leads you to believe that Microsoft applies the curve at that level?

tedunangst 13y ago
There's an evil way to do stack ranking and a not evil way to do stack ranking. Obviously, Microsoft does it the evil way because Microsoft.

varunsrin 13y ago

Stack ranking at Microsoft has a min populaton size (x) - you generally walk up the org chart until the number of employees >= x, and then apply stack ranking to that group.

Metrop0218 13y ago
I think opponents of stack ranking generally overlook this fact. Yes, it's terrible if you have a 5 person team, but the data becomes more normal as you add more people.
- JoeAltmaier 13y ago
  You don't get anything like 'normal' without scores of samples. Any small (<50?) group is going to deviate significantly. Which means stacking is abusive to somebody.
- crazcarl 13y ago
  How do you know the data becomes more normal as your add more people? Especially if you are performing the rankings independently on each individual. If you feel like your company is hiring against a certain set of high standards, are your employees at the low end of the 'normalized' curve holding you back as much as the people at the high end are pushing you forward?
  - berntb 13y ago
    If we put it this way, you answer your own question: :-) >> How do you know the data becomes more normal as your add [data points]? That said, yeah this is crazy. The worst problem with stack ratings is not what it do (firing people which don't work out will happen, it is often good even for those fired). The worst problem is that it destroys team culture as it is described at Microsoft: "if you're in a too good team, switch or you'll be fired." It gets hard for managers to build good teams when the individuals even have an incentive to sabotage each others. And so on. Edit: The first part had a ":-)" on it. I think that is not interesting, compared to the idea of being in a team with people that have motivation to screw their team mates over. Like being in prison or playing the Paranoia RPG (an orc in Sauron's army?)

deleted 13y ago

[deleted]

yajoe 13y ago

Completely agree, and I didn't know this practice had spread so widely. While doing the HR-sanctioned stack ranking requires X > MIN_POP_SIZE, in practice each line manager (i.e. lead) provides a pre-sorted list up the chain for merge-sorting. That way the line managers minimize the number of arguments during the official org-wide stack ranking. Many managers have gotten good at balancing their teams with high and low performers so they come "pre-sorted" as the lingo goes. If you don't believe me, ask about "pre-sorting." The fact your manager recognizes the term should be telling... Remember, managers get graded on how well they appear to their peers in stack ranking. It's ok as a manager to have low performers so long as you are "managing them." Managers have a disincentive to overrepresent their team to their peers, and managers are perfectly fine pre-sorting their lists. Again, it's easier and better for them to do so. The unofficial pre-sorting when X < MIN_POP_SIZE is what most individual workers notice (managers play favorites, careers are differential functions anyway), and HR has prepared the logical response (MIN_POP_SIZE is larger than your team) for managers to say when an employee asks. It's clear that the primary effect of stack ranking is to induce churn in the org at the expense of loyalty and morale. The people making the decisions are smart, so I presume the effect is part of the strategy. My theory? Software development isn't as skilled as we like to think it is, and it's often cheaper to hire a young kid and pay a "high" entry-level salary with vested bonuses than invest in long-term employees. Think about all the unvested money that the company saves by pushing employees out before 4 and 5 year maturity cycles. Plus, actual senior engineers (not the title!) tend to say "no" more often to clueless middle management and create unnecessary headaches.

eitally 13y ago
I think you're exactly right. The vast majority of enterprise software development is commodity work that can be done acceptably well by junior staff. The senior folks and leaders will bitch about it, but not to the point of getting themselves fired for insubordination.

jrochkind1 13y ago

In fact, with Amazon it seems they are not, that's probably what Molly Graham meant when she wrote: > Facebook has seven performance assessments as well as a guideline for what % of employees should be at each level, however it is explicitly not a forced curve, particularly for small teams. I think the author completely missed the point of the original criticism they were responding to -- the criticism was very specifically about the specifically required distributions in each category, within each team. The author of OP, on the other hand, seems to just be making the case that there are reasons to rate some people high and others low, and all the tech giants do it! That's the proverbial straw man.

deleted 13y ago

[deleted]

rdxm 13y ago

too true. i'm being forced to do this to my team at another very large tech company. the large data-set curve approach is completely toxic in small teams that are destroying themselves to meet crazy goals.

walshemj 13y ago

The even more evil part is where the stack ranking figures are adjusted to suit hr's desired outcome with no direct link between an individuals performance and your final score.

xtracto 13y ago

I think the following anecdote may illustrate the silliness of the ranking systems: My wife works in a big manufacturing company (which is just slightly related to IT). She started to work in a just-opened position for some department. As the position is new, they are not sure about her obligations. After one year, she had a end-of-year review, where her immediate boss rates her performance. Because she completed everything she was asked to (which was not a lot... given that they still are defining the role), her boss gave mainly 9s or 10s (out of 10) for her rating. Well... apparently, nobody in that company is supposed to get 9s or 10s, as that would mean that their work is perfect, and their motto is that everyone can improve. So my wife's boss was scolded by her boss, and had to decrease my wife's ratings. The worst thing is that my wife is the only person in her actual position, so she is not competing against anyone. ...

cpeterso 13y ago
When I bought a car recently, the sales person said the company would send me a survey about his service. On a scale of 1 to 10, he claimed (and I did believe him) that the company treats a 10 as a 10 and a 1-9 as a 0 when determining employee bonuses.
- yesiamyourdad 13y ago
  I read the same assertion in a couple car buying guides. It was given as advice going into negotiation: let the salesman and his(her) manager know that you know how the rating system works and you're happy to give perfect scores for a good buying experience.