8 ms·

If your company does 360 reviews or has a "rating" system (smart companies refuse to call it a ranking), it's the same thing. It's in the corporate america HR h

If your company does 360 reviews or has a "rating" system (smart companies refuse to call it a ranking), it's the same thing. It's in the corporate america HR handbook. If you are ranked highly, someone had to be ranked lowly.

aetherson 10y ago
I'm a manager at my company. We have a rating system, and there is no need for someone else to be ranked lowly for you to be ranked highly. What that DOES mean, though, is that someone has to keep pressure on people not to rank everyone as awesome. Because, look: a. Managers are, contrary to rumor, people too. We don't enjoy the awkward, painful meetings in which we deliver negative feedback. We don't enjoy telling people that they aren't getting a raise, or that they need to improve or be terminated. That sucks. Personally, it keeps me up nights, and I've heard the same from many other managers. b. Managers are also graded themselves on how effective their employees are. Like, if your employees are persistently rated below average, well, what's the common factor across your employees? Vice versa if they're all high performers. So if you just tell managers "rate your employees, follow your conscience," then there is a grade inflation incentive. Even if you're super-disciplined and make painfully realistic assessments of your employees' abilities, Bob one team over may give everyone fives, and then you end up looking like a bad manager and also maybe you get to go and put someone on a PIP. The company needs some way to stop this. Stack ranking is one such way. I don't like it, I think it's overall a bad way, but it's a mechanistic way, which is more valuable the larger your company. Small companies can have a powerful manager or small group of managers who "keep people honest." That doesn't scale. You can use another mechanic to keep people honest, but all mechanics are exploitable.
- dv_dt 10y ago
  So one manager is below average at hiring and randomly hires people, ending up with a nicely fit bell curve range of high and low performers. Another manager is great at hiring highly performing people, but now if they rank correctly, you look at their assessments with your cheat metric, the good manager is 'inflating' their ratings. In other words, looking at the rankings alone doesn't do anything for actually enforcing quality.
  - aetherson 10y ago
    Sure. There is no single silver bullet that actually enforces quality everywhere. But look, inflation is bad. If grades aren't inflated, and I rate someone a 3, then we say, "This person is a solid employee. We're happy they're here, we're giving them an average raise, everything's cool. Do we have a little constructive feedback that says, 'Hey, here's how you could take it to the next level?' Sure. But we're happy with them." If grades are inflated and I give someone a 3, then oh shit, maybe they're in trouble! Because some asshole had to give everyone in his team a 4 or 5, and now my solid employee looks like he's underperforming the company average. The fact is, a solid 4+ rating means you're a really great employee. Teams that are all high performing are super rare, and tend to be really, really, really obvious within a company. Now, is it a valid criticism of stack ranking that it makes it not just difficult but impossible to describe a team that's solidly above average? Yes, absolutely. Stack ranking is kind of dumb. But understand that despite interesting and compelling stories about people being unfairly rated lower than they deserve, that's not the big problem that companies face. In general, the incentives for everyone is to rate people higher than they deserve, not lower (with toxic exceptions like what Fowler faced). And if you set up your whole system to make really sure that it embraces the idea of everyone being super awesome, you probably don't get a highly functional rating system out the other side. It's probably better to set one up that embraces the idea of average teams (average for your company) and let people override the system when you say, "Oh my god, this entire team is just killing it this review cycle."
    - sheepmullet 10y ago
      > Teams that are all high performing are super rare, and tend to be really, really, really obvious within a company. Teams that don't fit a normal distribution are incredibly common. And I think the bigger problem is money allocated for pay increases. If the average 3 rating employee in your team can get a 10% annualised return each time they change jobs then the company should be budgeting 10% for a grade of 3. In practice this doesn't happen.
      - sheepmullet 10y ago
- mahyarm 10y ago
  So what other mechanics would you use?
  - a3n 10y ago
    At review time, get all of the group in a room, including the manager, and vote on who should leave. Because in a toxic environment, people are discussing that privately anyway, so let's be open and transparent. The person voted isn't necessarily made to leave. It's an opportunity for them to focus on improving themselves personally, possibly in a program.
    - padraic7a 10y ago
      Why would anyone have to leave? This isn't the worst idea ever, for that you'd have to include knives or acid.
    - mratzloff 10y ago
      That would encourages a lot of unworkable incentives and behaviors. It doesn't forward the goal of promoting team cohesion at all.
      - a3n 10y ago
        No, I suppose it doesn't. But neither does secret stack ranking. I once asked exactly where I was on the stack, and was told that's proprietary information.
      - prodigal_erik 10y ago
        The only way to get team cohesion is positive-sum priorities. If I spend more time helping you improve long-term and less time delivering on my own responsibilities, that needs to improve my next evaluation, otherwise I'm pretty much compelled to stop doing it.
    - koolba 10y ago
      > At review time, get all of the group in a room, including the manager, and vote on who should leave. Because in a toxic environment, people are discussing that privately anyway, so let's be open and transparent. This a genius idea. That way you push the liability of discrimination from the company to the employees at an individual level. It's like mob rule but for companies. > The person voted isn't necessarily made to leave. It's an opportunity for them to focus on improving themselves personally, possibly in a program. I can't begin to imagine how awkward it would be to be voted off the island and stick around anyway.
- sbov 10y ago
  Why would a manager be graded on how they grade their own people? Also, stack ranking "solves" the problem only because it makes the idea obviously ridiculous, because all managers would be graded equally.
  - Broken_Hippo 10y ago
    If I read correctly, they rank yet do not have quotas. Someone performing well doesn't mean someone got scored low. In this system, a low performing person can become average by improving skills. With this in mind, it makes sense to grade the manager. If a manager has a lot of poor performers, they might not be giving staff the training, support, and feedback that is necessary to improve. If they all get high scores, they might be grading too easily. I'm guessing they have controls to help lessen this.
- sebastos 10y ago
  I don't really like "systems" for ranking and scoring people in general, but just as an exercise in systems design here, what about this for a strategy? 1. Each manager gives their people an absolute score 2. A meta-manager checks in with all managers over the course of the year and keeps tabs on performance of a randomly selected group of people (some scalable fraction of the total workforce. As this number goes up you could hire a few more people to do this job). 3. Each meta-manager looks for obvious goofy scoring discrepancies (how can person A have a lower rating than person B?). 4. Each manager is on the hook to defend those ratings in the face of scrutiny to justify why they rated Tony Terrible better than Andy Awesome. This expectation of accountability prevents them from inflating their scores too much. Sort of like QA for management assessments.
koolba 10y ago
> If your company does 360 reviews or has a "rating" system (smart companies refuse to call it a ranking), it's the same thing. It's in the corporate america HR handbook. If you are ranked highly, someone had to be ranked lowly. Damn straight and that's a natural side effect of having a pool of limited resources (i.e. bonus money and promotions) to offer.
danudey 10y ago
Stack ranking is not a rating system. In a normal rating system, fi you have a team of all-star engineers you can give everyone 4/5 or 5/5. In a stack ranking system, you have (for example) two slots for 5/5, two slots for 1/5, etc. This means that even if you have the ten best engineers in the company on your team, two of them are going to get the lowest possible rating (and thus be unable to leave the team). They won't get promotions, they won't get bonuses, and they won't be appreciated. The problem with stack ranking isn't that employees can be ranked higher or lower than one another (as in any normal rating system), it's that you absolutely have to give someone a failing mark, even if no one actually failed. This means that if you get transferred to a team where everyone is highly skilled, then you run a real chance of being the person who gets ranked terrible and can never get a raise or promotion unless your other teammates leave – or unless you sabotage their progress or refuse to help them in an effort to try to lower their performance.
sjg007 10y ago
360 reviews underpin the hit CBS show Survivor. I'm sure some eager type can take that premise and run with it to form a critique of corporate America. Also note that MBAs and other business types have learnt to game the system by providing 5 star all around. Why? Your average tenure at a company is low (2-3 years). You need your peers for your next job. Why denigrate each other.
- maxxxxx 10y ago
  In my company moving up the ranks is pretty difficult in tech whereas other departments are very generous with titles (and the salaries that come with them). It seems engineers are not smart enough to game the system and actually rank each other honestly. Whereas the business and law guys know better.
yourapostasy 10y ago
The rational response from employees to stack ranking is immediately (or ASAP) quit upon receiving a low-ranking slot. Company is (ostensibly) happy to get rid of a "low performer", employee finds their market value, everyone is theoretically happy. Ironically, if your hiring process (arguably a difficult aspect of management and leadership) is firing on all cylinders, stack ranking is blind to that, and thus strongly disincentivizes improving your hiring practices in the first place. It's a perverse form of letting bugs slip through careful coding and design practices when they are far cheaper to fix, and letting them loose into production facing customers when they are way more expensive (monetarily and soft factors) to resolve, but in the hiring and retention realm. I learned about stack ranking the hard way. When I quit, it caused all sorts of commotion, culminating in the CEO personally calling me to nicely and politely ask me to stay, far contrary to my worst-slot ranking would suggest. That opened my eyes to its drawbacks. I can see a use case for stack ranking as a leadership tool in situations with clear featherbedding, but when used as a permanent policy it's management cargo-culting at best, Jack Welch fetishization at worst.
- Mithaldu 10y ago
  What did you end up telling the CEO?
  - yourapostasy 10y ago
    Politely declined; burned no bridges. I already accepted an offer by the time I quit that added 50% to my base pay, with bonuses on top and lots of career development benefits (pay to send me to speaking engagements at conferences, for example), and with a direct manager who was eager for my contributions. In that particular instance, stack ranking's practical effect served to severely depress my skill base's value and my compensation plan, as there were several fast similar offers, and the frantic response from the stack rank-using company's management layers highlighted they had a good idea of the value I was delivering. The CEO understood both the surface and subtext, after some very good salesmanship got nowhere. If any readers ever find themselves in a similar situation, do not explain the details of why you refuse to stay on, as that only cracks opens a door to high-pressure negotiation. Easiest way out is simply state you already accepted a very attractive offer, if pressed for details respond the other company swore you to not disclose the terms to anyone, and stay polite. Like a proper diet and working out where there is no substitute for doing the work, stack ranking rarely if ever substitutes for accomplished, attentive leadership (as contrasted with tactical aspects of management, a core prerequisite to said leadership). It is a useful scalpel if/when wielded properly, and is best left in the toolbox otherwise.
fencepost 10y ago
One thing I don't recall seeing mentioned in these discussions is validity checking based on the different sources of ratings. If you're seeing widely varying ratings of a single person that seems like it'd be a big red flag - if the boss says 1 or 2 but coworkers or reports average 3.5+ that's a situation that seems to cry out for further investigation. Similarly, if the boss says "4" but coworkers and reports average 1.5, there's probably also a serious problem. I'm sure investigating involves costs that companies don't want to incur, but if you have that kind of disconnect going on it may be cheaper to investigate and resolve it than to deal with the eventual fallout. I've seen the personal aftermath of bad bosses and work environments on people in the past, and it's really not pretty. If you're talking with someone about work and say "Mean Girls" and get a response of "Oh my god yes!" that's a pretty good indicator of a toxic work environment.
lijason 10y ago
Unless I'm misunderstanding something, a 360 review doesn't require this kind of ranking. A 360 review just indicates that you're getting feedback/reviews from all directions relative to the org chart (manager, reports, peers, etc). What you do with that feedback and if you stack rank or something similar is what creates the zero-sum game.

aetherson 10y ago

I'm a manager at my company. We have a rating system, and there is no need for someone else to be ranked lowly for you to be ranked highly. What that DOES mean, though, is that someone has to keep pressure on people not to rank everyone as awesome. Because, look: a. Managers are, contrary to rumor, people too. We don't enjoy the awkward, painful meetings in which we deliver negative feedback. We don't enjoy telling people that they aren't getting a raise, or that they need to improve or be terminated. That sucks. Personally, it keeps me up nights, and I've heard the same from many other managers. b. Managers are also graded themselves on how effective their employees are. Like, if your employees are persistently rated below average, well, what's the common factor across your employees? Vice versa if they're all high performers. So if you just tell managers "rate your employees, follow your conscience," then there is a grade inflation incentive. Even if you're super-disciplined and make painfully realistic assessments of your employees' abilities, Bob one team over may give everyone fives, and then you end up looking like a bad manager and also maybe you get to go and put someone on a PIP. The company needs some way to stop this. Stack ranking is one such way. I don't like it, I think it's overall a bad way, but it's a mechanistic way, which is more valuable the larger your company. Small companies can have a powerful manager or small group of managers who "keep people honest." That doesn't scale. You can use another mechanic to keep people honest, but all mechanics are exploitable.

dv_dt 10y ago
So one manager is below average at hiring and randomly hires people, ending up with a nicely fit bell curve range of high and low performers. Another manager is great at hiring highly performing people, but now if they rank correctly, you look at their assessments with your cheat metric, the good manager is 'inflating' their ratings. In other words, looking at the rankings alone doesn't do anything for actually enforcing quality.
- aetherson 10y ago
  Sure. There is no single silver bullet that actually enforces quality everywhere. But look, inflation is bad. If grades aren't inflated, and I rate someone a 3, then we say, "This person is a solid employee. We're happy they're here, we're giving them an average raise, everything's cool. Do we have a little constructive feedback that says, 'Hey, here's how you could take it to the next level?' Sure. But we're happy with them." If grades are inflated and I give someone a 3, then oh shit, maybe they're in trouble! Because some asshole had to give everyone in his team a 4 or 5, and now my solid employee looks like he's underperforming the company average. The fact is, a solid 4+ rating means you're a really great employee. Teams that are all high performing are super rare, and tend to be really, really, really obvious within a company. Now, is it a valid criticism of stack ranking that it makes it not just difficult but impossible to describe a team that's solidly above average? Yes, absolutely. Stack ranking is kind of dumb. But understand that despite interesting and compelling stories about people being unfairly rated lower than they deserve, that's not the big problem that companies face. In general, the incentives for everyone is to rate people higher than they deserve, not lower (with toxic exceptions like what Fowler faced). And if you set up your whole system to make really sure that it embraces the idea of everyone being super awesome, you probably don't get a highly functional rating system out the other side. It's probably better to set one up that embraces the idea of average teams (average for your company) and let people override the system when you say, "Oh my god, this entire team is just killing it this review cycle."
  - sheepmullet 10y ago
    > Teams that are all high performing are super rare, and tend to be really, really, really obvious within a company. Teams that don't fit a normal distribution are incredibly common. And I think the bigger problem is money allocated for pay increases. If the average 3 rating employee in your team can get a 10% annualised return each time they change jobs then the company should be budgeting 10% for a grade of 3. In practice this doesn't happen.
    - sheepmullet 10y ago
mahyarm 10y ago
So what other mechanics would you use?
- a3n 10y ago
  At review time, get all of the group in a room, including the manager, and vote on who should leave. Because in a toxic environment, people are discussing that privately anyway, so let's be open and transparent. The person voted isn't necessarily made to leave. It's an opportunity for them to focus on improving themselves personally, possibly in a program.
  - padraic7a 10y ago
    Why would anyone have to leave? This isn't the worst idea ever, for that you'd have to include knives or acid.
  - mratzloff 10y ago
    That would encourages a lot of unworkable incentives and behaviors. It doesn't forward the goal of promoting team cohesion at all.
    - a3n 10y ago
      No, I suppose it doesn't. But neither does secret stack ranking. I once asked exactly where I was on the stack, and was told that's proprietary information.
    - prodigal_erik 10y ago
      The only way to get team cohesion is positive-sum priorities. If I spend more time helping you improve long-term and less time delivering on my own responsibilities, that needs to improve my next evaluation, otherwise I'm pretty much compelled to stop doing it.
  - koolba 10y ago
    > At review time, get all of the group in a room, including the manager, and vote on who should leave. Because in a toxic environment, people are discussing that privately anyway, so let's be open and transparent. This a genius idea. That way you push the liability of discrimination from the company to the employees at an individual level. It's like mob rule but for companies. > The person voted isn't necessarily made to leave. It's an opportunity for them to focus on improving themselves personally, possibly in a program. I can't begin to imagine how awkward it would be to be voted off the island and stick around anyway.
sbov 10y ago
Why would a manager be graded on how they grade their own people? Also, stack ranking "solves" the problem only because it makes the idea obviously ridiculous, because all managers would be graded equally.
- Broken_Hippo 10y ago
  If I read correctly, they rank yet do not have quotas. Someone performing well doesn't mean someone got scored low. In this system, a low performing person can become average by improving skills. With this in mind, it makes sense to grade the manager. If a manager has a lot of poor performers, they might not be giving staff the training, support, and feedback that is necessary to improve. If they all get high scores, they might be grading too easily. I'm guessing they have controls to help lessen this.
sebastos 10y ago
I don't really like "systems" for ranking and scoring people in general, but just as an exercise in systems design here, what about this for a strategy? 1. Each manager gives their people an absolute score 2. A meta-manager checks in with all managers over the course of the year and keeps tabs on performance of a randomly selected group of people (some scalable fraction of the total workforce. As this number goes up you could hire a few more people to do this job). 3. Each meta-manager looks for obvious goofy scoring discrepancies (how can person A have a lower rating than person B?). 4. Each manager is on the hook to defend those ratings in the face of scrutiny to justify why they rated Tony Terrible better than Andy Awesome. This expectation of accountability prevents them from inflating their scores too much. Sort of like QA for management assessments.

koolba 10y ago

> If your company does 360 reviews or has a "rating" system (smart companies refuse to call it a ranking), it's the same thing. It's in the corporate america HR handbook. If you are ranked highly, someone had to be ranked lowly. Damn straight and that's a natural side effect of having a pool of limited resources (i.e. bonus money and promotions) to offer.

danudey 10y ago

Stack ranking is not a rating system. In a normal rating system, fi you have a team of all-star engineers you can give everyone 4/5 or 5/5. In a stack ranking system, you have (for example) two slots for 5/5, two slots for 1/5, etc. This means that even if you have the ten best engineers in the company on your team, two of them are going to get the lowest possible rating (and thus be unable to leave the team). They won't get promotions, they won't get bonuses, and they won't be appreciated. The problem with stack ranking isn't that employees can be ranked higher or lower than one another (as in any normal rating system), it's that you absolutely have to give someone a failing mark, even if no one actually failed. This means that if you get transferred to a team where everyone is highly skilled, then you run a real chance of being the person who gets ranked terrible and can never get a raise or promotion unless your other teammates leave – or unless you sabotage their progress or refuse to help them in an effort to try to lower their performance.

sjg007 10y ago

360 reviews underpin the hit CBS show Survivor. I'm sure some eager type can take that premise and run with it to form a critique of corporate America. Also note that MBAs and other business types have learnt to game the system by providing 5 star all around. Why? Your average tenure at a company is low (2-3 years). You need your peers for your next job. Why denigrate each other.

maxxxxx 10y ago
In my company moving up the ranks is pretty difficult in tech whereas other departments are very generous with titles (and the salaries that come with them). It seems engineers are not smart enough to game the system and actually rank each other honestly. Whereas the business and law guys know better.

yourapostasy 10y ago

The rational response from employees to stack ranking is immediately (or ASAP) quit upon receiving a low-ranking slot. Company is (ostensibly) happy to get rid of a "low performer", employee finds their market value, everyone is theoretically happy. Ironically, if your hiring process (arguably a difficult aspect of management and leadership) is firing on all cylinders, stack ranking is blind to that, and thus strongly disincentivizes improving your hiring practices in the first place. It's a perverse form of letting bugs slip through careful coding and design practices when they are far cheaper to fix, and letting them loose into production facing customers when they are way more expensive (monetarily and soft factors) to resolve, but in the hiring and retention realm. I learned about stack ranking the hard way. When I quit, it caused all sorts of commotion, culminating in the CEO personally calling me to nicely and politely ask me to stay, far contrary to my worst-slot ranking would suggest. That opened my eyes to its drawbacks. I can see a use case for stack ranking as a leadership tool in situations with clear featherbedding, but when used as a permanent policy it's management cargo-culting at best, Jack Welch fetishization at worst.

Mithaldu 10y ago
What did you end up telling the CEO?
- yourapostasy 10y ago
  Politely declined; burned no bridges. I already accepted an offer by the time I quit that added 50% to my base pay, with bonuses on top and lots of career development benefits (pay to send me to speaking engagements at conferences, for example), and with a direct manager who was eager for my contributions. In that particular instance, stack ranking's practical effect served to severely depress my skill base's value and my compensation plan, as there were several fast similar offers, and the frantic response from the stack rank-using company's management layers highlighted they had a good idea of the value I was delivering. The CEO understood both the surface and subtext, after some very good salesmanship got nowhere. If any readers ever find themselves in a similar situation, do not explain the details of why you refuse to stay on, as that only cracks opens a door to high-pressure negotiation. Easiest way out is simply state you already accepted a very attractive offer, if pressed for details respond the other company swore you to not disclose the terms to anyone, and stay polite. Like a proper diet and working out where there is no substitute for doing the work, stack ranking rarely if ever substitutes for accomplished, attentive leadership (as contrasted with tactical aspects of management, a core prerequisite to said leadership). It is a useful scalpel if/when wielded properly, and is best left in the toolbox otherwise.

fencepost 10y ago

One thing I don't recall seeing mentioned in these discussions is validity checking based on the different sources of ratings. If you're seeing widely varying ratings of a single person that seems like it'd be a big red flag - if the boss says 1 or 2 but coworkers or reports average 3.5+ that's a situation that seems to cry out for further investigation. Similarly, if the boss says "4" but coworkers and reports average 1.5, there's probably also a serious problem. I'm sure investigating involves costs that companies don't want to incur, but if you have that kind of disconnect going on it may be cheaper to investigate and resolve it than to deal with the eventual fallout. I've seen the personal aftermath of bad bosses and work environments on people in the past, and it's really not pretty. If you're talking with someone about work and say "Mean Girls" and get a response of "Oh my god yes!" that's a pretty good indicator of a toxic work environment.

lijason 10y ago

Unless I'm misunderstanding something, a 360 review doesn't require this kind of ranking. A 360 review just indicates that you're getting feedback/reviews from all directions relative to the org chart (manager, reports, peers, etc). What you do with that feedback and if you stack rank or something similar is what creates the zero-sum game.