4 ms·

> The problem here is that Blizzard Entertainment and their performance management system mixes the two approaches,” Grote said by email. “They’re using absolut

by jonfw 4y ago

> The problem here is that Blizzard Entertainment and their performance management system mixes the two approaches,” Grote said by email. “They’re using absolute comparison terms to make a relative comparison judgement.” I think this is an interesting take. If you have to stack rank employees, then that's at least a valid piece of data, a relative comparison. We can argue about the value of that relative data and how it's appropriate to use it, but at least it's a fairly honest piece of data. If you attempt to portray this as an absolute piece of data, either by saying the bottom 5% don't meet expectations, or by applying quotas to absolute ratings (i.e. 5% must not meet expectations), then now you're producing something fundamentally dishonest.

gumby 4y ago
> If you have to stack rank employees, then that's at least a valid piece of data, a relative comparison. It's hard to project the multivariate impact of any employee onto an ordered line. I don't consider the result "a valid piece of data". Imagine stack ranking a soccer team and then removing one of the players. You need to score goals and the goalkeeper doesn't score any goals, so...
- jonfw 4y ago
  I think you do have to have an idea of which soccer player is more valuable than another though, because when salary negotiations come you may have to pick which one you can keep. And you certainly have to know which is your best goalkeeper out of your goalkeepers, for example, because that's the guy who you want starting Hopefully you have coaches who know a little more about the game then to negatively evaluate a goalkeeper for not scoring goals
  - trynewideas 4y ago
    This whole analogy is quite bad. Goalkeeper performance isn't linear across all traits. GK1 might start because they're the better overall shot stopper, but they're awful in PKs. You don't play them in a knockout match against a team with no offense that you know is parking the bus to force a penalty shootout because you'd be a shit manager to waste a sub on that. Or GK1A might be the better shot stopper, but GK1B might be the better distributor. GK1A might have better chemistry with the backline, but GK1B reads midfield runs better and has a bigger leg, so can get you an assist in transition against a slower, higher-playing opponent. GK1A might come up clutch with acrobatic saves that make the highlight reels, but GK1B reads the game better enough that the whole team becomes less prone to giving up high-quality chances with them in net. Maybe GK1A fits your team's style better, so you keep them. But that's not an objective measure. You can't get there with stack ranking without manipulating the stack, at which point it's a rigged scheme, not a method. And even if you say "well what about GK3, the one who clearly isn't as good at any of those things as GK1A or GK1B? Can't we cut them?" I mean, sure you can. And then when GK1A's hip gives out during pre-match warmups, and GK1B takes a boot to the face during a set piece, I hope you know which backup field player has the best hands. Good luck! Or when you let GK3 walk on their contract because even though as their manager you saw their potential as a junior player, and then they go to your crosstown rivals and realize their potential — against your team, in a knockout round of a cup, where your GKs still performed well — what good did the stack ranking do you? Sometimes the player who isn't the best today is going to be the best given a chance in a year, but not if you have to jettison them to meet an arbitrary quota before they develop. In reality, teams do have to pick a starter. And in reality, sometimes a player isn't a good fit and walks or is asked not to return. But not all or many managers pick starters purely on a linear, objective measure, because despite everything you can assign a statistical value to, the sum of a person's performance is measurable neither linearly nor objectively. Doubly so within a team context. And this was the entire point of the analogy criticizing stack ranking anyway: > Hopefully you have coaches who know a little more about the game then to negatively evaluate a goalkeeper for not scoring goals Using stack ranking is intentionally asking a manager of a team that's objectively all performing well on all measurable metrics, to then find another arbitrary metric and identify one who isn't performing well on it. Because one has to be. There's no option to say "GK3 is contributing well, developing quickly, and could start for us next season". No, they're GK3. You already ranked them, they're at the bottom of the rank, they already were and didn't move up in time, so they're fired. On to the forwards. And unlike football, you can't even get a transfer fee for landing the odd one out at a potential competitor.
- skibidibipiti 4y ago
  [dead]
- sokoloff 4y ago
  > You need to score goals and the goalkeeper doesn't score any goals, so... You need to score more goals than the opponents score, so...
- edmundsauto 4y ago
  Companies large enough to stack rank are more like leagues. It’s trivial to identify with reasonable confidence who the bottom players are in a league.
  - torstenvl 4y ago
    Yes. But the bottom 5% of players in a league is not equal to the combined set of the bottom 5% of players from each team.
    - edmundsauto 4y ago
      Right, so is the conclusion that effective stank ranking requires sufficiently large org sizes to be effective? I don’t think I would support stack ranking even then - org context matters too much. The question, I think, is if I replaced these bottom 5% with average performers that I could hire, what is the delta in outcome? Some positions are setup and only able to produce bottom end performances (think baseball pitchers in Colorado, where offense was originally inflated by 20+% due to elevation affecting how the ball flies) Having the stats of an average pitcher, when throwing half your games in COL, puts a guy in the top 10% of the league. Put a good pitcher in a bad spot and you’ll see someone perform “poorly”.

by jonfw 4y ago

gumby 4y ago

> If you have to stack rank employees, then that's at least a valid piece of data, a relative comparison. It's hard to project the multivariate impact of any employee onto an ordered line. I don't consider the result "a valid piece of data". Imagine stack ranking a soccer team and then removing one of the players. You need to score goals and the goalkeeper doesn't score any goals, so...

jonfw 4y ago
I think you do have to have an idea of which soccer player is more valuable than another though, because when salary negotiations come you may have to pick which one you can keep. And you certainly have to know which is your best goalkeeper out of your goalkeepers, for example, because that's the guy who you want starting Hopefully you have coaches who know a little more about the game then to negatively evaluate a goalkeeper for not scoring goals
- trynewideas 4y ago
  This whole analogy is quite bad. Goalkeeper performance isn't linear across all traits. GK1 might start because they're the better overall shot stopper, but they're awful in PKs. You don't play them in a knockout match against a team with no offense that you know is parking the bus to force a penalty shootout because you'd be a shit manager to waste a sub on that. Or GK1A might be the better shot stopper, but GK1B might be the better distributor. GK1A might have better chemistry with the backline, but GK1B reads midfield runs better and has a bigger leg, so can get you an assist in transition against a slower, higher-playing opponent. GK1A might come up clutch with acrobatic saves that make the highlight reels, but GK1B reads the game better enough that the whole team becomes less prone to giving up high-quality chances with them in net. Maybe GK1A fits your team's style better, so you keep them. But that's not an objective measure. You can't get there with stack ranking without manipulating the stack, at which point it's a rigged scheme, not a method. And even if you say "well what about GK3, the one who clearly isn't as good at any of those things as GK1A or GK1B? Can't we cut them?" I mean, sure you can. And then when GK1A's hip gives out during pre-match warmups, and GK1B takes a boot to the face during a set piece, I hope you know which backup field player has the best hands. Good luck! Or when you let GK3 walk on their contract because even though as their manager you saw their potential as a junior player, and then they go to your crosstown rivals and realize their potential — against your team, in a knockout round of a cup, where your GKs still performed well — what good did the stack ranking do you? Sometimes the player who isn't the best today is going to be the best given a chance in a year, but not if you have to jettison them to meet an arbitrary quota before they develop. In reality, teams do have to pick a starter. And in reality, sometimes a player isn't a good fit and walks or is asked not to return. But not all or many managers pick starters purely on a linear, objective measure, because despite everything you can assign a statistical value to, the sum of a person's performance is measurable neither linearly nor objectively. Doubly so within a team context. And this was the entire point of the analogy criticizing stack ranking anyway: > Hopefully you have coaches who know a little more about the game then to negatively evaluate a goalkeeper for not scoring goals Using stack ranking is intentionally asking a manager of a team that's objectively all performing well on all measurable metrics, to then find another arbitrary metric and identify one who isn't performing well on it. Because one has to be. There's no option to say "GK3 is contributing well, developing quickly, and could start for us next season". No, they're GK3. You already ranked them, they're at the bottom of the rank, they already were and didn't move up in time, so they're fired. On to the forwards. And unlike football, you can't even get a transfer fee for landing the odd one out at a potential competitor.
skibidibipiti 4y ago
[dead]
sokoloff 4y ago
> You need to score goals and the goalkeeper doesn't score any goals, so... You need to score more goals than the opponents score, so...
edmundsauto 4y ago
Companies large enough to stack rank are more like leagues. It’s trivial to identify with reasonable confidence who the bottom players are in a league.
- torstenvl 4y ago
  Yes. But the bottom 5% of players in a league is not equal to the combined set of the bottom 5% of players from each team.
  - edmundsauto 4y ago
    Right, so is the conclusion that effective stank ranking requires sufficiently large org sizes to be effective? I don’t think I would support stack ranking even then - org context matters too much. The question, I think, is if I replaced these bottom 5% with average performers that I could hire, what is the delta in outcome? Some positions are setup and only able to produce bottom end performances (think baseball pitchers in Colorado, where offense was originally inflated by 20+% due to elevation affecting how the ball flies) Having the stats of an average pitcher, when throwing half your games in COL, puts a guy in the top 10% of the league. Put a good pitcher in a bad spot and you’ll see someone perform “poorly”.