6 ms·

One major thing I learned from 360 reviews is how bad developers are at judging each other or their manager. I've had employees on probation for lack of produc

One major thing I learned from 360 reviews is how bad developers are at judging each other or their manager. I've had employees on probation for lack of productivity judged as excellent by their peers because of their ability to socialize their "abilities". I had the best developer on my team (a 10x developer if ever there was one) get slammed on technical ability rankings because of his prickly personality. My VP and I used have quite the chuckle at all my "strengths" identified by my team and peers ... because they liked me and I was effective they attributed positive characteristics to me that I did not have.

ddoolin 13y ago
I've always been highly sociable compared to my other coworkers and have also noticed I get attributed things that weren't really accurate. Notably non-technical coworkers seem to think I work at break-neck pace when I really just work at pace with everyone else, most likely because I actually talk about what I'm doing with them more often.
YZF 13y ago
360 isn't perfect but it's better than the alternative. You're basically saying that those who worked daily in close interaction with the 10x developer and had to work with the code that developer produced, had lower regard for his technical ability than a manager who presumably is further removed? You can't ignore the social aspect. A team with social frictions isn't going to work as well as a team that enjoys working together. Obviously just being social isn't enough but a strong team needs a combination of the social and technical aspects. Also 360 should mean going beyond the team, so the manager should be evaluated by his peers, his managers and his team and technical people on the team should be evaluated by members of other teams they work with.
- Ensorceled 13y ago
  Yes, I know what a 360 is, I've participated in them. In the situations I indicated I was also a developer on the team so I knew how productive and skilled the 10x prickly developer was. His peers ranked him high on technical skills but only as high as more amicable and lessor skilled peers. I'm not saying I ignored the social aspect, in fact that is why I called him "prickly". I'm just saying that, while his teamwork related areas were properly low, his technical areas were improperly low because of this halo effect.
  - specialist 13y ago
    Did you manage a self-selected team? I swiped the Roman Evaluation Method from Luke Hohmann. During interviews, a No from one is a No from all. Self selecting teams is one of those agile advices. If your prickly superstar was chosen collectively, he would have been given a lot of latitude. Eyes wide open kinda thing. The only sticky interpersonal problems I've had is when some higher ranking genius assigns people to my teams, mostly unwanted and unwelcome. And sometimes just plan bad soup. People good individually, bad together. Like when the boss assigned an outspoken Lebonese and an outspoken Israeli to my team. They got along as well as peanut butter and lug nuts.
    - gtirloni 13y ago
      Self-selected teams while good from that perspective actually have a huge downside: politics and pleasing your friends. Next thing that happens is people start choosing members based on how much comfortable they are (which doesn't translate into technical skills and delivering stuff) or start talking to each other, trying to convince people.. this sucks! While it might work for small companies with not much variation in technical skills and sociability between employees.. for anything else it's kinda hard. Plus, that initial list of 2-3 employees that start talking and picking others was selected by someone, wasn't it? And the list that they can pick from was also selected, no? I think if you need a team of 10 and you have 30 possibly good candidates within your company (and they are all available to move to your team).. sure, why not. You as a manager already pre-selects a bunch of people and then let your team members filter it even further. But putting that responsibility from the beginning solely on the team members.. I guess that's risky. But anyway, how often does that happen? If we are talking about selecting 1-2 people for a already formed team than fine.. of course other team members should do it (but that's mostly the norm in a lot of companies.. the manager/tech lead runs the resumes through people he/she trusts, ask them to participate in interviews.. no big difference there).
  - munificent 13y ago
    > I knew how productive and skilled the 10x prickly developer was How? What makes you so sure you weren't biased?
- dreamfactory 13y ago
  Makes me wonder if he wasn't a prolific tech debt producer who was seen by management to GTD.
  - Ensorceled 13y ago
    No, but I know exactly what you mean. Years ago I was on a team where the "super star" was assigned all new work. The rest of us had to fix the bugs in their code. At the finale, 5 people were doing nothing but fixing bugs in the "stars" previous projects. That's when senior management finally clued in to the problem and fired both the manager and the "super star".
    - dreamfactory 13y ago
      going by the upvotes, we aren't the only people to have seen this anti-pattern...
      - Ensorceled 13y ago
        Yeah, this is why I kind of like project ownership even though it's now considered out of fashion. If you create technical debt, you own it and it's your future productivity that takes the hit. Eventually your productivity grinds to a halt as you fix problems in the crap you wrote before.
        dreamfactory 13y ago
        I look at it as technically informed product ownership - merging business and technical domain knowledge to make the right tradeoffs. How else can you execute effectively?
lmg643 13y ago
I once had a similar experience - in an interview situation. (I can be likeable in person, just not in comments...) The guy was attributing statements to me, from the conversation we were having in that moment, that i had never said. He would even say "what you said, or what you were going to say..." I found it extremely bizarre. I am a pretty precise person, and I wanted to correct him but i realized it is pointless. the lesson i take away was powerful - if people "like" you there is almost no level they won't go to make it work out. one thing we can do about this is cry about it. the other thing we can do is to figure out how to "hack" likability. i think my mother taught me most of the basics already. of course - much easier to do this for one hour in an interview, than for an entire year at a job. but there is something to be said for smiling, a few kind words, and then ... graceful evasion of everything else when you want to do real work.
- 300bps 13y ago
  the other thing we can do is to figure out how to "hack" likability Actually, the key to life is sincerity. Once you can fake that, you've got it made.
  - gboudrias 13y ago
    Protip to all non-sociopaths: You can't fake sincerity. Trying will backfire in the long term.
    - 300bps 13y ago
      Protip to all non-sociopaths: You can't fake sincerity. Trying will backfire in the long term. You seem to know a lot about psychology. What do you call someone who doesn't recognize jokes?
      - mynameishere 13y ago
        What do you call someone who posts ancient, moldy quotes?
tensor 13y ago
What makes you think that your judgements are any better than theirs? How do you judge performance? Do you look at code? Or do you simply go by how many feature check boxes they get through in a given time frame? edit: and I don't ask this to be insulting, but judging performance is a very difficult task and I've never seen a reliable methodology for doing it. Thus, I'm naturally extremely skeptical when someone claims that they can do it better than others. Especially so when it includes broad generalizations about a group of people.
- Ensorceled 13y ago
  I'm usually one of the strongest developers on my team, even when I'm managing. It's one of my strengths and why I stay with small companies. It also means I'm qualified to judge the technical skills of my developers, I know how long it should take. People who think judging performance is hard have a broken development methodology or no technical chops. That's also why I included the comment on my own rankings; I have some pretty serious weaknesses that were ranked as areas of strengths by my peers and reports.
  - tensor 13y ago
    >People who think judging performance is hard have a broken development methodology or no technical chops. And this is exactly why I asked the question. Here you have judged me, since I said judging performance was hard, on both my development methodology and technical chops. Yet you know nothing about me at all. The problem of evaluation is an important one, and one I'm always interested in reading more on. Unfortunately the strategy of taking one's self as a gold standard isn't exactly what I was looking for. It does seem obvious that being an expert in your domain is a necessary trait to be able to evaluate others, but it's not at all obvious that it is a sufficient trait. The idea of multi-source feedback seems intuitively good, despite the conflicting research on it. That more input will expose biases or inaccuracies is a decent hypothesis. On the other hand, judgment from a single person seems highly error prone. Whatever faults that one person has will never be exposed and reviews seem almost guaranteed to have bias. In the end, I'm left with no idea why you so quickly discard the reports of your peers.
jredwards 13y ago
Likability is underrated. Someone who is easy to work with, improves workplace morale, and makes other developers feel comfortable at work is a valuable asset. Certainly, it shouldn't be a substitute for technical ability, but it can be a very significant complimentary attribute.
- Ensorceled 13y ago
  Agreed. Team work is critical, but likability should affect the soft rankings only but instead had an overall positive or negative effect, even on technical skill rankings.
- meowface 13y ago
  The problem is when likability affects how someone reviews your technical abilities. It should be a separate category, but most people would have significant trouble disassociating the two; more so if you're friends with a coworker.
  - overgryphon 13y ago
    Is it so important that the individual components of a person's performance be evaluated individually? What about someone with exceptional technical abilities who uses those skills to ensure that blame for potential failure is always assigned to other people, rather than focus those impressive skills on the success of the overall project? Is it so important that person receive recognition for their technical skills? If someone's social behavior jeopardizes the success of projects they work on, then perhaps it doesn't matter how technically competent they are.
    - xymostech 13y ago
      The answer to your question is probably "no". However, most of the discussion was in reference to the stack ranking at Microsoft, where people's pay was being determined by how well they were ranked by their peers. If you were generally likable, people might also say that your coding skills were better than they really were. So, I think that you're right, and it's a good reason why the stack system didn't work very well.
tiglionabbit 13y ago
This is a perfect description of the Halo Effect. http://en.wikipedia.org/wiki/Halo_effect http://en.wikipedia.org/wiki/Halo_effect
- HelloMcFly 13y ago
  It's a good description that goes in both directions. 360 reviews are popular because they reduce single-source bias (a subclass of Common Method Variance). Different perspective of an employee can helps account for unique variance in the employees' true performance. From my perspective Ensorceled's comment demonstrate their value rather than their shortcomings.
  - Ensorceled 13y ago
    Exactly! I love 360 reviews because, in part, they help mitigate the halo effect. They were also good at revealing the halo effect.
deleted 13y ago
[deleted]
michaelt 13y ago
What's the point of doing a 360 review if you're going to attribute away all the results you don't agree with to bias? Why not just do the reviews yourself and save everyone else from wasting their time :)
- Ensorceled 13y ago
  I'm not sure why everybody is assuming I toss all of these reviews. They are not useless even when biased. The reason you do 360's is that you have to take every individual review with a grain of salt. When you have 7-10 reviews the biases start to average out.
kabouseng 13y ago
Thats a fair comment, but in a similar vein; do you trust your own ability (as manager I presume) to evaluate the work output and business value of the employees? Or otherwise do you trust a / your manager with this? One thing that is true, is often times the manager has more information about the various reasons for certain choices and goings on in the business, and thus is in a better position to evaluate people. The question then is just, how do you make sure that manager do not discriminate based on soft indicators (personalities match / clashes / favouritism etc.).
mathattack 13y ago
It's very hard to separate emotional liking of people with being able to rate their abilities, since rating for most white collar jobs is so subjective. This is why forced stacking can be so insidious, and also why ratings frequently seem unfair. It's also why office politics creep in to the process. People trade on their likeability because they're not confident that their skills will be properly weighted.

ddoolin 13y ago

I've always been highly sociable compared to my other coworkers and have also noticed I get attributed things that weren't really accurate. Notably non-technical coworkers seem to think I work at break-neck pace when I really just work at pace with everyone else, most likely because I actually talk about what I'm doing with them more often.

YZF 13y ago

360 isn't perfect but it's better than the alternative. You're basically saying that those who worked daily in close interaction with the 10x developer and had to work with the code that developer produced, had lower regard for his technical ability than a manager who presumably is further removed? You can't ignore the social aspect. A team with social frictions isn't going to work as well as a team that enjoys working together. Obviously just being social isn't enough but a strong team needs a combination of the social and technical aspects. Also 360 should mean going beyond the team, so the manager should be evaluated by his peers, his managers and his team and technical people on the team should be evaluated by members of other teams they work with.

Ensorceled 13y ago
Yes, I know what a 360 is, I've participated in them. In the situations I indicated I was also a developer on the team so I knew how productive and skilled the 10x prickly developer was. His peers ranked him high on technical skills but only as high as more amicable and lessor skilled peers. I'm not saying I ignored the social aspect, in fact that is why I called him "prickly". I'm just saying that, while his teamwork related areas were properly low, his technical areas were improperly low because of this halo effect.
- specialist 13y ago
  Did you manage a self-selected team? I swiped the Roman Evaluation Method from Luke Hohmann. During interviews, a No from one is a No from all. Self selecting teams is one of those agile advices. If your prickly superstar was chosen collectively, he would have been given a lot of latitude. Eyes wide open kinda thing. The only sticky interpersonal problems I've had is when some higher ranking genius assigns people to my teams, mostly unwanted and unwelcome. And sometimes just plan bad soup. People good individually, bad together. Like when the boss assigned an outspoken Lebonese and an outspoken Israeli to my team. They got along as well as peanut butter and lug nuts.
  - gtirloni 13y ago
    Self-selected teams while good from that perspective actually have a huge downside: politics and pleasing your friends. Next thing that happens is people start choosing members based on how much comfortable they are (which doesn't translate into technical skills and delivering stuff) or start talking to each other, trying to convince people.. this sucks! While it might work for small companies with not much variation in technical skills and sociability between employees.. for anything else it's kinda hard. Plus, that initial list of 2-3 employees that start talking and picking others was selected by someone, wasn't it? And the list that they can pick from was also selected, no? I think if you need a team of 10 and you have 30 possibly good candidates within your company (and they are all available to move to your team).. sure, why not. You as a manager already pre-selects a bunch of people and then let your team members filter it even further. But putting that responsibility from the beginning solely on the team members.. I guess that's risky. But anyway, how often does that happen? If we are talking about selecting 1-2 people for a already formed team than fine.. of course other team members should do it (but that's mostly the norm in a lot of companies.. the manager/tech lead runs the resumes through people he/she trusts, ask them to participate in interviews.. no big difference there).
- munificent 13y ago
  > I knew how productive and skilled the 10x prickly developer was How? What makes you so sure you weren't biased?
dreamfactory 13y ago
Makes me wonder if he wasn't a prolific tech debt producer who was seen by management to GTD.
- Ensorceled 13y ago
  No, but I know exactly what you mean. Years ago I was on a team where the "super star" was assigned all new work. The rest of us had to fix the bugs in their code. At the finale, 5 people were doing nothing but fixing bugs in the "stars" previous projects. That's when senior management finally clued in to the problem and fired both the manager and the "super star".
  - dreamfactory 13y ago
    going by the upvotes, we aren't the only people to have seen this anti-pattern...
    - Ensorceled 13y ago
      Yeah, this is why I kind of like project ownership even though it's now considered out of fashion. If you create technical debt, you own it and it's your future productivity that takes the hit. Eventually your productivity grinds to a halt as you fix problems in the crap you wrote before.
      - dreamfactory 13y ago
        I look at it as technically informed product ownership - merging business and technical domain knowledge to make the right tradeoffs. How else can you execute effectively?

lmg643 13y ago

I once had a similar experience - in an interview situation. (I can be likeable in person, just not in comments...) The guy was attributing statements to me, from the conversation we were having in that moment, that i had never said. He would even say "what you said, or what you were going to say..." I found it extremely bizarre. I am a pretty precise person, and I wanted to correct him but i realized it is pointless. the lesson i take away was powerful - if people "like" you there is almost no level they won't go to make it work out. one thing we can do about this is cry about it. the other thing we can do is to figure out how to "hack" likability. i think my mother taught me most of the basics already. of course - much easier to do this for one hour in an interview, than for an entire year at a job. but there is something to be said for smiling, a few kind words, and then ... graceful evasion of everything else when you want to do real work.

300bps 13y ago
the other thing we can do is to figure out how to "hack" likability Actually, the key to life is sincerity. Once you can fake that, you've got it made.
- gboudrias 13y ago
  Protip to all non-sociopaths: You can't fake sincerity. Trying will backfire in the long term.
  - 300bps 13y ago
    Protip to all non-sociopaths: You can't fake sincerity. Trying will backfire in the long term. You seem to know a lot about psychology. What do you call someone who doesn't recognize jokes?
    - mynameishere 13y ago
      What do you call someone who posts ancient, moldy quotes?

tensor 13y ago

What makes you think that your judgements are any better than theirs? How do you judge performance? Do you look at code? Or do you simply go by how many feature check boxes they get through in a given time frame? edit: and I don't ask this to be insulting, but judging performance is a very difficult task and I've never seen a reliable methodology for doing it. Thus, I'm naturally extremely skeptical when someone claims that they can do it better than others. Especially so when it includes broad generalizations about a group of people.

Ensorceled 13y ago
I'm usually one of the strongest developers on my team, even when I'm managing. It's one of my strengths and why I stay with small companies. It also means I'm qualified to judge the technical skills of my developers, I know how long it should take. People who think judging performance is hard have a broken development methodology or no technical chops. That's also why I included the comment on my own rankings; I have some pretty serious weaknesses that were ranked as areas of strengths by my peers and reports.
- tensor 13y ago
  >People who think judging performance is hard have a broken development methodology or no technical chops. And this is exactly why I asked the question. Here you have judged me, since I said judging performance was hard, on both my development methodology and technical chops. Yet you know nothing about me at all. The problem of evaluation is an important one, and one I'm always interested in reading more on. Unfortunately the strategy of taking one's self as a gold standard isn't exactly what I was looking for. It does seem obvious that being an expert in your domain is a necessary trait to be able to evaluate others, but it's not at all obvious that it is a sufficient trait. The idea of multi-source feedback seems intuitively good, despite the conflicting research on it. That more input will expose biases or inaccuracies is a decent hypothesis. On the other hand, judgment from a single person seems highly error prone. Whatever faults that one person has will never be exposed and reviews seem almost guaranteed to have bias. In the end, I'm left with no idea why you so quickly discard the reports of your peers.

jredwards 13y ago

Likability is underrated. Someone who is easy to work with, improves workplace morale, and makes other developers feel comfortable at work is a valuable asset. Certainly, it shouldn't be a substitute for technical ability, but it can be a very significant complimentary attribute.

Ensorceled 13y ago
Agreed. Team work is critical, but likability should affect the soft rankings only but instead had an overall positive or negative effect, even on technical skill rankings.
meowface 13y ago
The problem is when likability affects how someone reviews your technical abilities. It should be a separate category, but most people would have significant trouble disassociating the two; more so if you're friends with a coworker.
- overgryphon 13y ago
  Is it so important that the individual components of a person's performance be evaluated individually? What about someone with exceptional technical abilities who uses those skills to ensure that blame for potential failure is always assigned to other people, rather than focus those impressive skills on the success of the overall project? Is it so important that person receive recognition for their technical skills? If someone's social behavior jeopardizes the success of projects they work on, then perhaps it doesn't matter how technically competent they are.
  - xymostech 13y ago
    The answer to your question is probably "no". However, most of the discussion was in reference to the stack ranking at Microsoft, where people's pay was being determined by how well they were ranked by their peers. If you were generally likable, people might also say that your coding skills were better than they really were. So, I think that you're right, and it's a good reason why the stack system didn't work very well.

tiglionabbit 13y ago

This is a perfect description of the Halo Effect. http://en.wikipedia.org/wiki/Halo_effect http://en.wikipedia.org/wiki/Halo_effect

HelloMcFly 13y ago
It's a good description that goes in both directions. 360 reviews are popular because they reduce single-source bias (a subclass of Common Method Variance). Different perspective of an employee can helps account for unique variance in the employees' true performance. From my perspective Ensorceled's comment demonstrate their value rather than their shortcomings.
- Ensorceled 13y ago
  Exactly! I love 360 reviews because, in part, they help mitigate the halo effect. They were also good at revealing the halo effect.

deleted 13y ago

[deleted]

michaelt 13y ago

What's the point of doing a 360 review if you're going to attribute away all the results you don't agree with to bias? Why not just do the reviews yourself and save everyone else from wasting their time :)

Ensorceled 13y ago
I'm not sure why everybody is assuming I toss all of these reviews. They are not useless even when biased. The reason you do 360's is that you have to take every individual review with a grain of salt. When you have 7-10 reviews the biases start to average out.

kabouseng 13y ago

Thats a fair comment, but in a similar vein; do you trust your own ability (as manager I presume) to evaluate the work output and business value of the employees? Or otherwise do you trust a / your manager with this? One thing that is true, is often times the manager has more information about the various reasons for certain choices and goings on in the business, and thus is in a better position to evaluate people. The question then is just, how do you make sure that manager do not discriminate based on soft indicators (personalities match / clashes / favouritism etc.).

mathattack 13y ago

It's very hard to separate emotional liking of people with being able to rate their abilities, since rating for most white collar jobs is so subjective. This is why forced stacking can be so insidious, and also why ratings frequently seem unfair. It's also why office politics creep in to the process. People trade on their likeability because they're not confident that their skills will be properly weighted.