4 ms·

Structuring the review around objectively verifiable data creates other problems. What if there's a developer on the team who delivers 1/3rd the features as ev

Structuring the review around objectively verifiable data creates other problems. What if there's a developer on the team who delivers 1/3rd the features as everyone else, but they offer such effective mentorship that their five teammates deliver features 2x faster? If you design your review process around objectively verifiable data, then it's almost certain to undervalue a developer who's great at mentorship because mentorship is hard to measure. And it's not just mentorship. There are tons of contributions that seem pretty obviously beneficial but aren't objectively measurable: recognizing simpler ways of achieving business objectives, communicating clearly and effectively, being pleasant to be around, etc. But if you go the other way and leave it up to pure human judgement, then you end up with promotion being based on favoritism, office politics, and irrational biases.

throwway120385 3y ago
> But if you go the other way and leave it up to pure human judgement, then you end up with promotion being based on favoritism, office politics, and irrational biases. And my personal favorite: luck of the draw.
willcipriano 3y ago
> who delivers 1/3rd the features as everyone else, but they offer such effective mentorship that their five teammates deliver features 2x faster? I haven't really encountered this. Generally speaking people who can teach others to be much more productive like that are unsurprisingly themselves, more productive. This fictional person would be better suited to a people management or coaching role as a individual contractor they only achieve a third of the average.
- zachmu 3y ago
  Sometimes you get the cranky old guy archetype who doesn't really get much done but is really good at making sure the kids don't make any really dumb mistakes, which is its own kind of value
- tanjtanjtanj 3y ago
  Every team I've ever worked with (11) has had at least one person that would meet this archetype. Their team-based contribution usually wilts under the kind of individualized metrics being pushed for by some here and the team suffers as a result even if their own metrics get better.
  - willcipriano 3y ago
    Then do my second case, this person gets a different title so we can evaluate them differently. Now with the major objection to objective performance review gone we can do that for the actual individual contributors.
    - tanjtanjtanj 3y ago
      I have no need to do that as I'm actually involved in my teams and don't need arbitrary numbers to see that someone is contributing. I'm fortunate that I have been able to avoid (or actively dodge) metric perverts in my career.
- skeeter2020 3y ago
  this type of person is exactly who we try to identify for staff and higher roles. They could be 20% or even 50% more productive as an individual, but if they raise everyone's game even 2% across 50 or more people they are producing far more value. The reality is they crush fewer tickets individually, and that's a good thing, but it gets perverted by a purely metrics-driven assessment.
  - milesvp 3y ago
    My experience is that the best devs should not be crushing tickets. Just like the best surgeons have some of the worst survival rates, because they get the hardest patients, the best engineers should be working on the hardest problems the team faces. They should be spending a lot more time doing research, de-risking and clearing the way so that the rest of the team can crush the tickets.
    - willcipriano 3y ago
      Not just tickets. Lines of code (not just in main but in branches), deployments, runs of the automated build/test suite, lines of documentation written, number of responses in your company technical emergency slack channel, after hours pager incidents resolved, etc, etc If the people near the top of those leader boards are near the top of the comp leaderboard your job is done as that makes sense by the numbers, if not you need a really good justification as to why* if you want to keep those literal top performers for long. *or crossed fingers nobody discusses comp before you exit
      - drewcoo 3y ago
        These things are vanity metrics. > Lines of code (not just in main but in branches) IMO, the best devs can end up with negative LOCs per measurement period. Neophytes will never have negative LOCs. Mediocre devs who copy and paste a lot can have really high LOCs. > lines of documentation written And the more documentation you have the more out of sync it is with reality.
        3y ago
zachmu 3y ago
People making lots of soft contributions like mentorship / leadership tend to be the kind of well-liked personable sorts who do well on non-objective assessments, which always manage to color evaluations even when you try to make them as objective as possible.
- AdrianB1 3y ago
  Not in my experience. I worked in 2 occasions with some older people (~ 65) that were the guru of their departments, very respected and listened. While people learned a lot from these 2 people, they were not too agreeable, but quite grumpy, I could say. That limited their careers, even if they were by far the best experts in their departments and overall the most valuable contributors, they were never considered for promotions. On the other side, most well-liked people I met were professionally superficial at best, masters of talking and networking, but not delivering much at the end of the day. They were promoted first, so that the entire organization was full of these people promoting similar ones.
  - sgarland 3y ago
    Same. The best people I ever worked with were grumpy graybeards (in spirit if not physically). Brusque is the kindest way to describe them. If you knew your shit, or at least demonstrated that you had exhausted your knowledge, they would help you. That said, it isn’t _required_ for you to be a jerk; I’ve also worked with incredibly talented people who were generally kind. But they did not go out of their way to network and schmooze. IME, people who are concerned with what others think of them are the least accomplished.
  - pyuser583 3y ago
    I wonder if something about technical competence corresponds to being a jerk. I’ve seen this happen again and again. Nice people become jerks when they become good at tech. It doesn’t always happen. Plenty of compassionate tech wizards. I suspect it’s due to the fact that being a jerk gets reactions quicker. “You messed this up, fix it like I tell you or I’ll ruin your reputation” gets more effect than “Let me show you how this works.” I also suspect that programmers are often so lost in their heads, they don’t take good advice when it’s given. I’ve been guilty of this. I’ve decided a problem is from X part of the stack, that when an expert in Y says to look in Y I’m like “but that doesn’t effect X!”

throwway120385 3y ago

> But if you go the other way and leave it up to pure human judgement, then you end up with promotion being based on favoritism, office politics, and irrational biases. And my personal favorite: luck of the draw.

willcipriano 3y ago

> who delivers 1/3rd the features as everyone else, but they offer such effective mentorship that their five teammates deliver features 2x faster? I haven't really encountered this. Generally speaking people who can teach others to be much more productive like that are unsurprisingly themselves, more productive. This fictional person would be better suited to a people management or coaching role as a individual contractor they only achieve a third of the average.

zachmu 3y ago
Sometimes you get the cranky old guy archetype who doesn't really get much done but is really good at making sure the kids don't make any really dumb mistakes, which is its own kind of value
tanjtanjtanj 3y ago
Every team I've ever worked with (11) has had at least one person that would meet this archetype. Their team-based contribution usually wilts under the kind of individualized metrics being pushed for by some here and the team suffers as a result even if their own metrics get better.
- willcipriano 3y ago
  Then do my second case, this person gets a different title so we can evaluate them differently. Now with the major objection to objective performance review gone we can do that for the actual individual contributors.
  - tanjtanjtanj 3y ago
    I have no need to do that as I'm actually involved in my teams and don't need arbitrary numbers to see that someone is contributing. I'm fortunate that I have been able to avoid (or actively dodge) metric perverts in my career.
skeeter2020 3y ago
this type of person is exactly who we try to identify for staff and higher roles. They could be 20% or even 50% more productive as an individual, but if they raise everyone's game even 2% across 50 or more people they are producing far more value. The reality is they crush fewer tickets individually, and that's a good thing, but it gets perverted by a purely metrics-driven assessment.
- milesvp 3y ago
  My experience is that the best devs should not be crushing tickets. Just like the best surgeons have some of the worst survival rates, because they get the hardest patients, the best engineers should be working on the hardest problems the team faces. They should be spending a lot more time doing research, de-risking and clearing the way so that the rest of the team can crush the tickets.
  - willcipriano 3y ago
    Not just tickets. Lines of code (not just in main but in branches), deployments, runs of the automated build/test suite, lines of documentation written, number of responses in your company technical emergency slack channel, after hours pager incidents resolved, etc, etc If the people near the top of those leader boards are near the top of the comp leaderboard your job is done as that makes sense by the numbers, if not you need a really good justification as to why* if you want to keep those literal top performers for long. *or crossed fingers nobody discusses comp before you exit
    - drewcoo 3y ago
      These things are vanity metrics. > Lines of code (not just in main but in branches) IMO, the best devs can end up with negative LOCs per measurement period. Neophytes will never have negative LOCs. Mediocre devs who copy and paste a lot can have really high LOCs. > lines of documentation written And the more documentation you have the more out of sync it is with reality.
      - 3y ago

zachmu 3y ago

People making lots of soft contributions like mentorship / leadership tend to be the kind of well-liked personable sorts who do well on non-objective assessments, which always manage to color evaluations even when you try to make them as objective as possible.

AdrianB1 3y ago
Not in my experience. I worked in 2 occasions with some older people (~ 65) that were the guru of their departments, very respected and listened. While people learned a lot from these 2 people, they were not too agreeable, but quite grumpy, I could say. That limited their careers, even if they were by far the best experts in their departments and overall the most valuable contributors, they were never considered for promotions. On the other side, most well-liked people I met were professionally superficial at best, masters of talking and networking, but not delivering much at the end of the day. They were promoted first, so that the entire organization was full of these people promoting similar ones.
- sgarland 3y ago
  Same. The best people I ever worked with were grumpy graybeards (in spirit if not physically). Brusque is the kindest way to describe them. If you knew your shit, or at least demonstrated that you had exhausted your knowledge, they would help you. That said, it isn’t _required_ for you to be a jerk; I’ve also worked with incredibly talented people who were generally kind. But they did not go out of their way to network and schmooze. IME, people who are concerned with what others think of them are the least accomplished.
- pyuser583 3y ago
  I wonder if something about technical competence corresponds to being a jerk. I’ve seen this happen again and again. Nice people become jerks when they become good at tech. It doesn’t always happen. Plenty of compassionate tech wizards. I suspect it’s due to the fact that being a jerk gets reactions quicker. “You messed this up, fix it like I tell you or I’ll ruin your reputation” gets more effect than “Let me show you how this works.” I also suspect that programmers are often so lost in their heads, they don’t take good advice when it’s given. I’ve been guilty of this. I’ve decided a problem is from X part of the stack, that when an expert in Y says to look in Y I’m like “but that doesn’t effect X!”