4 ms·

I'm a software engineer turned software engineering manager. What I'm about to say is not gospel but a way I've found/used to measure productivity within my cur

I'm a software engineer turned software engineering manager. What I'm about to say is not gospel but a way I've found/used to measure productivity within my current and previous organization. The number of pull requests and/or work items completed. It's not perfect, but when the numbers are crunched over the course of 6+ months or longer then it approximately aligns with our anecdotal observations about "our best developers". Metrics that (I believe) are important to track. 1. number of days (for a new employee) to complete first pull request. 2. number of pull requests complete within first thirty days. 3. number of pull requests complete within first six months. 4. number of kickbacks (a work item that goes back to development because it failed testing). #4 is kind of difficult to measure. The work item systems that I have experience with do not have an easy way to track this metric so it takes effort to produce/track. Finally, I'm transparent with my direct reports about this information and that I believe it's important. I ask them if they feel it's fair, or if there is another metric/measurement I should use or look at to assess productivity/performance. Thus far, no one has yet to bring anything serious to me so either people don't really care or they feel this is fair/sufficient (or maybe I'm a giant smug jerk who is hostile to feedback from subordinates, I don't believe this to be the case but this is Hacker News after all so very prepared for this response).

mandeepj 4y ago
Number of pull requests can be gamed and hard to measure - easy vs complex tasks. I'm surprised you did not mention anything about Velocity points.
- hnrodey 4y ago
  I'm not surprised to see your criticism that is missing an alternative way to measure. The answer is not "measuring developer productivity is impossible". Otherwise, I'd challenge you to provide a metric that cannot be gamed. Humans respond to incentives and thus there will always be people that could/will intentionally try to rise above the rest based purely on the measurement.
  - taeric 4y ago
    My alternative suggestion would be to worry less about individuals and more worry about if the organization is producing something of value. After that, you can start to pick and prod at the individual contributions if you want, but expect that any team will likely have a step function that is their effectiveness. As such, you may remove the least functioning member of a team, and then find that you dropped the entire team to nothing. It is almost certainly true that there can be drags on the morale of a team. People that are causing things to go slower in detrimental ways. It is also almost certainly true that sometimes having someone around that is just fun to talk to for the rest of the team is enough to keep the entire team functioning.
  - nradov 4y ago
    Measuring individual developer productivity is impossible without also creating perverse incentives that destroy business value. The smallest unit you can usefully measure is a single Scrum team (and even at that level most metrics will have wide error bars and a lack of statistical significance).
    - musicale 4y ago
      > Measuring individual developer productivity is impossible without also creating perverse incentives that destroy business value. Probably should be printed at the top and bottom of every page of every performance review.
- bluesnowmonkey 4y ago
  I agree with hnrodey, and here’s the thing: trying to game this metric is a pretty good way to turn into an actually productive programmer! People who crank out lots of small, easy tasks build up a lot of the very skills they need to take on large, complex projects. Another way to put this is, I’ve never encountered a highly productive programmer whose approach was to occasionally land a big important change. It just doesn’t work out that way.
varun_ex 4y ago
This is an interesting approach. I wonder how many EMs are currently following this approach. DO you use any tool to track these data?
pydry 4y ago
https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
guhcampos 4y ago
You must be soo easily fooled by low-hanging-fruit hogs
- stronglikedan 4y ago
  Likely weighted by (at least) story points (I hope).
dropit_sphere 4y ago
This is a great discovery! Have you tried tying developer pay directly to these metrics?
musicale 4y ago
> The number of pull requests and/or work items completed Good idea in an environment such as an assembly line where the same tasks are repeated over and over. Probably useless in a software development project where the tasks are different every time.

mandeepj 4y ago

Number of pull requests can be gamed and hard to measure - easy vs complex tasks. I'm surprised you did not mention anything about Velocity points.

hnrodey 4y ago
I'm not surprised to see your criticism that is missing an alternative way to measure. The answer is not "measuring developer productivity is impossible". Otherwise, I'd challenge you to provide a metric that cannot be gamed. Humans respond to incentives and thus there will always be people that could/will intentionally try to rise above the rest based purely on the measurement.
- taeric 4y ago
  My alternative suggestion would be to worry less about individuals and more worry about if the organization is producing something of value. After that, you can start to pick and prod at the individual contributions if you want, but expect that any team will likely have a step function that is their effectiveness. As such, you may remove the least functioning member of a team, and then find that you dropped the entire team to nothing. It is almost certainly true that there can be drags on the morale of a team. People that are causing things to go slower in detrimental ways. It is also almost certainly true that sometimes having someone around that is just fun to talk to for the rest of the team is enough to keep the entire team functioning.
- nradov 4y ago
  Measuring individual developer productivity is impossible without also creating perverse incentives that destroy business value. The smallest unit you can usefully measure is a single Scrum team (and even at that level most metrics will have wide error bars and a lack of statistical significance).
  - musicale 4y ago
    > Measuring individual developer productivity is impossible without also creating perverse incentives that destroy business value. Probably should be printed at the top and bottom of every page of every performance review.
bluesnowmonkey 4y ago
I agree with hnrodey, and here’s the thing: trying to game this metric is a pretty good way to turn into an actually productive programmer! People who crank out lots of small, easy tasks build up a lot of the very skills they need to take on large, complex projects. Another way to put this is, I’ve never encountered a highly productive programmer whose approach was to occasionally land a big important change. It just doesn’t work out that way.

varun_ex 4y ago

This is an interesting approach. I wonder how many EMs are currently following this approach. DO you use any tool to track these data?

pydry 4y ago

https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

guhcampos 4y ago

You must be soo easily fooled by low-hanging-fruit hogs

stronglikedan 4y ago
Likely weighted by (at least) story points (I hope).

dropit_sphere 4y ago

This is a great discovery! Have you tried tying developer pay directly to these metrics?

musicale 4y ago

> The number of pull requests and/or work items completed Good idea in an environment such as an assembly line where the same tasks are repeated over and over. Probably useless in a software development project where the tasks are different every time.