7 ms·

I'll ask again: What is the objective and repeatable metric of programmer productivity? One of the reasons I avoid writing about productivity is that I have no

I'll ask again: What is the objective and repeatable metric of programmer productivity? One of the reasons I avoid writing about productivity is that I have no idea how to quantify it. But if someone can show me how to measure it, I'm confident I can show him how to get paid by it.

c2 16y ago
There is none. In fact this whole good programmers are 10x-100x more productive then average programmers is treated as gospel around here, but I don't trust that either as a first principal or obvious assumption. In my experience, there are good programmers and bad programmers, just like good project managers, bad project managers, good people managers, bad people managers, etc. A bad employee is bad for your company period.
- 6ren 16y ago
  Not just here, but industry-wide. The x10 figure comes from studies. Brooks talks about them (mythical man-month), and I think Peopleware mentions some. The question then turns to what these studies are actually measuring... But that's just for programming. When you get into the application of software - unmet needs - you can easily get x100 or far far higher. This is because the value of software is more closely related to the need it meets (that exists in users) rather than any quality of the software (that exists in code).
palish 16y ago
The only objective and repeatable metric of programmer productivity is if a single programmer is assigned the task of delivering a tool or component, from scratch, by himself. His productivity is the inverse of the time to delivery. And that doesn't take into account maintenance.
- mchusma 16y ago
  Unfortunately employers (myself included) like to have different programmers working on different tasks, where it can be difficult to measure non-equivalent tasks.
- matthiaswh 16y ago
  Productivity is a factor of efficiency and effectiveness. Efficiency being the measurement of how fast things get done, and effectiveness a measure of how well things get done (meeting the requirements, including all features, number of bugs/issues that crop up, quality of code). This is essentially the same thing you are saying, just broken down. Still, it's difficult (not impossible) to measure one programmer against the next in terms of productivity. Using these metrics it is quite possible to measure a programmer against himself over time. The problem is that measuring the necessary factors affects the level of productivity being measured.
  - dean 16y ago
    "This is essentially the same thing you are saying, just broken down." Actually, I took his meaning as productivity being only a measure of how fast the work gets done, your efficiency, and not of how well the code is written, your effectiveness. His comment: "And that doesn't take into account maintenance.", implies to me that code quality is not a factor. Unfortunately, this is the measure that seems to predominate, especially with non-technical managers. If it takes twice as long to do it right, all the non-technical manager sees is that it took twice as long. The subsequent reduction in maintenance time and cost from doing it right doesn't seem to get noticed. Of course, the ideal is to get the job done fast and right.
    - ultrasaurus 16y ago
      So much of it comes down to managers not being able to tell good code from bad. If you look at quality as a crapshoot (it's genuinely hard to tell architecture astronauts from good code) then indeterminate quality quickly beats indeterminate quality slowly.
- raganwald 16y ago
  that doesn't take into account maintenance This excellent insight takes us to a place where productivity is even harder to measure: How does one measure the value of a piece of software? Software that has subtle bugs flying under the rader of our test suite has some kind of negative value. Software that lowers the "productivity" (however we measure it) of future programmers who need to extend or change it has negative value associated with it. How do we measure that? Imagine the exact same formally specified requirements handed to four different programmers: The first programmer, "Ned," does the job in a straightforward fashion, and delivers working code passing all tests. "Fred" does the job using uncommon techniques (parser combinators, for example) in less time and produces less code. "Ed" proposes that if some of the requirements are relaxed, there is an open source solution that can do the job with trivial integration. The last programmer, "Jed," sees some commonality with some existing code and proposes that the scope be expanded to produce a new piece of infrastructure (message queues, web services, SOAP, &c) solving a more general problem. How do we judge Ned, Ed, Fred, and Jed?
  - palish 16y ago
    By how fun they are. As people. I'm convinced the only real-world metric that matters is "how much other people want to work with you". Being an easygoing, fun person facilitates that dramatically.
    - endtime 16y ago
      All (but not only) people with good people skills and zero productivity would be motivated to say this. Edit: I think I might have misunderstood what palish meant by "By how fun they are." to exclude productivity and focus entirely on personality. My mistake, if so.
      - alex_c 16y ago
        That might be true in the very short term, but in the medium- or long-term, no one actually enjoys working with "fun" people who have zero or negative productivity.
- HedgeMage 16y ago
  I disagree. It's faster to write garbage than it is to write good, clean code.
  - techdmn 16y ago
    I'm not sure I agree with this. Granted good design takes some extra work. But for the most part, in my experience, coders who write bad code also do it slowly (perhaps because they don't understand it), while those who write cleaning code are able to move more quickly. As a project grows, you have to refer back to your own work more often. The price for poor code starts to be paid almost as soon as the cursor leaves the page.
- T-hawk 16y ago
  > And that doesn't take into account maintenance. Which is the elephant in the development room. IMO, maintainability is exactly what good programmers should be judged on. Any programming project of any meaningful size and usefulness will spend at least an order of magnitude more time in maintenance and enhancements than in initial development. For the largest projects (Google, Facebook), maintainability becomes 1000 or more times more important than the original development time. Problem is, you can't quantify maintainability. What a good programmer contributes to maintainability and enhanceability is the absence of certain problems or classes of problems. Everything from choosing an exotic languange that nobody else can maintain, to the choice and organization of source control. We take that sort of thing for granted in HN type projects, but problems like "nobody knows how to compile that" are a really big problem at corporate enterprise scales, especially companies that are not fundamentally technologists, say finance or medical or shipping. An individual programmer is more likely to incur reward if he looks like he's playing the hero in a bad platform and ecosystem ("only this guy has the wizardry to manage that!") than if he built a good understandable platform in the first place.
  - chris_j 16y ago
    Very true. A friend of mine, who worked for a very large company at the time, once described the perfect system for achieving recognition as a developer. Firstly, write code that's full of bugs and is hard to maintain. Secondly, once the code is shipped and starts going wrong, make sure you step up to the plate and fix all the bugs that you created. That way, you get kudos for delivering code rapidly and for all the customer support work you do. Contrast that with the guy who writes good quality code who doesn't ship as rapidly and who doesn't have half as much customer contact.
    - joshuacc 16y ago
      I don't have the citation on hand, but this goes along with a study I read about customer loyalty. Customers who have a problem which is promptly corrected by the company are more loyal than customers that never have any problems.
sudhirc 16y ago
I agree but then why salaries of top mgmt. is not decided using the same criteria.
stcredzero 16y ago
A friend of mine used this successfully with groups practicing Test Driven Development: - First, institute "Test First" development - Randomly pick some fraction of new tests to be reviewed (Demand a minimum quality level.) - Measure productivity in terms of new passing tests completed with some multiplier for the quality score This system could be gamed, but it would require a conspiracy consisting of a large fraction of the team, and any system could be gamed in that situation. This is the only method I know of that works when analyzed logically and has been shown to work in practice.
- michaelbuckbee 16y ago
  I'd be concerned that the approach you laid out is something of a proxy for lines of code delivered. My most productive days are the ones where I've removed huge blocks of unnecessary code.
  - rhizome 16y ago
    KLOC mentality applied to TDD.
    - stcredzero 16y ago
      That would be correct, except that the tests are peer reviewed in my proposal. You can't "code me a new minivan" in this situation, unless the test review process gets corrupted.
  - stcredzero 16y ago
    I'd be concerned that the approach you laid out is something of a proxy for lines of code delivered. Absolutely incorrect. Did you actually read my proposal? If the tests are of high quality, then the code passing them will be substantive. Also, in new development, what matters is functional specs delivered, and in a properly run TDD project, these two are strongly related.
    - michaelbuckbee 16y ago
deleted 16y ago
[deleted]
dwc 16y ago
Delivering a quality product within time and budget constraints, consistently. Oh, but that's a team effort, you say? Well, that's the working environment, isn't it? Projects are an awfully coarse measurement, you say? Well, that's the only deliverable that matters. I recall reading years ago (in Peopleware or one of its contemporaries) about a company's evaluation of one of its coders. She was definitely mediocre by every measure they had. But someone noticed that every project she was on succeeded, over many projects and many years. Though she wasn't a monster at the keyboard, something she brought to the team engendered success. How productive was she? Would you want to hire her and have her on your team? You must measure what you actually care about. Measuring things that you think are factors is fine and noble, but if you're not measuring the actual "product" of "productivity" then you'll never know how well your factors correlate with the real goal.
- raganwald 16y ago
  But someone noticed that every project she was on succeeded, over many projects and many years. Though she wasn't a monster at the keyboard, something she brought to the team engendered success. Correlation does not equal causation. Another possible explanation: She was a monster at predicting project success and worked her way onto projects that were going to succeed with or without her.
  - dwc 16y ago
    Sure, in general. In this case managers were fairly familiar with her and the teams and posited that the success was due to what she added to team discussions. I think they were probably correct. OTOH, having a project success divining rod could have its own value.
  - henrybaxter 16y ago
    So ask her if she's willing to be on this project! Seriously though, ask your developer why there is a correlation between them and project success - if they can tell you, even if it shows they should be a business analyst, you're on to something very useful and valuable.
  - jerf 16y ago
Periodic 16y ago
I worked at an enterprise software company with a Scrum system that used two-week sprints. This means we would estimate all the tasks for the next two weeks in a big meeting at the beginning of the sprint, then at the end of the sprint we'd have another big meeting to close out the tasks and do any analysis on met/missed targets. Tasks tended to get separated so there wasn't too much direct collaboration between coders within a sprint, and the estimations were mostly a group effort, but the person who was getting assigned a task had final say to tweak the numbers. In this environment, it was fairly easy to see who was the more productive programmers. Some people got their tasks done quickly and could easily take parts of other features, and other people were usually late and needed others to help them finish their features. If you were a programmer there, you could easily rank the programmers based on productivity. I bet QA could do the same if they tried. However, management focused on it being a team effort. As long as we finished everything up by the end of the week they did their best to reward everyone and fire no one.
jarin 16y ago
I'm of the opinion that programming is more an art than a science, so to me asking the objective metric of programmer productivity is like asking the objective metric of painter productivity. In my experience, if you're technical you just kind of know if a programmer is good or not. If you're not technical, you find a technical person you trust and ask them.

c2 16y ago

There is none. In fact this whole good programmers are 10x-100x more productive then average programmers is treated as gospel around here, but I don't trust that either as a first principal or obvious assumption. In my experience, there are good programmers and bad programmers, just like good project managers, bad project managers, good people managers, bad people managers, etc. A bad employee is bad for your company period.

6ren 16y ago
Not just here, but industry-wide. The x10 figure comes from studies. Brooks talks about them (mythical man-month), and I think Peopleware mentions some. The question then turns to what these studies are actually measuring... But that's just for programming. When you get into the application of software - unmet needs - you can easily get x100 or far far higher. This is because the value of software is more closely related to the need it meets (that exists in users) rather than any quality of the software (that exists in code).

palish 16y ago

The only objective and repeatable metric of programmer productivity is if a single programmer is assigned the task of delivering a tool or component, from scratch, by himself. His productivity is the inverse of the time to delivery. And that doesn't take into account maintenance.

mchusma 16y ago
Unfortunately employers (myself included) like to have different programmers working on different tasks, where it can be difficult to measure non-equivalent tasks.
matthiaswh 16y ago
Productivity is a factor of efficiency and effectiveness. Efficiency being the measurement of how fast things get done, and effectiveness a measure of how well things get done (meeting the requirements, including all features, number of bugs/issues that crop up, quality of code). This is essentially the same thing you are saying, just broken down. Still, it's difficult (not impossible) to measure one programmer against the next in terms of productivity. Using these metrics it is quite possible to measure a programmer against himself over time. The problem is that measuring the necessary factors affects the level of productivity being measured.
- dean 16y ago
  "This is essentially the same thing you are saying, just broken down." Actually, I took his meaning as productivity being only a measure of how fast the work gets done, your efficiency, and not of how well the code is written, your effectiveness. His comment: "And that doesn't take into account maintenance.", implies to me that code quality is not a factor. Unfortunately, this is the measure that seems to predominate, especially with non-technical managers. If it takes twice as long to do it right, all the non-technical manager sees is that it took twice as long. The subsequent reduction in maintenance time and cost from doing it right doesn't seem to get noticed. Of course, the ideal is to get the job done fast and right.
  - ultrasaurus 16y ago
    So much of it comes down to managers not being able to tell good code from bad. If you look at quality as a crapshoot (it's genuinely hard to tell architecture astronauts from good code) then indeterminate quality quickly beats indeterminate quality slowly.
raganwald 16y ago
that doesn't take into account maintenance This excellent insight takes us to a place where productivity is even harder to measure: How does one measure the value of a piece of software? Software that has subtle bugs flying under the rader of our test suite has some kind of negative value. Software that lowers the "productivity" (however we measure it) of future programmers who need to extend or change it has negative value associated with it. How do we measure that? Imagine the exact same formally specified requirements handed to four different programmers: The first programmer, "Ned," does the job in a straightforward fashion, and delivers working code passing all tests. "Fred" does the job using uncommon techniques (parser combinators, for example) in less time and produces less code. "Ed" proposes that if some of the requirements are relaxed, there is an open source solution that can do the job with trivial integration. The last programmer, "Jed," sees some commonality with some existing code and proposes that the scope be expanded to produce a new piece of infrastructure (message queues, web services, SOAP, &c) solving a more general problem. How do we judge Ned, Ed, Fred, and Jed?
- palish 16y ago
  By how fun they are. As people. I'm convinced the only real-world metric that matters is "how much other people want to work with you". Being an easygoing, fun person facilitates that dramatically.
  - endtime 16y ago
    All (but not only) people with good people skills and zero productivity would be motivated to say this. Edit: I think I might have misunderstood what palish meant by "By how fun they are." to exclude productivity and focus entirely on personality. My mistake, if so.
    - alex_c 16y ago
      That might be true in the very short term, but in the medium- or long-term, no one actually enjoys working with "fun" people who have zero or negative productivity.
HedgeMage 16y ago
I disagree. It's faster to write garbage than it is to write good, clean code.
- techdmn 16y ago
  I'm not sure I agree with this. Granted good design takes some extra work. But for the most part, in my experience, coders who write bad code also do it slowly (perhaps because they don't understand it), while those who write cleaning code are able to move more quickly. As a project grows, you have to refer back to your own work more often. The price for poor code starts to be paid almost as soon as the cursor leaves the page.
T-hawk 16y ago
> And that doesn't take into account maintenance. Which is the elephant in the development room. IMO, maintainability is exactly what good programmers should be judged on. Any programming project of any meaningful size and usefulness will spend at least an order of magnitude more time in maintenance and enhancements than in initial development. For the largest projects (Google, Facebook), maintainability becomes 1000 or more times more important than the original development time. Problem is, you can't quantify maintainability. What a good programmer contributes to maintainability and enhanceability is the absence of certain problems or classes of problems. Everything from choosing an exotic languange that nobody else can maintain, to the choice and organization of source control. We take that sort of thing for granted in HN type projects, but problems like "nobody knows how to compile that" are a really big problem at corporate enterprise scales, especially companies that are not fundamentally technologists, say finance or medical or shipping. An individual programmer is more likely to incur reward if he looks like he's playing the hero in a bad platform and ecosystem ("only this guy has the wizardry to manage that!") than if he built a good understandable platform in the first place.
- chris_j 16y ago
  Very true. A friend of mine, who worked for a very large company at the time, once described the perfect system for achieving recognition as a developer. Firstly, write code that's full of bugs and is hard to maintain. Secondly, once the code is shipped and starts going wrong, make sure you step up to the plate and fix all the bugs that you created. That way, you get kudos for delivering code rapidly and for all the customer support work you do. Contrast that with the guy who writes good quality code who doesn't ship as rapidly and who doesn't have half as much customer contact.
  - joshuacc 16y ago
    I don't have the citation on hand, but this goes along with a study I read about customer loyalty. Customers who have a problem which is promptly corrected by the company are more loyal than customers that never have any problems.

sudhirc 16y ago

I agree but then why salaries of top mgmt. is not decided using the same criteria.

stcredzero 16y ago

A friend of mine used this successfully with groups practicing Test Driven Development: - First, institute "Test First" development - Randomly pick some fraction of new tests to be reviewed (Demand a minimum quality level.) - Measure productivity in terms of new passing tests completed with some multiplier for the quality score This system could be gamed, but it would require a conspiracy consisting of a large fraction of the team, and any system could be gamed in that situation. This is the only method I know of that works when analyzed logically and has been shown to work in practice.

michaelbuckbee 16y ago
I'd be concerned that the approach you laid out is something of a proxy for lines of code delivered. My most productive days are the ones where I've removed huge blocks of unnecessary code.
- rhizome 16y ago
  KLOC mentality applied to TDD.
  - stcredzero 16y ago
    That would be correct, except that the tests are peer reviewed in my proposal. You can't "code me a new minivan" in this situation, unless the test review process gets corrupted.
- stcredzero 16y ago
  I'd be concerned that the approach you laid out is something of a proxy for lines of code delivered. Absolutely incorrect. Did you actually read my proposal? If the tests are of high quality, then the code passing them will be substantive. Also, in new development, what matters is functional specs delivered, and in a properly run TDD project, these two are strongly related.
  - michaelbuckbee 16y ago

deleted 16y ago

[deleted]

dwc 16y ago

Delivering a quality product within time and budget constraints, consistently. Oh, but that's a team effort, you say? Well, that's the working environment, isn't it? Projects are an awfully coarse measurement, you say? Well, that's the only deliverable that matters. I recall reading years ago (in Peopleware or one of its contemporaries) about a company's evaluation of one of its coders. She was definitely mediocre by every measure they had. But someone noticed that every project she was on succeeded, over many projects and many years. Though she wasn't a monster at the keyboard, something she brought to the team engendered success. How productive was she? Would you want to hire her and have her on your team? You must measure what you actually care about. Measuring things that you think are factors is fine and noble, but if you're not measuring the actual "product" of "productivity" then you'll never know how well your factors correlate with the real goal.

raganwald 16y ago
But someone noticed that every project she was on succeeded, over many projects and many years. Though she wasn't a monster at the keyboard, something she brought to the team engendered success. Correlation does not equal causation. Another possible explanation: She was a monster at predicting project success and worked her way onto projects that were going to succeed with or without her.
- dwc 16y ago
  Sure, in general. In this case managers were fairly familiar with her and the teams and posited that the success was due to what she added to team discussions. I think they were probably correct. OTOH, having a project success divining rod could have its own value.
- henrybaxter 16y ago
  So ask her if she's willing to be on this project! Seriously though, ask your developer why there is a correlation between them and project success - if they can tell you, even if it shows they should be a business analyst, you're on to something very useful and valuable.
- jerf 16y ago

Periodic 16y ago

I worked at an enterprise software company with a Scrum system that used two-week sprints. This means we would estimate all the tasks for the next two weeks in a big meeting at the beginning of the sprint, then at the end of the sprint we'd have another big meeting to close out the tasks and do any analysis on met/missed targets. Tasks tended to get separated so there wasn't too much direct collaboration between coders within a sprint, and the estimations were mostly a group effort, but the person who was getting assigned a task had final say to tweak the numbers. In this environment, it was fairly easy to see who was the more productive programmers. Some people got their tasks done quickly and could easily take parts of other features, and other people were usually late and needed others to help them finish their features. If you were a programmer there, you could easily rank the programmers based on productivity. I bet QA could do the same if they tried. However, management focused on it being a team effort. As long as we finished everything up by the end of the week they did their best to reward everyone and fire no one.

jarin 16y ago

I'm of the opinion that programming is more an art than a science, so to me asking the objective metric of programmer productivity is like asking the objective metric of painter productivity. In my experience, if you're technical you just kind of know if a programmer is good or not. If you're not technical, you find a technical person you trust and ask them.