7 ms·

Funny. I read a book a long time ago about developer productivity. They started with saying: "measuring productivity witk KLoCs is terribad". And later on: but

Funny. I read a book a long time ago about developer productivity. They started with saying: "measuring productivity witk KLoCs is terribad". And later on: but we only have that, so let's use it anyway. "Stopped reading there". And here in 2022 it's exactly The Same Thing: > Measuring developer outputs can be detrimental And then: > Design and coding: The number of design papers and specs, work items, pull requests, commits, and code reviews, as well as their volume or count. All they do is add more and more metrics. But this has the exact same problem as with the infamous KLoCs measure: how do you interpret it? How do you know it is not gamed, to begin with? Actually, now you have two problems: collecting and analyzing this mass of metrics can have a significant cost.

arthurcoudouy 5y ago
I'm personally not a huge fan of collecting quantitative data to evaluate engineering productivity. The context of such metrics is usually more important that the data itself, meaning using some discrepancies in your results to identify business needs or issues. When I work with quantitative data, I try to find pattern rather than analyzing the data iself (why do pull requests last longer on Monday afternoon? do we have too many meetings there?..)
- robocat 5y ago
  I suggest that in every comment, you add “author here” or similar to make that very clear, because it isn’t always very obvious from your comments.
  - arthurcoudouy 5y ago
    Thank you for the suggestion! Will do
astrobe_ 5y ago
One more thing: "put your money where your mouth is". Bug bounties work, why wouldn't "feature bounties" also work? You say you want those features, preferably bug-free, for this deadline. And there's $5K for the team if the objectives are met. Then, your metrics problem boils down to how to impartially measure customer satisfaction, or how well the objectives are met (in some contexts, bugs are unavoidable etc.). Metrics can still be important to help the team identify their problems (or rather, confirm that their intuitions about the problem). It's an optimization problem: measure first, then do something about the actual bottlenecks. That said, some programmers are such nerds that more money is not the highest motivation. One can use some creativity here.
- jbay808 5y ago
  It's very hard to align an explicit incentive scheme with the outcome that you actually want. In this case, you'll get your features, but they very likely won't be bug-free. They'll probably be quite slow and fragile. They might not scale. They might not be well thought-out. They might break backwards compatibility, or break other features that your customers are already using. In other words, why wouldn't a developer borrow limitless technical debt in order to claim the bounty as fast as possible and move on to their next bounty?
  - marcinzm 5y ago
    Especially since the developer will likely be at a new company or team in 1-2 years.
  - astrobe_ 5y ago
    Because I am talking about programmers, not about mercenary-developers.
    - bluefirebrand 5y ago
      Of course. No true programmer would be motivated by reward to produce shoddy work.
    - jbay808 5y ago
      Then what change in behaviour were you expecting your incentive scheme to result in?
      - astrobe_ 5y ago
        The "scheme" a) draws from gamification b) prevents the sometimes hostile reception of the use of metrics c) reinforces the feedback loop. Positive perception leads to positive behavior.
- tonyedgecombe 5y ago
  There is plenty of research that show extrinsic rewards don't really work that well, especially for intellectual endeavours. On top of that you have the problem that now your developers will follow the set objectives to the letter even if it transpires that something else was required.

arthurcoudouy 5y ago

I'm personally not a huge fan of collecting quantitative data to evaluate engineering productivity. The context of such metrics is usually more important that the data itself, meaning using some discrepancies in your results to identify business needs or issues. When I work with quantitative data, I try to find pattern rather than analyzing the data iself (why do pull requests last longer on Monday afternoon? do we have too many meetings there?..)

robocat 5y ago
I suggest that in every comment, you add “author here” or similar to make that very clear, because it isn’t always very obvious from your comments.
- arthurcoudouy 5y ago
  Thank you for the suggestion! Will do

astrobe_ 5y ago

One more thing: "put your money where your mouth is". Bug bounties work, why wouldn't "feature bounties" also work? You say you want those features, preferably bug-free, for this deadline. And there's $5K for the team if the objectives are met. Then, your metrics problem boils down to how to impartially measure customer satisfaction, or how well the objectives are met (in some contexts, bugs are unavoidable etc.). Metrics can still be important to help the team identify their problems (or rather, confirm that their intuitions about the problem). It's an optimization problem: measure first, then do something about the actual bottlenecks. That said, some programmers are such nerds that more money is not the highest motivation. One can use some creativity here.

jbay808 5y ago
It's very hard to align an explicit incentive scheme with the outcome that you actually want. In this case, you'll get your features, but they very likely won't be bug-free. They'll probably be quite slow and fragile. They might not scale. They might not be well thought-out. They might break backwards compatibility, or break other features that your customers are already using. In other words, why wouldn't a developer borrow limitless technical debt in order to claim the bounty as fast as possible and move on to their next bounty?
- marcinzm 5y ago
  Especially since the developer will likely be at a new company or team in 1-2 years.
- astrobe_ 5y ago
  Because I am talking about programmers, not about mercenary-developers.
  - bluefirebrand 5y ago
    Of course. No true programmer would be motivated by reward to produce shoddy work.
  - jbay808 5y ago
    Then what change in behaviour were you expecting your incentive scheme to result in?
    - astrobe_ 5y ago
      The "scheme" a) draws from gamification b) prevents the sometimes hostile reception of the use of metrics c) reinforces the feedback loop. Positive perception leads to positive behavior.
tonyedgecombe 5y ago
There is plenty of research that show extrinsic rewards don't really work that well, especially for intellectual endeavours. On top of that you have the problem that now your developers will follow the set objectives to the letter even if it transpires that something else was required.