13 ms·

The thing to note here is that this is a one-way correlation: top performers tend to produce lots of commits. That does not mean that people who produce a high

The thing to note here is that this is a one-way correlation: top performers tend to produce lots of commits. That does not mean that people who produce a high amount of commits are the top performers in your company. I've had to deal with plenty of colleagues who moved very fast, committed extremely often, and were praised by management for the amount of work they produced. But in reality their work was rushed, sloppy, riddled with issues and a nightmare to maintain. And it would inevitably fall upon us "lesser" performers to actually investigate and fix all those issues, making us look less productive in the process. In my opinion a better metric to quantify someone's performance is to count the amount of problems they solve vs. the amount of problems they create. I bet that many of the rockstar programmers out there would land on the negative side of that particular scale.

pydry 6y ago
Most problems are multi causal and if you have blame culture will degenerate into a litany of finger pointing.
- lucideer 6y ago
  I'm not sure what "blame culture" is, but any professional software team should ideally have some kind of "accountability culture". Whatever the fallout from people feeling "blamed" may be, attrition of all of your genuine programming talent due to tech-debt-machine peers being promoted ahead of them is not exactly an ideal outcome either. What's more, very often genuine potential in naturally talented new programmers can be stunted if they're rewarded for lazy faux "productivity". I've seen this: a programmer has a genuine interest and passion for quality, but loses it over time due to a focus on doing (different) work that gets them promoted. Code ownership (being required to "own" ones work along with the bugs & maintenance burden that come with it) is one of the most valuable ways programmers learn. This needs to be balanced, as one runs the risk of having a bus factor of 1, but it's still vital.
  - pydry 6y ago
    Blame culture is when bugs, downtime, etc. happen and people both look for somebody to blame and simultaneously look to exculpate their own behavior. It leads to CYA behavior, backstabbing and massive risk aversion. It can also lead to or be a result of a toxic work environment. In multi causal bugs it can lead to people downplaying causes which they had something to do with and exaggerating the effect that co-worker thry don't like had something to do with. This often leads to confused attribution and poor rectification of systemic issues - e.g. writing more unit tests even when more unit tests won't really help. "Accountability culture" sounds like it could be the same thing. Or not. I'm not really sure.
    - lucideer 6y ago
      This sounds like a bad faith culture in general: i.e. "paying a price for mistakes", rather than "owning postmortems". Treating a mistake as an opportunity for punishment (be it purely social or otherwise), rather than for learning, is more about how you treat staff & peers than how you deal with mistakes. > Accountability culture" sounds like it could be the same thing. Can I ask what specific part of my description of accountability culture sounded like it lined up with your description of blame culture?
      - jdlshore 6y ago
        > This sounds like a bad faith culture Yes, that’s what blame culture is: a dysfunctional organizational culture where, when mistakes are made, the organizational priority is to find and punish those responsible. The consequence is CYA and finger pointing.
- marcosdumay 6y ago
  There is "blame culture", where every problem turns into a witch hunt, and there is the opposite, call it "anti-blame culture" where no problem can be declared, because, well, it's never the only problem to solve, and thus can never be solved. Competitive entities nearly never fall victim of the anti-blame kind, but where there is little competition, they are just as common as their opposite.
deepGem 6y ago
This is somewhat analogous to programmers who type very fast but delete half of what they type. This is very subjective again, most of the deletes can be legit deletes. There is a difference in the mistakes that are deleted and deletes that arise because of a thorough process change. I digress, but to a routine observer the person typing fast appears like a pro, someone who is really good at their job. While this can be true of other professions like a carpenter being good at his craft will be extremely fast, am not sure the same holds true for the typing speed of programmers. If you are a trial and error type of programmer then of course speed matters a lot because it has a direct correlation on how fast you can iterate and increase your sample space of trials. The frequency of commits and PRs can be seen in the same light. So IMO it's hard to find a true universal measure to identify the top programmers.
- AnHonestComment 6y ago
  Carpenters are masters at being “good enough”. I think you overstate speed in carpentry, though: you can’t just backspace or amend a commit to fix your mistakes. Great carpenters are smooth and purposeful — they can do an entire wall without breaking rhythm with a mistake. That’s where the real speed comes from: ever tried to backspace a joint you built wrong? That can take days. I would say that much the same is true of software - it’s more important to be purposeful and accurate than fast, because re-doing work adds a lot of cost. If you turn in work 1.33 times as fast, but have to do it twice... you actually work 0.66 as fast.
ericb 6y ago
> praised by management for the amount of work they produced Management literally can't tell the difference, sometimes even if it is a former dev. There are many ways to ship code faster. -Don't test -Don't worry about sanitizing inputs--extra effort slows you down. -Optimize for writing, not maintaining -Take a tech-debt loan. Bolt your feature somewhere convenient it doesn't belong. -Put on blinders. Your n+1 query problem is Tomorrow's Problem -Avoid refactors -Choose the first architecture you think of, not the one that is simpler, more flexible or more useful -DRY code takes too long to write--you'd have to read more code to do that! That's bottom performer thinking! -Securing credentials is hard. Hardcode them! Remember, if you want to be a top performer and have breathless articles written about you, ship many commits, fast! Also, this is a virtuous circle--if any of these become "problems" you'll get to ship more commits!
- kevmo314 6y ago
  > Avoid refactors On the contrary, small refactors are a great way to boost commit count! Management once tried to boost productivity by rewarding commits. Our team basically started manually committing automatic refactorings. We won by a landslide, but I don't think the company did. I got a nice prize out of it though.
  - ericb 6y ago
    Hah! That's some top performer thinking! I tip my hat to your team!
  - deleted 6y ago
    [deleted]
- eeZah7Ux 6y ago
  > -Don't test In an environment that rewards commits, unit tests can be a source of many little commits.
  - ericb 6y ago
    Tests are fiddly to get working and could expose problems! Maybe in a new commit and PR later?
ben7799 6y ago
I've seen this over and over too. Put a Non-technical VP/Manager in charge. Maybe someone who started out as a UI designer or non technical PM and then somehow became VP of engineering because they were an early loyal employee. Charismatic coders will take advantage and the manager will eat it up. The "rock stars" commit tons of code put create massive tech debt. They write their own buggy versions of stuff they should have just imported industry standard OSS versions of. Rule evaluation engines, broken file formats, broken home grown encryption, broken distributed job systems, heavily broken home grown DB frameworks. They'll have a huge bias towards terse code that doesn't make a lot of sense. Abuse of functional code. Everything is the opposite of KISS. Everyone else on the team is just constantly fixing their showstopper level bugs, of which there are always many. They talk a good game and management constantly thinks the rock star is super smart and the rest of the team is deficient. Then the tech debt reaches ridiculous levels, or you get bought and different management comes in and sees right through it. Managers get let go, new managers don't buy the rock star story. Rock star gets frustrated, leaves for another company where they can pull the same trick, and puts all this inflated stuff on the resume that they wrote all this stuff they shouldn't have written. A new naive manager falls for it think they must be really smart cause they're never going to find out how broken all the stuff was. All this is WAY easier for a rock start to pull off post around 2010 because the office culture in tech has become so PC that no one can be blamed for anything. Lots of stuff that used to get someone shown the door in my early career would not even get called out today at all.
- ikiris 6y ago
  Upvoted and then retracted due to the PC bullshit at the end. You had great points, you shouldn't ruin it with your bitterness about having to be professional.
  - vnlmk 6y ago
    "bullshit", unfounded accusations of bitterness, using "professional" as a euphemism for letting pushy people do what they want.
hkt 6y ago
Maybe a proxy for good contributions is the lifespan of their contributions rather than frequency?
- sova 6y ago
  https://dl.acm.org/doi/pdf/10.1145/3386321 https://dl.acm.org/doi/pdf/10.1145/3386321 Check out pages 71:26 and 71:27 for "Codebase introduction and retention" between Clojure and Scala. I'd like to see more graphics like these to illustrate "lifespan of commits"
  - hkt 6y ago
    Those are great graphics. I really wouldn't mind my contributions being measured in this way - a sense I've never had a out commit metrics before.
- yellowbeard 6y ago
  I think this can also be problematic. Long lived code _must_ work, which is good. However, it can also live a long time for bad reasons. I have seen long lived code that only lives a long time because it is so complicated that no one can understand how it works, or written in a way that is very hard to change. So it lives a long time, because no one wants to take the time or effort to touch it. Finally, when it _must_ be changed to support a new feature, it might require a full rewrite.
emilecantin 6y ago
I've worked with this type of colleague. I remember a specific example where, fresh out of university, I was assigned to pair with one. We had to create a front-end validator for the password reset feature, for users stored in LDAP. I was thinking "Wow, getting the password policies from LDAP must be a pain, it'll take a while to implement that feature". Nope, just copied whatever settings were currently in LDAP into our app's config file. A couple months later, a pair of engineers on the team had to spend 4+ weeks developing an "installer" to properly install & configure our app, as it had grown too complex to install & configure by hand. Management couldn't really figure out why...
that_guy_iain 6y ago
> I've had to deal with plenty of colleagues who moved very fast, committed extremely often, and were praised by management for the amount of work they produced. But in reality their work was rushed, sloppy, riddled with issues and a nightmare to maintain. And it would inevitably fall upon us "lesser" performers to actually investigate and fix all those issues, making us look less productive in the process. This is one of the most common complaints I hear about fast devs, is the produce lots of bugs. But what I've seen is fast dev produces 5x more code than normal devs and produces 5x more bugs. Which means the ratio is the same but it feels different because they're producing so much more code. You then get the devs who say they have to spend lots of time investigating these bugs so look worse. But I've literally seen the fast dev go on to a bug that one of the other developers was spending 2 weeks investigating and find the issue in the data within 5 minutes. You then have the slow and careful devs who will write 5 functional tests, 1 unit test and add 10 seconds to the build (not much but it adds up when it's ever issue) and still have the same ratio of bugs as the guy who is doing 1 units tests. I think the realitity is, a good dev is someone who produces the most business value. And something lots of devs don't want to hear is that the tiny little tweaks they make to improve the code quality, adds very little business value. Where as a required feature that works 90% of the time adds lots of business value. I think a lot of the complaining about these rockstar devs is just jealously. Code quality is one of the smallest tech problems yet so many devs think it's super important. No, getting stuff delivered so other departments and customers can do things is. Being able to plan things out is super important. Having a good data model is super important.
- Tehchops 6y ago
  > a good dev is someone who produces the most business value this this this. More code might equal more bugs, but if it's a net gain in business value everything else is a secondary concern. It doesn't mean you can just ship crap all the time. If customers start complaining/seeing errors/bad performance, business value decreases and there are going to be some $discussions. If developers are rewarded for shoveling garbage into the pipeline all the time, there are deeper organizational issues afoot.
  - 6y ago
cratermoon 6y ago
Yeah this is a terrible measure of productivity. Counting commits and PRs strikes me as the same as counting lines of code, just with more steps. What's even worse is if this metric ever becomes a target, Goodhart's Law will apply, and then Campbell's law will make it as useless as LOC.
gfxgirl 6y ago
This used to drive me nuts and I feel like I wish the rule was, if your code breaks something you are not allowed to work on anything else until it's fixed and if possible it will be reverted. There needs to be some incentive to not let people shit all over the code based for everyone else to clean up. Reviews are enough. All code was required to be reviewed by owners and there was still lots of this.

pydry 6y ago

Most problems are multi causal and if you have blame culture will degenerate into a litany of finger pointing.

lucideer 6y ago
I'm not sure what "blame culture" is, but any professional software team should ideally have some kind of "accountability culture". Whatever the fallout from people feeling "blamed" may be, attrition of all of your genuine programming talent due to tech-debt-machine peers being promoted ahead of them is not exactly an ideal outcome either. What's more, very often genuine potential in naturally talented new programmers can be stunted if they're rewarded for lazy faux "productivity". I've seen this: a programmer has a genuine interest and passion for quality, but loses it over time due to a focus on doing (different) work that gets them promoted. Code ownership (being required to "own" ones work along with the bugs & maintenance burden that come with it) is one of the most valuable ways programmers learn. This needs to be balanced, as one runs the risk of having a bus factor of 1, but it's still vital.
- pydry 6y ago
  Blame culture is when bugs, downtime, etc. happen and people both look for somebody to blame and simultaneously look to exculpate their own behavior. It leads to CYA behavior, backstabbing and massive risk aversion. It can also lead to or be a result of a toxic work environment. In multi causal bugs it can lead to people downplaying causes which they had something to do with and exaggerating the effect that co-worker thry don't like had something to do with. This often leads to confused attribution and poor rectification of systemic issues - e.g. writing more unit tests even when more unit tests won't really help. "Accountability culture" sounds like it could be the same thing. Or not. I'm not really sure.
  - lucideer 6y ago
    This sounds like a bad faith culture in general: i.e. "paying a price for mistakes", rather than "owning postmortems". Treating a mistake as an opportunity for punishment (be it purely social or otherwise), rather than for learning, is more about how you treat staff & peers than how you deal with mistakes. > Accountability culture" sounds like it could be the same thing. Can I ask what specific part of my description of accountability culture sounded like it lined up with your description of blame culture?
    - jdlshore 6y ago
      > This sounds like a bad faith culture Yes, that’s what blame culture is: a dysfunctional organizational culture where, when mistakes are made, the organizational priority is to find and punish those responsible. The consequence is CYA and finger pointing.
marcosdumay 6y ago
There is "blame culture", where every problem turns into a witch hunt, and there is the opposite, call it "anti-blame culture" where no problem can be declared, because, well, it's never the only problem to solve, and thus can never be solved. Competitive entities nearly never fall victim of the anti-blame kind, but where there is little competition, they are just as common as their opposite.

deepGem 6y ago

This is somewhat analogous to programmers who type very fast but delete half of what they type. This is very subjective again, most of the deletes can be legit deletes. There is a difference in the mistakes that are deleted and deletes that arise because of a thorough process change. I digress, but to a routine observer the person typing fast appears like a pro, someone who is really good at their job. While this can be true of other professions like a carpenter being good at his craft will be extremely fast, am not sure the same holds true for the typing speed of programmers. If you are a trial and error type of programmer then of course speed matters a lot because it has a direct correlation on how fast you can iterate and increase your sample space of trials. The frequency of commits and PRs can be seen in the same light. So IMO it's hard to find a true universal measure to identify the top programmers.

AnHonestComment 6y ago
Carpenters are masters at being “good enough”. I think you overstate speed in carpentry, though: you can’t just backspace or amend a commit to fix your mistakes. Great carpenters are smooth and purposeful — they can do an entire wall without breaking rhythm with a mistake. That’s where the real speed comes from: ever tried to backspace a joint you built wrong? That can take days. I would say that much the same is true of software - it’s more important to be purposeful and accurate than fast, because re-doing work adds a lot of cost. If you turn in work 1.33 times as fast, but have to do it twice... you actually work 0.66 as fast.

ericb 6y ago

> praised by management for the amount of work they produced Management literally can't tell the difference, sometimes even if it is a former dev. There are many ways to ship code faster. -Don't test -Don't worry about sanitizing inputs--extra effort slows you down. -Optimize for writing, not maintaining -Take a tech-debt loan. Bolt your feature somewhere convenient it doesn't belong. -Put on blinders. Your n+1 query problem is Tomorrow's Problem -Avoid refactors -Choose the first architecture you think of, not the one that is simpler, more flexible or more useful -DRY code takes too long to write--you'd have to read more code to do that! That's bottom performer thinking! -Securing credentials is hard. Hardcode them! Remember, if you want to be a top performer and have breathless articles written about you, ship many commits, fast! Also, this is a virtuous circle--if any of these become "problems" you'll get to ship more commits!

kevmo314 6y ago
> Avoid refactors On the contrary, small refactors are a great way to boost commit count! Management once tried to boost productivity by rewarding commits. Our team basically started manually committing automatic refactorings. We won by a landslide, but I don't think the company did. I got a nice prize out of it though.
- ericb 6y ago
  Hah! That's some top performer thinking! I tip my hat to your team!
- deleted 6y ago
  [deleted]
eeZah7Ux 6y ago
> -Don't test In an environment that rewards commits, unit tests can be a source of many little commits.
- ericb 6y ago
  Tests are fiddly to get working and could expose problems! Maybe in a new commit and PR later?

ben7799 6y ago

I've seen this over and over too. Put a Non-technical VP/Manager in charge. Maybe someone who started out as a UI designer or non technical PM and then somehow became VP of engineering because they were an early loyal employee. Charismatic coders will take advantage and the manager will eat it up. The "rock stars" commit tons of code put create massive tech debt. They write their own buggy versions of stuff they should have just imported industry standard OSS versions of. Rule evaluation engines, broken file formats, broken home grown encryption, broken distributed job systems, heavily broken home grown DB frameworks. They'll have a huge bias towards terse code that doesn't make a lot of sense. Abuse of functional code. Everything is the opposite of KISS. Everyone else on the team is just constantly fixing their showstopper level bugs, of which there are always many. They talk a good game and management constantly thinks the rock star is super smart and the rest of the team is deficient. Then the tech debt reaches ridiculous levels, or you get bought and different management comes in and sees right through it. Managers get let go, new managers don't buy the rock star story. Rock star gets frustrated, leaves for another company where they can pull the same trick, and puts all this inflated stuff on the resume that they wrote all this stuff they shouldn't have written. A new naive manager falls for it think they must be really smart cause they're never going to find out how broken all the stuff was. All this is WAY easier for a rock start to pull off post around 2010 because the office culture in tech has become so PC that no one can be blamed for anything. Lots of stuff that used to get someone shown the door in my early career would not even get called out today at all.

ikiris 6y ago
Upvoted and then retracted due to the PC bullshit at the end. You had great points, you shouldn't ruin it with your bitterness about having to be professional.
- vnlmk 6y ago
  "bullshit", unfounded accusations of bitterness, using "professional" as a euphemism for letting pushy people do what they want.

hkt 6y ago

Maybe a proxy for good contributions is the lifespan of their contributions rather than frequency?

sova 6y ago
https://dl.acm.org/doi/pdf/10.1145/3386321 https://dl.acm.org/doi/pdf/10.1145/3386321 Check out pages 71:26 and 71:27 for "Codebase introduction and retention" between Clojure and Scala. I'd like to see more graphics like these to illustrate "lifespan of commits"
- hkt 6y ago
  Those are great graphics. I really wouldn't mind my contributions being measured in this way - a sense I've never had a out commit metrics before.
yellowbeard 6y ago
I think this can also be problematic. Long lived code _must_ work, which is good. However, it can also live a long time for bad reasons. I have seen long lived code that only lives a long time because it is so complicated that no one can understand how it works, or written in a way that is very hard to change. So it lives a long time, because no one wants to take the time or effort to touch it. Finally, when it _must_ be changed to support a new feature, it might require a full rewrite.

emilecantin 6y ago

I've worked with this type of colleague. I remember a specific example where, fresh out of university, I was assigned to pair with one. We had to create a front-end validator for the password reset feature, for users stored in LDAP. I was thinking "Wow, getting the password policies from LDAP must be a pain, it'll take a while to implement that feature". Nope, just copied whatever settings were currently in LDAP into our app's config file. A couple months later, a pair of engineers on the team had to spend 4+ weeks developing an "installer" to properly install & configure our app, as it had grown too complex to install & configure by hand. Management couldn't really figure out why...

that_guy_iain 6y ago

> I've had to deal with plenty of colleagues who moved very fast, committed extremely often, and were praised by management for the amount of work they produced. But in reality their work was rushed, sloppy, riddled with issues and a nightmare to maintain. And it would inevitably fall upon us "lesser" performers to actually investigate and fix all those issues, making us look less productive in the process. This is one of the most common complaints I hear about fast devs, is the produce lots of bugs. But what I've seen is fast dev produces 5x more code than normal devs and produces 5x more bugs. Which means the ratio is the same but it feels different because they're producing so much more code. You then get the devs who say they have to spend lots of time investigating these bugs so look worse. But I've literally seen the fast dev go on to a bug that one of the other developers was spending 2 weeks investigating and find the issue in the data within 5 minutes. You then have the slow and careful devs who will write 5 functional tests, 1 unit test and add 10 seconds to the build (not much but it adds up when it's ever issue) and still have the same ratio of bugs as the guy who is doing 1 units tests. I think the realitity is, a good dev is someone who produces the most business value. And something lots of devs don't want to hear is that the tiny little tweaks they make to improve the code quality, adds very little business value. Where as a required feature that works 90% of the time adds lots of business value. I think a lot of the complaining about these rockstar devs is just jealously. Code quality is one of the smallest tech problems yet so many devs think it's super important. No, getting stuff delivered so other departments and customers can do things is. Being able to plan things out is super important. Having a good data model is super important.

Tehchops 6y ago
> a good dev is someone who produces the most business value this this this. More code might equal more bugs, but if it's a net gain in business value everything else is a secondary concern. It doesn't mean you can just ship crap all the time. If customers start complaining/seeing errors/bad performance, business value decreases and there are going to be some $discussions. If developers are rewarded for shoveling garbage into the pipeline all the time, there are deeper organizational issues afoot.
- 6y ago

cratermoon 6y ago

Yeah this is a terrible measure of productivity. Counting commits and PRs strikes me as the same as counting lines of code, just with more steps. What's even worse is if this metric ever becomes a target, Goodhart's Law will apply, and then Campbell's law will make it as useless as LOC.

gfxgirl 6y ago

This used to drive me nuts and I feel like I wish the rule was, if your code breaks something you are not allowed to work on anything else until it's fixed and if possible it will be reverted. There needs to be some incentive to not let people shit all over the code based for everyone else to clean up. Reviews are enough. All code was required to be reviewed by owners and there was still lots of this.