13 ms·

"engineer productivity remained constant for both Android and iOS, whether measured by lines of code pushed or the number of pushes." Both of those metrics are

"engineer productivity remained constant for both Android and iOS, whether measured by lines of code pushed or the number of pushes." Both of those metrics are incredibly shit ways to measure productivity. I guess thats one explanation as to why the facebook app is 200mb+. They've been superproductive with all those lines of code. A better metric would rely on actual features or bugs, imo.

Osmose 9y ago
I think it's obvious that they're bad at a small scale (individually or for smallish teams, maybe < 30 engineers?), but I don't think they're _obviously_ bad for teams of 50+ engineers. Also, there is a difference between using them as metrics that you want to raise vs metrics you just don't want to drop or fluctuate wildly over time.
- nimchimpsky 9y ago
  I just think they are bad for anyone. Its not hard to measure actual useful things like features or bug. Why even bother measuring lines of code/number of commits.
  - dikaiosune 9y ago
    > Its not hard to measure actual useful things like features or bug. Presumably your consulting business has really taken off now that you have such unique insight into a historically unempirical area of our field?
  - richardwhiuk 9y ago
    Measuring bugs and features leads to other equally horrendous problematic behaviors.
- ktRolster 9y ago
  If you use "number of lines of code" as a metric for productivity, soon people will double-space their code. We went through that in the 90s.
  - khazhou 9y ago
    Were there any other signs you had shitty programmers on your team?
    - ktRolster 9y ago
      It's not about shitty or good, it's about management. Whatever you reward, that's what people will give you. See also: https://users.cs.duke.edu/~ola/courses/cps108/code/java/harpoon/images/dilbert/bug.gif https://users.cs.duke.edu/~ola/courses/cps108/code/java/harp...
      - Osmose 9y ago
        You're arguing against an extreme, though. Sure, if LOC added (or even just changed, which is the saner metric) was linked directly to bonuses, it'd be ripe for abuse. But that completely ignores the more reasonable use in the article, as one (of several) metrics used to measure the impact of their release process.
        deleted 9y ago
        [deleted]
        ktRolster 9y ago
        You should measure what you want more of. Unless for some reason you want more lines of code, then use a different metric.
        gardnr 9y ago
        They should measure and reward lines of code removed instead.
    - testvox 9y ago
      Measuring productivity in LOC...
- 0xbear 9y ago
  Some people at Google are actually quite proud of six digit _negative_ line counts they've contributed. And I think they have every right to be proud of that.
  - Swizec 9y ago
    That’s why you track changes made. Throughout my career imve found that number of changes comitted correlates pretty well with features delivered and bugs fixed. Also with feature and bug size. Yes there are edge cases when it takes a day of debugging to make a 1-line fix, but those are rare. Just like it’s very rare to deliver a useful new feature by changing a single line. Yes there are also features that are tracked as a real ticket and require a 1-line copy change and nothing else. Nobody thinks doing those makes you hella productive, it just needs to be done. As for padding lines and changes. That’s what code review is for.
    - 0xbear 9y ago
      That would be a bad metric for those "negative" folks as well. Their job is called "code cultivation", and they modify everything in Google's monorepo to make it suck less. Often this is done using automated tools, with a subsequent full run of affected tests and human code review. I've never seen anything like this anywhere else, and I probably won't for the rest of my career.
    - 0xbear 9y ago
      As to what one should track, one should track _results_ if you've managed to accomplish something useful with very little code, that is decidedly better than accomplishing the same with much more code and hundreds of check-ins. Simplicity is the ultimate sophistication.
      - Swizec 9y ago
        > one should track _results_ One should, but companies often care more about effort than results. They can manage based on effort, they can't manage based on results. If you spend 2 days getting the same results as somebody else does in 5 days, guess what, they don't want you milling around those extra 3 days and bringing morale down. Gotta give you more work!
khazhou 9y ago
Some people, perhaps you as well, take offense at LOC metrics because they're (in theory) easy to game, via redundant code, whitespace, verbose vs concise languages, etc. And because good developers know that small code and systems mean fewer opportunities for lurking bugs and points of failure. BUT, if a team of developers is not padding their code, nor has shitty programmers who write page after page of nonsense, then in the end the proportion of functionality to LOC should be fairly stable, and thus LOC is a useful and meaningful proxy metric for the amount of new or changed functionality in a codebase.
- nimchimpsky 9y ago
  I'm not offended I just think its shit. Its not difficult to track "velocity" via story points,features or bugs and is at least a step in the right direction. LOC going down is often adding the most value.
  - khazhou 9y ago
    Sure, deleting unused or bad code is a good thing, or refactoring or rewriting something more tightly. But you can't implement a new feature just by deleting code. I've never been in a codebase that did more and more by getting smaller. People also take offense because a management weenie can misread the metric and penalize people who delete code, or write concise code, or solved a super tricky critical bug after a month of debugging with a one-line change. But I still claim, these all normalize in the end. If a developer on my team has committed only 30 lines in three months, sorry but I'm gonna be skeptical of their actual productivity.
    - lomnakkus 9y ago
      > But you can't implement a new feature just by deleting code. No, but you can implement a "story", i.e. the removal of code is presumably a ticket somewhere in the system. (Or could become one.) > I've never been in a codebase that did more and more by getting smaller. Run on a smaller system? Run faster? (I'm not sure whether or not I agree with the parent's point, nor yours, just offering suggestions.)
lloydde 9y ago
I'd guess LoC changed is implicit in "lines of code pushed", which is a blunt measurement, but isn't the same as the traditional LOC metric and doesn't directly correlate to (rewarding) a large download.
ahupp 9y ago
> Both of those metrics are incredibly shit ways to measure productivity. LOC or commits are certainly bad ways of comparing two different people, teams, etc. But they are fine ways of measuring whether a specific event at a specific time changed productivity, assuming a decently large team etc.
- nebabyte 9y ago
  Not when they know that's what they're being measured on...
  - Cthulhu_ 9y ago
    That's not really relevant; if they knew they were being measured before and after the change, it would still show if the new system had an impact on productivity - even faked.
  - ahupp 9y ago
    Sure, nobody should setup incentives based on LOC. But in this case it's not about incentives, it just about seeing if something changed. Nobody writing code is going to be rewarded or penalized if this number changes.
mahyarm 9y ago
I think what they are trying to measure is how much is this build/release system effecting productivity of the engineers and changes their behavior. Not how productive the engineers are per say. If the build system is super slow and just breaks all the time, your going to see a drop in the number of commits landed per day. If git is completely down and commits per day drop per 0, then you have an outage ;)
elgenie 9y ago
Literally the next sentence: """Similarly, the number of critical issues arising from mobile releases is almost constant regardless of the number of deployments, indicating that our code quality does not suffer as we continue to scale.""" This is 'productivity' in the sense of the release eng team attempting to measure the volume of code changes they're pushing out and whether changes they made to the push/release process impacted other teams' ability to iterate. Unless you have some insight as to why adopting a more continuous deployment process would change the ratio between LOC pushed or code-changing-diffs pushed to features-or-bugs specifically at Facebook such that they would not function as acceptable proxies, questioning the metric chosen seems hollow. It's certainly not 'productivity' in the sense of whether people on product teams are considered to have done a good job or are deserving of bonuses / promotions.
coldtea 9y ago
>Both of those metrics are incredibly shit ways to measure productivity. In theory yes. In practice, when you know your team and their average output and quality, no. >I guess thats one explanation as to why the facebook app is 200mb+. They've been superproductive with all those lines of code Or you know, most of it is assets, debugging symbols, stray frameworks and libs not stripped, etc. As is the case with almost any app beyond a few MB.
hacker_9 9y ago
Better measure would be number of tests passing that prove business logic.
rodorgas 9y ago
Obligatory Bill Gates quote: Measuring programming progress by lines of code is like measuring aircraft building progress by weight.
- dogruck 9y ago
  I like this quote because it's so accurate. LOC does provide some information. You'd be unhappy if either LOC or weight were close to zero. That said, one of my favorite metrics is number of deleted LOC.
tehlike 9y ago
Better metric for a business must include revenue.

Osmose 9y ago

I think it's obvious that they're bad at a small scale (individually or for smallish teams, maybe < 30 engineers?), but I don't think they're _obviously_ bad for teams of 50+ engineers. Also, there is a difference between using them as metrics that you want to raise vs metrics you just don't want to drop or fluctuate wildly over time.

nimchimpsky 9y ago
I just think they are bad for anyone. Its not hard to measure actual useful things like features or bug. Why even bother measuring lines of code/number of commits.
- dikaiosune 9y ago
  > Its not hard to measure actual useful things like features or bug. Presumably your consulting business has really taken off now that you have such unique insight into a historically unempirical area of our field?
- richardwhiuk 9y ago
  Measuring bugs and features leads to other equally horrendous problematic behaviors.
ktRolster 9y ago
If you use "number of lines of code" as a metric for productivity, soon people will double-space their code. We went through that in the 90s.
- khazhou 9y ago
  Were there any other signs you had shitty programmers on your team?
  - ktRolster 9y ago
    It's not about shitty or good, it's about management. Whatever you reward, that's what people will give you. See also: https://users.cs.duke.edu/~ola/courses/cps108/code/java/harpoon/images/dilbert/bug.gif https://users.cs.duke.edu/~ola/courses/cps108/code/java/harp...
    - Osmose 9y ago
      You're arguing against an extreme, though. Sure, if LOC added (or even just changed, which is the saner metric) was linked directly to bonuses, it'd be ripe for abuse. But that completely ignores the more reasonable use in the article, as one (of several) metrics used to measure the impact of their release process.
      - deleted 9y ago
        [deleted]
      - ktRolster 9y ago
        You should measure what you want more of. Unless for some reason you want more lines of code, then use a different metric.
      - gardnr 9y ago
        They should measure and reward lines of code removed instead.
  - testvox 9y ago
    Measuring productivity in LOC...
0xbear 9y ago
Some people at Google are actually quite proud of six digit _negative_ line counts they've contributed. And I think they have every right to be proud of that.
- Swizec 9y ago
  That’s why you track changes made. Throughout my career imve found that number of changes comitted correlates pretty well with features delivered and bugs fixed. Also with feature and bug size. Yes there are edge cases when it takes a day of debugging to make a 1-line fix, but those are rare. Just like it’s very rare to deliver a useful new feature by changing a single line. Yes there are also features that are tracked as a real ticket and require a 1-line copy change and nothing else. Nobody thinks doing those makes you hella productive, it just needs to be done. As for padding lines and changes. That’s what code review is for.
  - 0xbear 9y ago
    That would be a bad metric for those "negative" folks as well. Their job is called "code cultivation", and they modify everything in Google's monorepo to make it suck less. Often this is done using automated tools, with a subsequent full run of affected tests and human code review. I've never seen anything like this anywhere else, and I probably won't for the rest of my career.
  - 0xbear 9y ago
    As to what one should track, one should track _results_ if you've managed to accomplish something useful with very little code, that is decidedly better than accomplishing the same with much more code and hundreds of check-ins. Simplicity is the ultimate sophistication.
    - Swizec 9y ago
      > one should track _results_ One should, but companies often care more about effort than results. They can manage based on effort, they can't manage based on results. If you spend 2 days getting the same results as somebody else does in 5 days, guess what, they don't want you milling around those extra 3 days and bringing morale down. Gotta give you more work!

khazhou 9y ago

Some people, perhaps you as well, take offense at LOC metrics because they're (in theory) easy to game, via redundant code, whitespace, verbose vs concise languages, etc. And because good developers know that small code and systems mean fewer opportunities for lurking bugs and points of failure. BUT, if a team of developers is not padding their code, nor has shitty programmers who write page after page of nonsense, then in the end the proportion of functionality to LOC should be fairly stable, and thus LOC is a useful and meaningful proxy metric for the amount of new or changed functionality in a codebase.

nimchimpsky 9y ago
I'm not offended I just think its shit. Its not difficult to track "velocity" via story points,features or bugs and is at least a step in the right direction. LOC going down is often adding the most value.
- khazhou 9y ago
  Sure, deleting unused or bad code is a good thing, or refactoring or rewriting something more tightly. But you can't implement a new feature just by deleting code. I've never been in a codebase that did more and more by getting smaller. People also take offense because a management weenie can misread the metric and penalize people who delete code, or write concise code, or solved a super tricky critical bug after a month of debugging with a one-line change. But I still claim, these all normalize in the end. If a developer on my team has committed only 30 lines in three months, sorry but I'm gonna be skeptical of their actual productivity.
  - lomnakkus 9y ago
    > But you can't implement a new feature just by deleting code. No, but you can implement a "story", i.e. the removal of code is presumably a ticket somewhere in the system. (Or could become one.) > I've never been in a codebase that did more and more by getting smaller. Run on a smaller system? Run faster? (I'm not sure whether or not I agree with the parent's point, nor yours, just offering suggestions.)

lloydde 9y ago

I'd guess LoC changed is implicit in "lines of code pushed", which is a blunt measurement, but isn't the same as the traditional LOC metric and doesn't directly correlate to (rewarding) a large download.

ahupp 9y ago

> Both of those metrics are incredibly shit ways to measure productivity. LOC or commits are certainly bad ways of comparing two different people, teams, etc. But they are fine ways of measuring whether a specific event at a specific time changed productivity, assuming a decently large team etc.

nebabyte 9y ago
Not when they know that's what they're being measured on...
- Cthulhu_ 9y ago
  That's not really relevant; if they knew they were being measured before and after the change, it would still show if the new system had an impact on productivity - even faked.
- ahupp 9y ago
  Sure, nobody should setup incentives based on LOC. But in this case it's not about incentives, it just about seeing if something changed. Nobody writing code is going to be rewarded or penalized if this number changes.

mahyarm 9y ago

I think what they are trying to measure is how much is this build/release system effecting productivity of the engineers and changes their behavior. Not how productive the engineers are per say. If the build system is super slow and just breaks all the time, your going to see a drop in the number of commits landed per day. If git is completely down and commits per day drop per 0, then you have an outage ;)

elgenie 9y ago

Literally the next sentence: """Similarly, the number of critical issues arising from mobile releases is almost constant regardless of the number of deployments, indicating that our code quality does not suffer as we continue to scale.""" This is 'productivity' in the sense of the release eng team attempting to measure the volume of code changes they're pushing out and whether changes they made to the push/release process impacted other teams' ability to iterate. Unless you have some insight as to why adopting a more continuous deployment process would change the ratio between LOC pushed or code-changing-diffs pushed to features-or-bugs specifically at Facebook such that they would not function as acceptable proxies, questioning the metric chosen seems hollow. It's certainly not 'productivity' in the sense of whether people on product teams are considered to have done a good job or are deserving of bonuses / promotions.

coldtea 9y ago

>Both of those metrics are incredibly shit ways to measure productivity. In theory yes. In practice, when you know your team and their average output and quality, no. >I guess thats one explanation as to why the facebook app is 200mb+. They've been superproductive with all those lines of code Or you know, most of it is assets, debugging symbols, stray frameworks and libs not stripped, etc. As is the case with almost any app beyond a few MB.

hacker_9 9y ago

Better measure would be number of tests passing that prove business logic.

rodorgas 9y ago

Obligatory Bill Gates quote: Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

dogruck 9y ago
I like this quote because it's so accurate. LOC does provide some information. You'd be unhappy if either LOC or weight were close to zero. That said, one of my favorite metrics is number of deleted LOC.

tehlike 9y ago

Better metric for a business must include revenue.