4 ms·

So the thing is that volume of output is still important even in creative fields; if you run a logo design business and A makes the logos that satisfies 10 comp

So the thing is that volume of output is still important even in creative fields; if you run a logo design business and A makes the logos that satisfies 10 companies and B makes the logos for 1 company that is a pretty direct business outcome. There's counter-examples of the one logo was your whale customer that needs perfection versus 10 small businesses where you can just put the company name in a few fonts, but the raw volume of output is extremely relevant aspect of employee performance. With coding in practice the high performers who deliver business value are often (but not always) the same people writing a lot of code; low performers might only make a couple PRs per month while a high performer did a couple per day. The problem is that is more of a proxy outcome: it's also easily possible for a low performer to shovel out several negative-value PRs per day and for someone who does only a couple small PRs per month to be doing something difficult that delivers enormous value (especially if those few changes are like updating an ML model or finding key performance optimization opportunities).

nine_zeros 3y ago
> if you run a logo design business and A makes the logos that satisfies 10 companies and B makes the logos for 1 company that is a pretty direct business outcome There's a flaw with equating this to # of commits or lines of code. That flaw is - the amount of revenue increases with the number of logos produced but the amount of revenue does NOT increase with the # of commits or number of lines produced.
- evilduck 3y ago
  There's the clear connection to revenue potential between zero and one commits. I don't think commits and LoC are linear to someone's value to the codebase or org, but IME there's strong correlation between consistently low commits and lines of code output and ineffectiveness. Hard problems and low output happen situationally, generally not for quarters and years on end.
  - nine_zeros 3y ago
    > There's the clear connection to revenue potential between zero and one commits. But there is no correlation to revenue potential between 99 and 100 commits. And most companies are not stuck at the first commit. > Hard problems and low output happen situationally, generally not for quarters and years on end. Absolutely untrue. The hardest problems I have solved required me to build observability over 2 quarters, write RFCs and get buy-in from other engineers. None of these contribute to commits.
    - evilduck 3y ago
      > The hardest problems I have solved required me to build observability And you were able to do this without writing a line of code and committing it?
      - nine_zeros 3y ago
        > The hardest problems I have solved required me to build observability Yes. I personally did not commit a single line because I had to influence other engineers to do apply said commits in their own repositories. I had 0 commits for 6 months.
- twh270 3y ago
  Hence this being a proxy measurement. But managers really want a way to measure "developer productivity", and the feeling is that proxy measurements are better than nothing, even though they aren't. (Two simple examples: (1) how do you measure the productivity of a lead developer who spends much of his time mentoring, pairing, and helping the team deliver working code? (2) Is a developer who commits many small PR's "worth more" to the company than a developer who commits fewer, larger PR's?)
- esrauch 3y ago
  I intended the logo example because it has the same property: someone who makes 100 terrible logos is still pretty useless compared to someone who makes 10 good ones. But very often you have two people that are outputting 100 vs 10 and the quality is about the same. The reality is that when I look at who was strong and who wasn't there's a pretty strong correlation between people writing more code and also their business impact being larger. Many smaller tasks do go "X lines of new code are needed to fix a problem", and the person who delivered on 10 tasks did write 10x lines compared to the person who did one task, with all tasks size, difficulty and business value held constant. There's obviously tons of noise in this including incredibly far outliers of the guy who writes zero lines but whose knowledge is a lynchpin, and the guy who writes 5k lines of negative value garbage code. And when it becomes a known measure it surely does create very perverse incentives, etc, but companies end up using it as a proxy metric because there is literally no other objective measure of code quality or business impact for engineers available: the sole two things to measure SWEs on is lines of code and feelings.
  - nine_zeros 3y ago
    > but companies end up using it as a proxy metric because there is literally no other objective measure of code quality or business impact for engineers available: the sole two things to measure SWEs on is lines of code and feelings. And you have hit the nail on the head. The problem is that businesses have to rely on proxies because there is NO metric to quantify this creative work. The best thing for businesses is to first define what the right outcomes are: they certainly aren't lines of code, stack ranking, number of commits etc. But it could be features, bug fixes, investigation of systems etc - you know, actual work engineers do. Once the business identifies this, all a business needs to do is hire skilled managers who understand that proxies are not outcomes. The proxies, rules built on those proxies, and firing people on these proxies is literally the stupidest thing to do. Like, it is literally costing actual $$ on the business that has too many managers who are using simplist metrics. This cost is in terms of time. Cost = (number of days to hire + number of days to onboard + number of days wasted on BS metrics + number of days management spent on measuring and stack ranking on BS metrics + number of days to fire + number of days to backfill the role + number of days to get them back up to the skill up to the last engineer's level) Find these skilled managers who understand this cost is not worth any BS metrics, hire these managers. That's it.

nine_zeros 3y ago

> if you run a logo design business and A makes the logos that satisfies 10 companies and B makes the logos for 1 company that is a pretty direct business outcome There's a flaw with equating this to # of commits or lines of code. That flaw is - the amount of revenue increases with the number of logos produced but the amount of revenue does NOT increase with the # of commits or number of lines produced.

evilduck 3y ago
There's the clear connection to revenue potential between zero and one commits. I don't think commits and LoC are linear to someone's value to the codebase or org, but IME there's strong correlation between consistently low commits and lines of code output and ineffectiveness. Hard problems and low output happen situationally, generally not for quarters and years on end.
- nine_zeros 3y ago
  > There's the clear connection to revenue potential between zero and one commits. But there is no correlation to revenue potential between 99 and 100 commits. And most companies are not stuck at the first commit. > Hard problems and low output happen situationally, generally not for quarters and years on end. Absolutely untrue. The hardest problems I have solved required me to build observability over 2 quarters, write RFCs and get buy-in from other engineers. None of these contribute to commits.
  - evilduck 3y ago
    > The hardest problems I have solved required me to build observability And you were able to do this without writing a line of code and committing it?
    - nine_zeros 3y ago
      > The hardest problems I have solved required me to build observability Yes. I personally did not commit a single line because I had to influence other engineers to do apply said commits in their own repositories. I had 0 commits for 6 months.
twh270 3y ago
Hence this being a proxy measurement. But managers really want a way to measure "developer productivity", and the feeling is that proxy measurements are better than nothing, even though they aren't. (Two simple examples: (1) how do you measure the productivity of a lead developer who spends much of his time mentoring, pairing, and helping the team deliver working code? (2) Is a developer who commits many small PR's "worth more" to the company than a developer who commits fewer, larger PR's?)
esrauch 3y ago
I intended the logo example because it has the same property: someone who makes 100 terrible logos is still pretty useless compared to someone who makes 10 good ones. But very often you have two people that are outputting 100 vs 10 and the quality is about the same. The reality is that when I look at who was strong and who wasn't there's a pretty strong correlation between people writing more code and also their business impact being larger. Many smaller tasks do go "X lines of new code are needed to fix a problem", and the person who delivered on 10 tasks did write 10x lines compared to the person who did one task, with all tasks size, difficulty and business value held constant. There's obviously tons of noise in this including incredibly far outliers of the guy who writes zero lines but whose knowledge is a lynchpin, and the guy who writes 5k lines of negative value garbage code. And when it becomes a known measure it surely does create very perverse incentives, etc, but companies end up using it as a proxy metric because there is literally no other objective measure of code quality or business impact for engineers available: the sole two things to measure SWEs on is lines of code and feelings.
- nine_zeros 3y ago
  > but companies end up using it as a proxy metric because there is literally no other objective measure of code quality or business impact for engineers available: the sole two things to measure SWEs on is lines of code and feelings. And you have hit the nail on the head. The problem is that businesses have to rely on proxies because there is NO metric to quantify this creative work. The best thing for businesses is to first define what the right outcomes are: they certainly aren't lines of code, stack ranking, number of commits etc. But it could be features, bug fixes, investigation of systems etc - you know, actual work engineers do. Once the business identifies this, all a business needs to do is hire skilled managers who understand that proxies are not outcomes. The proxies, rules built on those proxies, and firing people on these proxies is literally the stupidest thing to do. Like, it is literally costing actual $$ on the business that has too many managers who are using simplist metrics. This cost is in terms of time. Cost = (number of days to hire + number of days to onboard + number of days wasted on BS metrics + number of days management spent on measuring and stack ranking on BS metrics + number of days to fire + number of days to backfill the role + number of days to get them back up to the skill up to the last engineer's level) Find these skilled managers who understand this cost is not worth any BS metrics, hire these managers. That's it.