4 ms·

KPIs for Software Engineers

21 points by Sevii 3y ago

bicijay 3y ago
> lines of code Yep, my manager will be implementing this post...
angarg12 3y ago
Aren't most of these actually anti-patterns? The best example is lines of code written. If we measure individual performance on that, it's trivial for people to game the system. Also, number of lines of code doesn't correlate to business outcomes. I much prefer to solve the same business problem with half the lines of code if possible! If I had to choose KPIs to measure the performance of teams (yes, teams), I would choose the DORA metrics, or something of the sort.
- 0wis 3y ago
  I think the author wanted to improve on the more flawed idea of counting the number of hours worked. DORA metrics seems to be an even better improvement on the author’s propositions. Thanks for sharing, I did not knew about it and it is interesting. If I can share my experience, i know that at the root, any measure is flawed and incomplete. However it is always better to measure than not to measure if something is important to you. I know time spent is not a good measure, but I also know I will make more progress at a task if I allocate and track time spent on it. And engineering output is knowingly hard to measure.
- cratermoon 3y ago
  https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
codetrotter 3y ago
> lines of code Someone should read this old story: https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...
mystickphoenix 3y ago
Always good to remember "what gets measured gets managed". I.e., every single one of these metrics can easily be gamed in a way that would be hard for a non-technical manager or "bean counter" to determine. Ex: oh, you're measuring number of commits, well, I'll stop squashing and start breaking commits down to single lines. Boom, I'm a 100x software engineer. Does anyone have a decent list of individual contributor KPI's that are actually useful or insightful? I've, so far (~20 years of experience), never run into individual KPI's that weren't able to be gamed and were actually meaningful.
- dboreham 3y ago
  The whole concept of KPI is bogus and worthless.
  - aleph_minus_one 3y ago
    I somewhat disagree; I rather like Goodhart's law: "When a measure becomes a target, it ceases to be a good measure". > https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law In this sense I do consider many KPIs often to be good metrics to find out whether the company is on the right path, or as an early warning system for problems that the company might become confronted with in the future. But as soon as you use KPIs to assess managers, they will become gamed, and thus loose a lot of their value.
  - parpfish 3y ago
    i think that KPIs often make sense at an organization level of a business, such as "We want to boost revenue from this product line by X%". but when you try to translate that to the individual level, it's almost impossible because it's hard/impossible to properly attribute value to individual actions.
    - hdjjhhvvhga 3y ago
      I think this is the key distinction. After two decades in the industry I can easily game any variable. Ant things change constantly. A person who seem unproductive today may surprise you tomorrow. You may consider yourself an excellent problem solver and then bump into something that is very difficult for you - but turns out easy for someone else. And so on and so forth.
mynameisjody 3y ago
Good lord, please take this down. These are all horrendous measurements that will lead to nothing but trouble.
chasing 3y ago
Dear god. > We want to avoid subjective judgements of how much of task X engineer 1 did vs engineer 2. None of these metrics remove subjective judgments. > meetings attended Beginning to think this is some sort of joke.
arnaudsm 3y ago
This is terrible advice. I've had teammates with impressive velocity and loc/day, but code quality so low they destroyed the codebase in weeks. "When a measure becomes a target, it ceases to be a good measure" https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law
- alexk307 3y ago
  Same. Most technical debt is caused by folks with “impressive velocity” in my experience. If any prospective employer tells you that they are measuring your lines of code written, run. They’re too lazy and cost driven to have understand.
Smeevy 3y ago
I don't see a single item in that dystopian list that can't be gamed by the individual being measured. How much is there to measure if everyone quits after 3 months?
wg0 3y ago
Counting lines of code from a developer is like doing a word count on a poet's work. Every single KPI listed above can be manipulated, means nothing, shows nothing except many will try to game the system by: - Needlessly adding lines of code. - Committing excessively. - Closing too many PRs too early or opening too many PRs too soon. - Meeting attended, people will needlessly attend meetings or create meetings. - Prod deployments, people will needlessly push to prod. And like that for all the listed.
handwarmers 3y ago
This list is the control freak/micromanager's wet dream.
galoisscobi 3y ago
Hard to take productivity KPI posts seriously when there’s no discussion of Goodhart’s law.
for_i_in_range 3y ago
Net Profit After Taxes = most underrated KPI in tech.
- zeroonetwothree 3y ago
  “Individually attributable”
parpfish 3y ago
letting ICs set their own KPIs feels like a trap. your manager likely has some rubric for how they'll measure your success. it may not even be explicitly formulated in their mind, but they have some sort of subjective/qualitative set of expectations. instead of going through the work of precisely defining those expectations and communicating them to the employee, they "empower" the employee to "set their own definition of success". however, this turns into a game of the IC guessing what the manager wants to hear, which results in two problems: - if you guess correctly, the manager is happy because they didn't have to go through the work of defining and communicating their expectations and they get to pretend that they are generous and letting employees "drive your own development" - if you guess incorrectly, you better hope that you accidentally meet the secret rubric your manager has in their head because you know that that's what is used to evaluate performance and determine raises/promotions regardless of what's written down in your quarterly KPIs
LaserToy 3y ago
Having that many KPI makes them useless. Also, lines of code written? One day, put all of them in PCA or tree model, and compare to your own observations on who is the most valuable eng in the org
szajbus 3y ago
All good, but without being actionable they're just vanity metrics that only get discussed during performance reviews. They don't provide any meaningful insights on how to improve actual performance in day-to-day work.
pdntspa 3y ago
Oh god no. None of these are good metrics. All are gameable. If my workplace enacted this then I would quit. And KPIs are the truncheon that managers use to smash others' heads in. There should be no metrics, for anyone, only RESULTS. "What gets measured gets managed"... why do they not teach the management class this
- jochem9 3y ago
  They do. They also teach you about leading and lagging KPIs. Results are lagging KPIs: nice to know if you are on track to achieving your goals, but useless if you want to know how to go faster/do better/etc.
Aperocky 3y ago
Does this article needs a sarcasm tag at the end? Or is this really what you mean?
therealdkz 3y ago
This reads like a chapter from some MBA software engineering book.
cimi_ 3y ago
I'm curious, who upvotes this and why? If you've upvoted and see this comment, can you please reply? FWIW, I agree with the rest of the comments here, I think the points on the list can easily be gamed and should not be used/trusted.
- mkeeter 3y ago
  I have the same question; this article feels like it was written by ChatGPT, and it's surprising to see something like that on the front page!
- parpfish 3y ago
  i'd upvote just because i want to see the discussion and commiserate with other HNers about terrible KPI experiences :)
pdimitar 3y ago
Fluff article that's mostly rehashing things that people kept trying to implement as metrics in software teams ever since the 1970s -- and failed every single time, as evidenced by the fact that people are still searching for reliable ways to gauge programmers. - Counting stuff does not work. People will stop doing atomic commits and will start pushing 1-3 lines of code or config all the time. Or will start breaking the issues into much smaller ones so as to seem like they are closing many of them. - Verifying means trying to engage more of a resource that's already precious and stretched thin i.e. engineer attention and time. You will end up squeezing your already strained people even more unless you hire extra. - Valuable, okay, but short-term or long-term? I met plenty of "valuable driven engineers with a can-do attitude" whose code I had to fix months later because the product development has ground to a halt. Who's more valuable, the person who makes a customer demo possible that pays $1M one-off investment after the demo, or the person who makes sure the customers with really deep pockets will get their features and start paying $500k a year? (Both are valuable and this was obviously a trick / trap question. Point is, you can't only worship the former group and claim the latter is less valuable.) - Individually attributable is a solved problem. Just count commits + coding lines and judge by that, if that tickles your fancy. Everything else does require the attention of other engineers which is not an efficient expenditure of time and energy. --- I'll give you one good KPI: make your programmer happy. Recognize that not all work is glorious because they do recognize that as well BUT periodically give them space and time to go off on small adventures to f.ex. optimize a hot path that's proven to be slower than what's deemed productive, or allow them to automate a process, or write a small library that can be reused between teams, or help engage with others who use their code and are unsure how to do so; things like that. Programming is a very mentally straining job. Help these people not hate their job. There are many well-documented ways to keep employees happy. Trying to measure everyone will never work until it gets fully automated, at which point it would be unproductive and illogical to do so -- because if you have such a smart automation then why don't you use it to get your actual work done with it, and not measure your inefficient flesh bag employees? In a nutshell: by the time you get to being able to gauge programmers in a fully automated matter, you would likely just "hire" a bunch of robots and fire the humans.
conwy 3y ago
I'm getting "Error establishing a database connection" after following this link. Maybe the developer who wrote the blog post can set a personal KPI of "99% uptime for blog site".
amriksohata 3y ago
Any KPI other than adding value to the business is not worth considering
majewsky 3y ago
> Error establishing a database connection Should've focused on KPIs for your site availability instead.
sprior 3y ago
Can I be too smug when I follow the story link and all I get is "Error establishing a database connection"?
alejandra265 3y ago
[dead]
alejandra265 3y ago
[dead]
workfromspace 3y ago
https://web.archive.org/web/20230716144740/https://www.sledgeworx.io/kpis-for-software-engineers/ https://web.archive.org/web/20230716144740/https://www.sledg... (website seems to be down due to a database error)

21 points by Sevii 3y ago

> lines of code Yep, my manager will be implementing this post...

Aren't most of these actually anti-patterns? The best example is lines of code written. If we measure individual performance on that, it's trivial for people to game the system. Also, number of lines of code doesn't correlate to business outcomes. I much prefer to solve the same business problem with half the lines of code if possible! If I had to choose KPIs to measure the performance of teams (yes, teams), I would choose the DORA metrics, or something of the sort.

0wis 3y ago
I think the author wanted to improve on the more flawed idea of counting the number of hours worked. DORA metrics seems to be an even better improvement on the author’s propositions. Thanks for sharing, I did not knew about it and it is interesting. If I can share my experience, i know that at the root, any measure is flawed and incomplete. However it is always better to measure than not to measure if something is important to you. I know time spent is not a good measure, but I also know I will make more progress at a task if I allocate and track time spent on it. And engineering output is knowingly hard to measure.
cratermoon 3y ago
https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...

codetrotter 3y ago

> lines of code Someone should read this old story: https://www.folklore.org/StoryView.py?story=Negative_2000_Lines_Of_Code.txt https://www.folklore.org/StoryView.py?story=Negative_2000_Li...

mystickphoenix 3y ago

Always good to remember "what gets measured gets managed". I.e., every single one of these metrics can easily be gamed in a way that would be hard for a non-technical manager or "bean counter" to determine. Ex: oh, you're measuring number of commits, well, I'll stop squashing and start breaking commits down to single lines. Boom, I'm a 100x software engineer. Does anyone have a decent list of individual contributor KPI's that are actually useful or insightful? I've, so far (~20 years of experience), never run into individual KPI's that weren't able to be gamed and were actually meaningful.

dboreham 3y ago
The whole concept of KPI is bogus and worthless.
- aleph_minus_one 3y ago
  I somewhat disagree; I rather like Goodhart's law: "When a measure becomes a target, it ceases to be a good measure". > https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law In this sense I do consider many KPIs often to be good metrics to find out whether the company is on the right path, or as an early warning system for problems that the company might become confronted with in the future. But as soon as you use KPIs to assess managers, they will become gamed, and thus loose a lot of their value.
- parpfish 3y ago
  i think that KPIs often make sense at an organization level of a business, such as "We want to boost revenue from this product line by X%". but when you try to translate that to the individual level, it's almost impossible because it's hard/impossible to properly attribute value to individual actions.
  - hdjjhhvvhga 3y ago
    I think this is the key distinction. After two decades in the industry I can easily game any variable. Ant things change constantly. A person who seem unproductive today may surprise you tomorrow. You may consider yourself an excellent problem solver and then bump into something that is very difficult for you - but turns out easy for someone else. And so on and so forth.

mynameisjody 3y ago

Good lord, please take this down. These are all horrendous measurements that will lead to nothing but trouble.

chasing 3y ago

Dear god. > We want to avoid subjective judgements of how much of task X engineer 1 did vs engineer 2. None of these metrics remove subjective judgments. > meetings attended Beginning to think this is some sort of joke.

arnaudsm 3y ago

This is terrible advice. I've had teammates with impressive velocity and loc/day, but code quality so low they destroyed the codebase in weeks. "When a measure becomes a target, it ceases to be a good measure" https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Goodhart%27s_law

alexk307 3y ago
Same. Most technical debt is caused by folks with “impressive velocity” in my experience. If any prospective employer tells you that they are measuring your lines of code written, run. They’re too lazy and cost driven to have understand.

Smeevy 3y ago

I don't see a single item in that dystopian list that can't be gamed by the individual being measured. How much is there to measure if everyone quits after 3 months?

wg0 3y ago

Counting lines of code from a developer is like doing a word count on a poet's work. Every single KPI listed above can be manipulated, means nothing, shows nothing except many will try to game the system by: - Needlessly adding lines of code. - Committing excessively. - Closing too many PRs too early or opening too many PRs too soon. - Meeting attended, people will needlessly attend meetings or create meetings. - Prod deployments, people will needlessly push to prod. And like that for all the listed.

handwarmers 3y ago

This list is the control freak/micromanager's wet dream.

galoisscobi 3y ago

Hard to take productivity KPI posts seriously when there’s no discussion of Goodhart’s law.

for_i_in_range 3y ago

Net Profit After Taxes = most underrated KPI in tech.

zeroonetwothree 3y ago
“Individually attributable”

parpfish 3y ago

letting ICs set their own KPIs feels like a trap. your manager likely has some rubric for how they'll measure your success. it may not even be explicitly formulated in their mind, but they have some sort of subjective/qualitative set of expectations. instead of going through the work of precisely defining those expectations and communicating them to the employee, they "empower" the employee to "set their own definition of success". however, this turns into a game of the IC guessing what the manager wants to hear, which results in two problems: - if you guess correctly, the manager is happy because they didn't have to go through the work of defining and communicating their expectations and they get to pretend that they are generous and letting employees "drive your own development" - if you guess incorrectly, you better hope that you accidentally meet the secret rubric your manager has in their head because you know that that's what is used to evaluate performance and determine raises/promotions regardless of what's written down in your quarterly KPIs

LaserToy 3y ago

Having that many KPI makes them useless. Also, lines of code written? One day, put all of them in PCA or tree model, and compare to your own observations on who is the most valuable eng in the org

szajbus 3y ago

All good, but without being actionable they're just vanity metrics that only get discussed during performance reviews. They don't provide any meaningful insights on how to improve actual performance in day-to-day work.

pdntspa 3y ago

Oh god no. None of these are good metrics. All are gameable. If my workplace enacted this then I would quit. And KPIs are the truncheon that managers use to smash others' heads in. There should be no metrics, for anyone, only RESULTS. "What gets measured gets managed"... why do they not teach the management class this

jochem9 3y ago
They do. They also teach you about leading and lagging KPIs. Results are lagging KPIs: nice to know if you are on track to achieving your goals, but useless if you want to know how to go faster/do better/etc.

Aperocky 3y ago

Does this article needs a sarcasm tag at the end? Or is this really what you mean?

therealdkz 3y ago

This reads like a chapter from some MBA software engineering book.

cimi_ 3y ago

I'm curious, who upvotes this and why? If you've upvoted and see this comment, can you please reply? FWIW, I agree with the rest of the comments here, I think the points on the list can easily be gamed and should not be used/trusted.

mkeeter 3y ago
I have the same question; this article feels like it was written by ChatGPT, and it's surprising to see something like that on the front page!
parpfish 3y ago
i'd upvote just because i want to see the discussion and commiserate with other HNers about terrible KPI experiences :)

pdimitar 3y ago

Fluff article that's mostly rehashing things that people kept trying to implement as metrics in software teams ever since the 1970s -- and failed every single time, as evidenced by the fact that people are still searching for reliable ways to gauge programmers. - Counting stuff does not work. People will stop doing atomic commits and will start pushing 1-3 lines of code or config all the time. Or will start breaking the issues into much smaller ones so as to seem like they are closing many of them. - Verifying means trying to engage more of a resource that's already precious and stretched thin i.e. engineer attention and time. You will end up squeezing your already strained people even more unless you hire extra. - Valuable, okay, but short-term or long-term? I met plenty of "valuable driven engineers with a can-do attitude" whose code I had to fix months later because the product development has ground to a halt. Who's more valuable, the person who makes a customer demo possible that pays $1M one-off investment after the demo, or the person who makes sure the customers with really deep pockets will get their features and start paying $500k a year? (Both are valuable and this was obviously a trick / trap question. Point is, you can't only worship the former group and claim the latter is less valuable.) - Individually attributable is a solved problem. Just count commits + coding lines and judge by that, if that tickles your fancy. Everything else does require the attention of other engineers which is not an efficient expenditure of time and energy. --- I'll give you one good KPI: make your programmer happy. Recognize that not all work is glorious because they do recognize that as well BUT periodically give them space and time to go off on small adventures to f.ex. optimize a hot path that's proven to be slower than what's deemed productive, or allow them to automate a process, or write a small library that can be reused between teams, or help engage with others who use their code and are unsure how to do so; things like that. Programming is a very mentally straining job. Help these people not hate their job. There are many well-documented ways to keep employees happy. Trying to measure everyone will never work until it gets fully automated, at which point it would be unproductive and illogical to do so -- because if you have such a smart automation then why don't you use it to get your actual work done with it, and not measure your inefficient flesh bag employees? In a nutshell: by the time you get to being able to gauge programmers in a fully automated matter, you would likely just "hire" a bunch of robots and fire the humans.

conwy 3y ago

I'm getting "Error establishing a database connection" after following this link. Maybe the developer who wrote the blog post can set a personal KPI of "99% uptime for blog site".

amriksohata 3y ago

Any KPI other than adding value to the business is not worth considering

majewsky 3y ago

> Error establishing a database connection Should've focused on KPIs for your site availability instead.

sprior 3y ago

Can I be too smug when I follow the story link and all I get is "Error establishing a database connection"?

alejandra265 3y ago

[dead]

alejandra265 3y ago

[dead]

workfromspace 3y ago

https://web.archive.org/web/20230716144740/https://www.sledgeworx.io/kpis-for-software-engineers/ https://web.archive.org/web/20230716144740/https://www.sledg... (website seems to be down due to a database error)