4 ms·

Lines of code written Lines of code rewritten (changed N days after) Lines of code removed Lines of code that contributed to linting problems Lines of code

Lines of code written Lines of code rewritten (changed N days after) Lines of code removed Lines of code that contributed to linting problems Lines of code that contributed to security issues (as reported by the static analyser) Average complexity measures CI/CD build failure rates Lines of code reviewed Gitlab has plugins to generate reports like this, and if you have enough data (multi year preferably) you can use this as a springboard for deeper analysis of members of the team, I find all these metrics together (non taken as absolutes because there’s many assumptions built into each one) can give a good overview of individuals in a team

BlargMcLarg 4y ago
Review + written encourages buddy-buddy dynamics. Build failure rates encourages frustration as long as you don't provide ways to run the exact tests in isolation locally, under the same rules as CI. Meanwhile, branch triggers and pre-push hooks trivialize this further. Most of these metrics are strange given they are trivial to check pre-merge. Meanwhile, running these statistics pre-merge discourages small commits and sharing code prior to having ideal builds. The moment this gets out is the moment devs will game your statistics and any benefit you gain goes out the door. And almost every one of these, even combined, is easy to game.
- malux85 4y ago
  Correct, we do not tell the devs these are being measured
  - BlargMcLarg 4y ago
    I cannot tell if you are aware you're setting devs up for failure or not. Half your mentioned metrics are tackled by hard forcing your devs to run branch CI prior to merging and giving them a slap on the wrist for not running CI checks locally pre-push. If they are important enough to measure, why not hard force them and focus on the remaining metrics instead?
    - malux85 4y ago
      See where I said “none taken as absolute because there are assumptions in each one” And “Use these as a springboard for deeper analysis of devs” These are only used as high level metrics to guide further analysis, we are keenly aware of the faults of these metrics - but the alternative of no-metrics or worse, “gut feel”, is slow and riddled with more problems. It doesn’t mean a dev is bad, maybe it means there’s some tooling failure, or organisational problem, or personal problem they need help with. These metrics give visibility into anomalies, and their imperfect nature doesn’t mean they are useless. There’s also like 20 more metrics but I’m on my phone and couldn’t recall them all right now Also I don’t know where all of this nonsense about “forcing” to run CI/CD before branch merges and “slap on the wrist” - all our devs can run the test suites locally in docker and inside their IDE, nobody is forced to do anything in the pipeline severs, and also nobody gets a slap on the wrist for CI/CD failures, actually I like to see larger changes with more failures because it means the dev is trying and I consider this normal behaviour, your incorrect assumptions about our test environment and how we interpret these metrics sound like you’re projecting.
      - enord 4y ago
        If you _need_ «metrics» it’s because you _really_ don’t understand the contribution of your reports. i.e. you are not a competent manager by the usual understanding of «competent».
        malux85 4y ago
        I’m not a manager, I’m a dev (well senior dev, that’s why I know about the metrics), but I am measured by the same standards as everyone else. When I became aware of then I viewed my own reports, and found areas I can improve on. I like it. The metrics actually showed no surprises when I saw them, had I been asked to name the worst 5 devs, I would have named the same people that the metrics would have highlighted.
        _djo_ 4y ago
        So what you're saying is that it's useful for devs to be able to view their own stats for this on a no blame basis, and you have used your privileged position of access to them to improve your own performance, but you don't let the other devs do the same because you want to keep using these as secret metrics to manage them. Do I have that right?
  - still_grokking 4y ago
    You're judge people by secret measures? What company are you working for? Human "resources" department, I guess? Looks like a place to strongly avoid because of very bad ethics and culture!
    - malux85 4y ago
      I don’t understand this response, all of us judge each other all of the time on secret, unspoken metrics, that’s human nature. Our entire culture and society is partly built on what is not said. Your bank is scoring you by machine, so is your insurance company, and loan companies, are their algorithms 100% open and publicly available? No. Because they are secret metrics. Your bank knows more about you than you know, and so do the loan companies, I know, I used to work there. The datasets the loan companies have are insane, I could see if you are in financial distress or not, the value of your house, whether you were a “complainer” or not, your favourite music, whether you’re a foodie or not, and heaps, heaps more, I think 250 attributes on each person in the USA at one company. What difference does it make if some metrics are collected by machine and not fuzzy, vague, human intuition? I think it’s an improvement. Yes the metrics inform the seniors decisions partially like I said above, and yes they are taken with a grain of salt, like I said above, they are inputs for deeper investigation, that’s it. Here’s an example that happened just the other day: I noticed heaps of CICD failures for one dev and I reached out to him. He told me he couldn’t easily run the test suite, so I spent an hour peer coding with him and showed him how to use VSCodes debugger. 1 day later he called me up and told me that “using the debugger was a life changer” and he was thrilled to be working so quickly, the faster feedback cycle from code->debug had restored his enthusiasm for work.
      - majjgepolja 4y ago
        the problem is some day the information will leak and the ones who know start gaming the metric.
still_grokking 4y ago
I'm not sure. > Lines of code written Less is better, right? Because it's really easy to throw code at a problem. OTOH it's hard to come up with an optimal minimal solution. > Lines of code rewritten (changed N days after) Means: Your devs commit unfinished, not thought out crap. Or: Management is a failure because they change requirements all the time. > Lines of code removed More is better, right? Because cleaning up your code constantly to keep it lean is key to future maintainability. But could be also a symptom of someone with a NIH attitude. Or, management is a failure because they don't know what they want. > Lines of code that contributed to linting problems If it's more than zero that's a symptom of slacking off and / or ignorance. Your local tools should have showed you the linting problems already. Not handling them is at least outright lazy, and maybe even someone is trying to waste time by committing things that need another iteration later on. Or, your linting rules are nonsense, and it's better to ignore them… > Lines of code that contributed to security issues (as reported by the static analyser) Quite similar to the previous. Ignorance or cluelessness. Maybe even the worst of form of "I don't care", while waiting whether your crap passes CI. In case someone doesn't know how to handle such warnings at all you have a dangerously uneducated person on the team… Or, what is at least equally likely: Your snack-oil security scanners are trash. Producing a lot of the usual false positives (while of course not "seeing" any real issues). > Average complexity measures Completely useless on it's own. Some code needs to be complex because the problem at hand is complex. Also more or less most of this measures are nonsense. The complexity measure goes usually down when you "smear" an implementation all over the place. But most of the time it's more favorable to concentrate and encapsulate complex parts of your code. Hundred one-line methods that call each other are more complex than one method with hundred lines of code. But the usual complexity measure would love the hundred one-liners but barf at the hundred line method. If there's someone who constantly writes "very complex" code (according to such measure done by some tool) this can mean that this person writes over-complicated code, as it could mean that this person is responsible for some complex parts of the code-base, or is consolidating and refactoring complexity that was scattered all across the place. Or maybe even something else. > CI/CD build failure rates This means people can't build and test their code locally… Or someone is lazy and / or ignorant. Or, equally possible, your CI/CD infra is shit. Or the people responsible for that are under-performers. Or your hardware is somehow broken… > Lines of code reviewed How do you even measure this? Just stamping "LGTM" everywhere quickly would let this measure look good. But is anything won by that? I guess the contrary is more likely. Also someone who's constantly complaining about others code would look good here… OTOH valuable and insightful code review is slow, takes a lot of time and effort but does not produces a lot of visible output. That's why I think it's disputable whether this can be measured even in a meaningful way. There is only one valid way to asses whether someone is productive: You need to answer the question whether what this person is doing makes sense in the light of the stated goals. But to answer this you need to look at the actual work / things produced by the person in question, and not on some straw man proxy measures. All measures can be gamed. But faking results is much harder (even still possible of course).
quickthrower2 4y ago
I think these are excellent metrics for the engineering team to discuss at retro in aggregate. But not so good for comparing Joe and Mo. Or even saying “how is Joe doing”. Unless the reviewer is very skilled in navigating that data without bias. Most people I have worked for wont be and will ruin the employee relationship by making some comment about how “this could improve” when the employee things “ok this is BS but better go along with it, and maybe i’ll open CV.docx tonight and give it a spring clean”. Bullshit is the biggest turn off for me in jobs and I will leave a job with too much (some is to be expected and a side effect of getting groups to work together). Team leaders tracking my time and commits closely to judge performance is this. How should they judge? Best way is get people to pair often. You will know what they are actually doing that way or can talk to someone who did. Any real laggers will be found naturally, and then find out why and fix (might be onboarding sux for example). People do hate paring I know but it doesn’t have to be all the time. Even an hour a day of someone jumping on to help will spread boats loads of useful information.

BlargMcLarg 4y ago

Review + written encourages buddy-buddy dynamics. Build failure rates encourages frustration as long as you don't provide ways to run the exact tests in isolation locally, under the same rules as CI. Meanwhile, branch triggers and pre-push hooks trivialize this further. Most of these metrics are strange given they are trivial to check pre-merge. Meanwhile, running these statistics pre-merge discourages small commits and sharing code prior to having ideal builds. The moment this gets out is the moment devs will game your statistics and any benefit you gain goes out the door. And almost every one of these, even combined, is easy to game.

malux85 4y ago
Correct, we do not tell the devs these are being measured
- BlargMcLarg 4y ago
  I cannot tell if you are aware you're setting devs up for failure or not. Half your mentioned metrics are tackled by hard forcing your devs to run branch CI prior to merging and giving them a slap on the wrist for not running CI checks locally pre-push. If they are important enough to measure, why not hard force them and focus on the remaining metrics instead?
  - malux85 4y ago
    See where I said “none taken as absolute because there are assumptions in each one” And “Use these as a springboard for deeper analysis of devs” These are only used as high level metrics to guide further analysis, we are keenly aware of the faults of these metrics - but the alternative of no-metrics or worse, “gut feel”, is slow and riddled with more problems. It doesn’t mean a dev is bad, maybe it means there’s some tooling failure, or organisational problem, or personal problem they need help with. These metrics give visibility into anomalies, and their imperfect nature doesn’t mean they are useless. There’s also like 20 more metrics but I’m on my phone and couldn’t recall them all right now Also I don’t know where all of this nonsense about “forcing” to run CI/CD before branch merges and “slap on the wrist” - all our devs can run the test suites locally in docker and inside their IDE, nobody is forced to do anything in the pipeline severs, and also nobody gets a slap on the wrist for CI/CD failures, actually I like to see larger changes with more failures because it means the dev is trying and I consider this normal behaviour, your incorrect assumptions about our test environment and how we interpret these metrics sound like you’re projecting.
    - enord 4y ago
      If you _need_ «metrics» it’s because you _really_ don’t understand the contribution of your reports. i.e. you are not a competent manager by the usual understanding of «competent».
      - malux85 4y ago
        I’m not a manager, I’m a dev (well senior dev, that’s why I know about the metrics), but I am measured by the same standards as everyone else. When I became aware of then I viewed my own reports, and found areas I can improve on. I like it. The metrics actually showed no surprises when I saw them, had I been asked to name the worst 5 devs, I would have named the same people that the metrics would have highlighted.
        _djo_ 4y ago
        So what you're saying is that it's useful for devs to be able to view their own stats for this on a no blame basis, and you have used your privileged position of access to them to improve your own performance, but you don't let the other devs do the same because you want to keep using these as secret metrics to manage them. Do I have that right?
- still_grokking 4y ago
  You're judge people by secret measures? What company are you working for? Human "resources" department, I guess? Looks like a place to strongly avoid because of very bad ethics and culture!
  - malux85 4y ago
    I don’t understand this response, all of us judge each other all of the time on secret, unspoken metrics, that’s human nature. Our entire culture and society is partly built on what is not said. Your bank is scoring you by machine, so is your insurance company, and loan companies, are their algorithms 100% open and publicly available? No. Because they are secret metrics. Your bank knows more about you than you know, and so do the loan companies, I know, I used to work there. The datasets the loan companies have are insane, I could see if you are in financial distress or not, the value of your house, whether you were a “complainer” or not, your favourite music, whether you’re a foodie or not, and heaps, heaps more, I think 250 attributes on each person in the USA at one company. What difference does it make if some metrics are collected by machine and not fuzzy, vague, human intuition? I think it’s an improvement. Yes the metrics inform the seniors decisions partially like I said above, and yes they are taken with a grain of salt, like I said above, they are inputs for deeper investigation, that’s it. Here’s an example that happened just the other day: I noticed heaps of CICD failures for one dev and I reached out to him. He told me he couldn’t easily run the test suite, so I spent an hour peer coding with him and showed him how to use VSCodes debugger. 1 day later he called me up and told me that “using the debugger was a life changer” and he was thrilled to be working so quickly, the faster feedback cycle from code->debug had restored his enthusiasm for work.
    - majjgepolja 4y ago
      the problem is some day the information will leak and the ones who know start gaming the metric.

still_grokking 4y ago

I'm not sure. > Lines of code written Less is better, right? Because it's really easy to throw code at a problem. OTOH it's hard to come up with an optimal minimal solution. > Lines of code rewritten (changed N days after) Means: Your devs commit unfinished, not thought out crap. Or: Management is a failure because they change requirements all the time. > Lines of code removed More is better, right? Because cleaning up your code constantly to keep it lean is key to future maintainability. But could be also a symptom of someone with a NIH attitude. Or, management is a failure because they don't know what they want. > Lines of code that contributed to linting problems If it's more than zero that's a symptom of slacking off and / or ignorance. Your local tools should have showed you the linting problems already. Not handling them is at least outright lazy, and maybe even someone is trying to waste time by committing things that need another iteration later on. Or, your linting rules are nonsense, and it's better to ignore them… > Lines of code that contributed to security issues (as reported by the static analyser) Quite similar to the previous. Ignorance or cluelessness. Maybe even the worst of form of "I don't care", while waiting whether your crap passes CI. In case someone doesn't know how to handle such warnings at all you have a dangerously uneducated person on the team… Or, what is at least equally likely: Your snack-oil security scanners are trash. Producing a lot of the usual false positives (while of course not "seeing" any real issues). > Average complexity measures Completely useless on it's own. Some code needs to be complex because the problem at hand is complex. Also more or less most of this measures are nonsense. The complexity measure goes usually down when you "smear" an implementation all over the place. But most of the time it's more favorable to concentrate and encapsulate complex parts of your code. Hundred one-line methods that call each other are more complex than one method with hundred lines of code. But the usual complexity measure would love the hundred one-liners but barf at the hundred line method. If there's someone who constantly writes "very complex" code (according to such measure done by some tool) this can mean that this person writes over-complicated code, as it could mean that this person is responsible for some complex parts of the code-base, or is consolidating and refactoring complexity that was scattered all across the place. Or maybe even something else. > CI/CD build failure rates This means people can't build and test their code locally… Or someone is lazy and / or ignorant. Or, equally possible, your CI/CD infra is shit. Or the people responsible for that are under-performers. Or your hardware is somehow broken… > Lines of code reviewed How do you even measure this? Just stamping "LGTM" everywhere quickly would let this measure look good. But is anything won by that? I guess the contrary is more likely. Also someone who's constantly complaining about others code would look good here… OTOH valuable and insightful code review is slow, takes a lot of time and effort but does not produces a lot of visible output. That's why I think it's disputable whether this can be measured even in a meaningful way. There is only one valid way to asses whether someone is productive: You need to answer the question whether what this person is doing makes sense in the light of the stated goals. But to answer this you need to look at the actual work / things produced by the person in question, and not on some straw man proxy measures. All measures can be gamed. But faking results is much harder (even still possible of course).

quickthrower2 4y ago

I think these are excellent metrics for the engineering team to discuss at retro in aggregate. But not so good for comparing Joe and Mo. Or even saying “how is Joe doing”. Unless the reviewer is very skilled in navigating that data without bias. Most people I have worked for wont be and will ruin the employee relationship by making some comment about how “this could improve” when the employee things “ok this is BS but better go along with it, and maybe i’ll open CV.docx tonight and give it a spring clean”. Bullshit is the biggest turn off for me in jobs and I will leave a job with too much (some is to be expected and a side effect of getting groups to work together). Team leaders tracking my time and commits closely to judge performance is this. How should they judge? Best way is get people to pair often. You will know what they are actually doing that way or can talk to someone who did. Any real laggers will be found naturally, and then find out why and fix (might be onboarding sux for example). People do hate paring I know but it doesn’t have to be all the time. Even an hour a day of someone jumping on to help will spread boats loads of useful information.