7 ms·

I don't agree with most of the advice in this article but rather than complain let me suggest an alternative. As a line manager, with software engineers report

by lacker 5y ago

I don't agree with most of the advice in this article but rather than complain let me suggest an alternative. As a line manager, with software engineers reporting directly to you, you should be able to use your personal judgment to understand the productivity of your software engineers. Don't measure it with acronyms, with metrics like the number of commits, or by paying attention to how many hours a week people are working. Pay attention to whether people get things done, and are they getting big important things done, or only little nice-but-not-critical things. Make sure you communicate enough so that individual software engineers understand how you think and what you prioritize. As a manager-of-managers, it is going to be very difficult for you to measure developer productivity. It's tempting to look at metrics like the number of code reviews a developer does. But these can at most be a sanity check, not the core metric to go for. Instead, you can measure productivity of teams. Is the team getting things done, and are they big important things, or only little nice-but-not-critical things? Sometimes, a line manager will insist that everyone on their team is performing excellently, and yet you observe the team overall is not achieving very much. Probably one of the two of you is incorrect, and you should dig in to figure that out. The opposite also happens, where a manager states that everything is a disaster, but you observe that the team has actually delivered a lot. The other thing you can do is to teach your line managers how to judge individual productivity. There's no silver bullet, it's just a natural outcome of having conversations about who is productive and who is not and how to tell and what to do about it, so be sure to have enough of those conversations. None of this is easy to quantify, but the hard truth is, there is no natural mapping from numbers to developer productivity and it is usually a bad idea to try to quantify productivity. You are much better off using human language and intelligent thinking to evaluate productivity, rather than reductionist metrics.

blurker 5y ago
I too have come to think that no simple metrics will ever replace the need for a competent manager who can use intangible, subjective context to evaluate their team. I think that even if you get some metrics that work well initially, the system will change such that the metric becomes the goal and the metrics then become much less effective.
- 52-6F-62 5y ago
  Well put. I couldn't agree more. Humans and relationships are nuanced, including work relationships and the responsibilities and expectations there. It's best to treat them as they are rather than trying to shoehorn those things into such a sweet little checkbox. Frameworks are alright, but they need flexibility built in. They certainly shouldn't be treated as religiously as they are commonly.
- qbasic_forever 5y ago
  You still have a goal that people are optimizing for though--how to increase that intangible, subjective measure known only to the manager. It leads to people optimizing for talking about the stuff they do more, showcasing their accomplishments, and in bad extremes brown-nosing, infighting and sabotage of others work. Whether any of those behaviors lead to a better product or outcome for the company is something to strongly consider.
arthurcoudouy 5y ago
I can't agree more with you. I tried to sum my thoughts in my first reference > One of the most common myths — and potentially most threatening to developer happiness — is the notion that productivity is all about developer activity, things like lines of code or number of commits. More activity can appear for various reasons: working longer hours may signal developers having to "brute-force" work to overcome bad systems or poor planning to meet a predefined release schedule. The SPACE framework is not about measuring quantitative data only. I feel the need to explain how certain metrics might be interesting, but rather to identify key issues or unexpected events during engineering sprints. Without data analysis, you would not be able to understand why there is a drop of productivity during certain periods, and usually, those drops were created by the management (too many meetings or lack of follow-up)
jgust 5y ago
There's an entire class of products I'll name "internal platform tools" whose primary objective is to improve the developer experience with the intent of having the side effect of increased developer productivity by making it easier & more enjoyable to build things within a company. The teams working on these tools need to understand how their products perform the same as a team building some widget for a "paying" customer. Without some quantifiable metric, how do these teams know if their products are getting better or worse? The discussion always goes to measuring developer happiness & developer productivity because we want with some degree of confidence to be improving or at least maintaining these metrics.
- cco 5y ago
  Generally you find the problems in your developer experience, and use those as your metrics. Maybe it takes three PRs and an hour and fifteen minutes to deploy to prod, lower number of PRs and minutes to deploy would be your metrics. Or maybe to introduce a new endpoint in your API takes X amount of boilerplate lines, Y files, etc and you post mortem new endpoints after your change to ensure that number is dropping. Talk to people, find out what their problems are, quantify the problem, measure.
- closeparen 5y ago
  Inefficiencies in the developer experience show up as frustrations for developers. Developers are very happy to tell you what frustrates them and how badly. Often what frustrates people is a latency, which is something you can measure and track. Other times it is an ugliness, surprising footgun, or lack of conceptual integrity - these are fundamentally human experiences, and subjective assessment is the only way.
- passivate 5y ago
  Agreed, we need both quantifiable metrics, and also a human brain to interpret them with subjectivity, context and compassion. I see many people wanting to take writing code into the liberal arts domain, but I am of the opinion that it may be more useful if we can overlap it with the engineering domain. IMHO the goal should be to repeatedly churn out high-quality bug-free code, and to create an objective process methodology so that time and money is well spent. We may end up with multiple different methodologies for various technologies, domains, etc.
giantg2 5y ago
We just need to be careful that this unquantifiable, subjective rating doesn't include biases.
- qbasic_forever 5y ago
  Exactly, I don't know what the answer is (probably a combo of subjective and objective measures, along with a healthy dose of independent oversight for both) but relying completely on 'gut feeling' is an express train to unconscious bias land. In reality I think someone who only looks at team members who are "doing the most" is really just measuring who is talking about their work the most. You need some kind of objective measures like features shipped, assigned bugs resolved, etc.
- marcinzm 5y ago
  Everything including "objective" metrics includes bias. And that's before you take into account people outright gaming metrics (objective or subjective).
  - giantg2 5y ago
    As a simplified example, if I write 1000 lines of code and you write 1000 lines of code. We should have the same rating if that's the metric used. There shouldn't be any bias there. It only introduces bias when the manager feel your code is better than mine, etc. Now the objective measure itself might have some sort of bias, but at least the rules are set and you're not getting screwed over based on someone's feelings. You can argue metrics, you can't argue your managers feelings.
    - bluefirebrand 5y ago
      If your 1000 lines of code generate four new bug tickets and mine doesn't generate any, is that biased to say mine is better? Or how about even if yours generates 10 comments on the PR correcting things to match code quality guidelines and mine doesn't? I don't think we often track things like that.
    - 5y ago
kelnos 5y ago
Absolutely agree. I think there is a conflict of interest there, though. Managers have a vested interest in saying that their team is highly productive. Managers of highly productive teams get raises and more head count, and eventually promotions. Anything else reflects poorly on the manager. So the managers-of-managers do need to keep their eyes on this too, but I also agree with you that it's harder for people in that higher-level position to evaluate this. I guess, as you hint at, the manager-of-managers can look at team output overall, and if that's below expectations, that's a starting point for discussion with the line manager.
- tonyedgecombe 5y ago
  Measuring team output seems just as difficult as for an individual. I was reading an interesting piece in the economist today about maintaining peoples performance on a long space flight. One part that stood out to me is people being productive makes them happy. I always assumed the causation would be the opposite way round. Perhaps the best a manager of managers can do is try and figure out if the members of each team are happy or not.
passivate 5y ago
I don't see how you can make the leap from "its hard to measure" to "no metrics are useful". As with anything you have to use your judgement, experience, and its a case-by-case thing. Everything is a signal. Lines of code, number of bugs fixed, number of bugs found, severity of bugs, hours in office, meeting project milestones, contribution in team meetings, etc, etc. Its up to you how to interpret each signal. As you rightfully said, there is no silver bullet.

by lacker 5y ago

blurker 5y ago

I too have come to think that no simple metrics will ever replace the need for a competent manager who can use intangible, subjective context to evaluate their team. I think that even if you get some metrics that work well initially, the system will change such that the metric becomes the goal and the metrics then become much less effective.

52-6F-62 5y ago
Well put. I couldn't agree more. Humans and relationships are nuanced, including work relationships and the responsibilities and expectations there. It's best to treat them as they are rather than trying to shoehorn those things into such a sweet little checkbox. Frameworks are alright, but they need flexibility built in. They certainly shouldn't be treated as religiously as they are commonly.
qbasic_forever 5y ago
You still have a goal that people are optimizing for though--how to increase that intangible, subjective measure known only to the manager. It leads to people optimizing for talking about the stuff they do more, showcasing their accomplishments, and in bad extremes brown-nosing, infighting and sabotage of others work. Whether any of those behaviors lead to a better product or outcome for the company is something to strongly consider.

arthurcoudouy 5y ago

I can't agree more with you. I tried to sum my thoughts in my first reference > One of the most common myths — and potentially most threatening to developer happiness — is the notion that productivity is all about developer activity, things like lines of code or number of commits. More activity can appear for various reasons: working longer hours may signal developers having to "brute-force" work to overcome bad systems or poor planning to meet a predefined release schedule. The SPACE framework is not about measuring quantitative data only. I feel the need to explain how certain metrics might be interesting, but rather to identify key issues or unexpected events during engineering sprints. Without data analysis, you would not be able to understand why there is a drop of productivity during certain periods, and usually, those drops were created by the management (too many meetings or lack of follow-up)

jgust 5y ago

There's an entire class of products I'll name "internal platform tools" whose primary objective is to improve the developer experience with the intent of having the side effect of increased developer productivity by making it easier & more enjoyable to build things within a company. The teams working on these tools need to understand how their products perform the same as a team building some widget for a "paying" customer. Without some quantifiable metric, how do these teams know if their products are getting better or worse? The discussion always goes to measuring developer happiness & developer productivity because we want with some degree of confidence to be improving or at least maintaining these metrics.

cco 5y ago
Generally you find the problems in your developer experience, and use those as your metrics. Maybe it takes three PRs and an hour and fifteen minutes to deploy to prod, lower number of PRs and minutes to deploy would be your metrics. Or maybe to introduce a new endpoint in your API takes X amount of boilerplate lines, Y files, etc and you post mortem new endpoints after your change to ensure that number is dropping. Talk to people, find out what their problems are, quantify the problem, measure.
closeparen 5y ago
Inefficiencies in the developer experience show up as frustrations for developers. Developers are very happy to tell you what frustrates them and how badly. Often what frustrates people is a latency, which is something you can measure and track. Other times it is an ugliness, surprising footgun, or lack of conceptual integrity - these are fundamentally human experiences, and subjective assessment is the only way.
passivate 5y ago
Agreed, we need both quantifiable metrics, and also a human brain to interpret them with subjectivity, context and compassion. I see many people wanting to take writing code into the liberal arts domain, but I am of the opinion that it may be more useful if we can overlap it with the engineering domain. IMHO the goal should be to repeatedly churn out high-quality bug-free code, and to create an objective process methodology so that time and money is well spent. We may end up with multiple different methodologies for various technologies, domains, etc.

giantg2 5y ago

We just need to be careful that this unquantifiable, subjective rating doesn't include biases.

qbasic_forever 5y ago
Exactly, I don't know what the answer is (probably a combo of subjective and objective measures, along with a healthy dose of independent oversight for both) but relying completely on 'gut feeling' is an express train to unconscious bias land. In reality I think someone who only looks at team members who are "doing the most" is really just measuring who is talking about their work the most. You need some kind of objective measures like features shipped, assigned bugs resolved, etc.
marcinzm 5y ago
Everything including "objective" metrics includes bias. And that's before you take into account people outright gaming metrics (objective or subjective).
- giantg2 5y ago
  As a simplified example, if I write 1000 lines of code and you write 1000 lines of code. We should have the same rating if that's the metric used. There shouldn't be any bias there. It only introduces bias when the manager feel your code is better than mine, etc. Now the objective measure itself might have some sort of bias, but at least the rules are set and you're not getting screwed over based on someone's feelings. You can argue metrics, you can't argue your managers feelings.
  - bluefirebrand 5y ago
    If your 1000 lines of code generate four new bug tickets and mine doesn't generate any, is that biased to say mine is better? Or how about even if yours generates 10 comments on the PR correcting things to match code quality guidelines and mine doesn't? I don't think we often track things like that.
  - 5y ago

kelnos 5y ago

Absolutely agree. I think there is a conflict of interest there, though. Managers have a vested interest in saying that their team is highly productive. Managers of highly productive teams get raises and more head count, and eventually promotions. Anything else reflects poorly on the manager. So the managers-of-managers do need to keep their eyes on this too, but I also agree with you that it's harder for people in that higher-level position to evaluate this. I guess, as you hint at, the manager-of-managers can look at team output overall, and if that's below expectations, that's a starting point for discussion with the line manager.

tonyedgecombe 5y ago
Measuring team output seems just as difficult as for an individual. I was reading an interesting piece in the economist today about maintaining peoples performance on a long space flight. One part that stood out to me is people being productive makes them happy. I always assumed the causation would be the opposite way round. Perhaps the best a manager of managers can do is try and figure out if the members of each team are happy or not.

passivate 5y ago

I don't see how you can make the leap from "its hard to measure" to "no metrics are useful". As with anything you have to use your judgement, experience, and its a case-by-case thing. Everything is a signal. Lines of code, number of bugs fixed, number of bugs found, severity of bugs, hours in office, meeting project milestones, contribution in team meetings, etc, etc. Its up to you how to interpret each signal. As you rightfully said, there is no silver bullet.