6 ms·
Even the 'good' metrics you listed are not that good and can be easily gamed if they are important enough, e.g. can influence salary, etc. Happy customers is t
by gedrap 9y ago
Even the 'good' metrics you listed are not that good and can be easily gamed if they are important enough, e.g. can influence salary, etc.
Happy customers is the only exception, but it's essentially impossible to attribute this to a team or an individual and is more of a company's health metric. You could just replace it with revenue.
>>> products shipped, features delivered per team
Will lead to releasing products that should never be released, or releasing too early, or creating a lot of features just to increase the counter.
>>> bugs fixed
Then why ship quality product in the first place, if it is more valuable to have easy bugs to fix later?
>>> documentation written
Can result in poorly written, or way too detailed documentation which will go out of date the next day.
>>> individual git commits
Some people commit more than a dozen of times a day and happily push them. Or you can create tons of commits that deliver very little value (e.g. adds a single comment), but increases the counter.
I understand that it is all well intended, it just seems like it creates a game rather than solves a problem. Some people are great at adapting to games and rules, some people - not so much.
Quantifying developer's work has been discussed a lot, and I am still largely skeptical about such low level metrics. Essentially, it motivates people to maximize the score, even if it means doing things that are only good for the score, instead of good for the company.
- fao_ 9y agoBut all of that assumes that you're just plotting it on a graph and checking how it looks. Humans are capable of doing more assessment than that. Really a manager should do at least some cursory checks of documentation, etc. to ensure that it's not poorly written, etc.
- solatic 9y agoI think reactions like this tend to sidestep the issue, because orgs should never adopt any one metric in isolation, but instead try to use many different metrics as part of a larger holistic picture, both of the organization and of individuals. If you measure how much stuff got done on paper within its agreed-upon deadline, sure, people may skimp on quality in order to ship something on time, and it probably would've been in the org's interest to delay release until all the work was really done. And sure, if you measure against the number of bugs opened against production versions, trying to incentivize the least number of bugs opened as possible, you may invite people dealing with issues through back channels, so that they don't show up in the metrics. And of course revenue is more of a company-wide metric. But if you combine all of them - if your teams are hitting their feature development targets, and customers aren't complaining through established support lines, and revenues are up... is that not what makes a successful company?
- jdbernard 9y agoIn addition to your points, with which I agree, it also depends a lot on team culture. If you do your best to hire responsible adults and then treat your employees like adults most of them act like adults. When there is a healthy team culture of shared responsibility people who aren't pulling their weight tend to stick out. The best way to avoid people gaming the system is to hire honest, hard working people who don't want to game the system. Then get out of the way and let them work.
- solatic 9y agoYou're right, of course, but hiring for culture fit is easier when you're a small company and you only have to hire for a few distinct roles. When you're a large company and you have a hiring quota of hundreds of engineers a year to deal with expected development requirements, well, that's where you start to need to "corporatize" and come up with good metrics.
- jdbernard 9y agoI've never run a large company, or been high enough in management to try this, but I still think there is a better way: In code we work in layers of abstraction. Large projects can be healthy when every layer is small enough to fit in your head. In the same way, I think we could build a large company in layers of teams. Each layer should be manageable in the way discussed above. Good modularity in code allows us to keep the whole problem in our head. Good team structure (requires good management) allows us to keep the management decisions at the tribe/human level that we are good at and not need the kind of mass, impersonal systems that lead to burdensome bureaucracy. Of course, this kind of leadership and culture has to come from the top. It also requires that leadership explicitly avoid micromanagement, breaking the team boundaries. And requires continued commitment to treating your employees at every level like adults. Of course, people are not code, but I think this organisational system of managing complexity can be applied to both. It would be interesting to do some case studies, but I believe there are several large companies that have done this successfully, until top leadership changes (founder dies, etc.) and the new leader doesn't have the same trust of and commitment to employee/team autonomy. The dynamic changes, people notice and either leave or start gaming the system, etc.
- conradfr 9y agoAt my last job we learned that productivity was measured with scrum points (so, estimates) done and your Github activity. It was ... not really efficient.
- Domenic_S 9y agoI could see measuring the "points committed" : "points complete" ratio, but the points themselves...?