3 ms·

This article is right about one thing: managers should avoid mechanical metrics (lines of code cranked, bugs fixed, etc.) as a way of measuring developer compet

This article is right about one thing: managers should avoid mechanical metrics (lines of code cranked, bugs fixed, etc.) as a way of measuring developer competence. Many developers are good at figuring out constraints and optimizing around them. It's a game for us. We're more productive when we're playing that game on behalf of our businesses and customers than on our own behalf. But if you award us points (dollars) based on optimizing metrics for ourselves, we'll probably do it. At the very least we'll have to spend mental energy resisting the temptation. These kinds of metrics inevitably generate perverse incentives. It's true that developers should remember that debugging code is harder than creating code. We should avoid creating code that is right at the limit of our cognitive capacity to understand. When we do create code like that, we've created code that's a stretch to debug. There's a management corollary: don't use the most ingenious incentives you can imagine to motivate your krewe. You'll have a hard time debugging the incentives. And your krewe may exploit your buggy incentives in the meantime. And, in both cases, imagine what will happen when your super-clever code or incentive scheme is handed off to your successor. Oh? You want to keep doing exactly what you're doing now for the rest of your career? OK, go for it.

Bartweiss 10y ago
> These kinds of metrics inevitably generate perverse incentives. Precisely. There's a post that goes around about finding good programmers, which ended up concluding that good developers almost inevitably make many commits with small per-commit changes. I had some issues with the methodology (their outlier-exclusion looked like it might force the result), but the outcome made intuitive sense. I would be horrified if anyone used that metric to judge employee quality. Even assuming it's flawless as a retroactive assessment, it's still trivially gamed. As soon as something that blandly quantitative affects people's work outcomes, it's ruined by the fact that optimizing for the metric is more efficient than optimizing for good work which happens to meet the metric.
- Nomentatus 10y ago
  Remember though, that the MMPI personality test actually trolls for those gaming the test (diagnosing them with personality disorders, usually.) It does that by tempting people to exaggerate their virtues, for example. So if you're willing to dive into the details of those commits, trumpeting that metric might be a nice honey trap. You don't want the serious gamers and players in your workplace.
  - Bartweiss 10y ago
    Yeah - basically all working real-world systems include "and don't get clever" as one of the rules. If you can't refine your rules to be unbeatable, just put in a ban on beating the rules! There's an old story about a Soviet pin factory that was judged by number of pins, so it made huge numbers of tiny, useless pins. In response they were switched to a weight metric, so they switched to making giant, 100 pound pins. It's not a true story, because whoever tried it would have gone to prison immediately. You don't actually need to legislate exactly how to behave, because you can just demand good faith and punish people who don't offer it. I might sound sarcastic there, but it's pretty true. Employee metrics can work on a code of "don't cheat, jerk" as long as the cheating is detectable.

Bartweiss 10y ago

> These kinds of metrics inevitably generate perverse incentives. Precisely. There's a post that goes around about finding good programmers, which ended up concluding that good developers almost inevitably make many commits with small per-commit changes. I had some issues with the methodology (their outlier-exclusion looked like it might force the result), but the outcome made intuitive sense. I would be horrified if anyone used that metric to judge employee quality. Even assuming it's flawless as a retroactive assessment, it's still trivially gamed. As soon as something that blandly quantitative affects people's work outcomes, it's ruined by the fact that optimizing for the metric is more efficient than optimizing for good work which happens to meet the metric.

Nomentatus 10y ago
Remember though, that the MMPI personality test actually trolls for those gaming the test (diagnosing them with personality disorders, usually.) It does that by tempting people to exaggerate their virtues, for example. So if you're willing to dive into the details of those commits, trumpeting that metric might be a nice honey trap. You don't want the serious gamers and players in your workplace.
- Bartweiss 10y ago
  Yeah - basically all working real-world systems include "and don't get clever" as one of the rules. If you can't refine your rules to be unbeatable, just put in a ban on beating the rules! There's an old story about a Soviet pin factory that was judged by number of pins, so it made huge numbers of tiny, useless pins. In response they were switched to a weight metric, so they switched to making giant, 100 pound pins. It's not a true story, because whoever tried it would have gone to prison immediately. You don't actually need to legislate exactly how to behave, because you can just demand good faith and punish people who don't offer it. I might sound sarcastic there, but it's pretty true. Employee metrics can work on a code of "don't cheat, jerk" as long as the cheating is detectable.