3 ms·

Does anybody know of a study documenting that measuring and reporting software code quality metrics actually improve the project/product team's ability to deliv

Does anybody know of a study documenting that measuring and reporting software code quality metrics actually improve the project/product team's ability to deliver? So far I've only found studies documenting that measuring code quality improve the team's ability to improve the measurement figures. That doesn't count. More or less any measurement will be gamed if anyone think they are used for something. But has anybody measured if these measurements actually are helpfull?

commandlinefan 7y ago
Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."
wbharding 7y ago
This is a terrific question, and a topic I've thought about quite a bit while we've built our product (context: as the OP, I've toiled on "how to prove metrics are signal-bearing" for a couple years now). The problem we always end up at is "what measurable output can we rely upon to tell the story that a project's fortunes are improving"? To date, its been challenging to find a compelling answer. This may be why I'm unaware of any study that addresses the question. One possible workaround would be to take experts in the field, and try to translate their qualitative interpretation into input that drives a metric. e.g., if Andrew Clark from React could list three directories that harbor disproportionate tech debt in their project, we could calibrate our Open Repo Directory Browser to identify the factors that were unique to that directory. This could then help managers pick out when and where its appropriate to make payments to reduce their tech debt. Unfortunately, the prominent open source developers aren't answering my emails at this point. And even if they did, it's still a long road to translate qualitative expert opinion into provable long-term benefits. But I think the environment for making that leap is as ripe as ever. We just need to keep working toward figuring out the most desirable+measurable outputs.
- namibj 7y ago
  This sounds like something Felix von Leitner might be interested in. Consider collaborating. https://blog.fefe.de https://blog.fefe.de
jasonpeacock 7y ago
Check out the book "Accelerate", it answers your questions about which metrics matter, and how.

commandlinefan 7y ago

Goodhart's law: "When a measure becomes a target, it ceases to be a good measure."

wbharding 7y ago

This is a terrific question, and a topic I've thought about quite a bit while we've built our product (context: as the OP, I've toiled on "how to prove metrics are signal-bearing" for a couple years now). The problem we always end up at is "what measurable output can we rely upon to tell the story that a project's fortunes are improving"? To date, its been challenging to find a compelling answer. This may be why I'm unaware of any study that addresses the question. One possible workaround would be to take experts in the field, and try to translate their qualitative interpretation into input that drives a metric. e.g., if Andrew Clark from React could list three directories that harbor disproportionate tech debt in their project, we could calibrate our Open Repo Directory Browser to identify the factors that were unique to that directory. This could then help managers pick out when and where its appropriate to make payments to reduce their tech debt. Unfortunately, the prominent open source developers aren't answering my emails at this point. And even if they did, it's still a long road to translate qualitative expert opinion into provable long-term benefits. But I think the environment for making that leap is as ripe as ever. We just need to keep working toward figuring out the most desirable+measurable outputs.

namibj 7y ago
This sounds like something Felix von Leitner might be interested in. Consider collaborating. https://blog.fefe.de https://blog.fefe.de

jasonpeacock 7y ago

Check out the book "Accelerate", it answers your questions about which metrics matter, and how.