5 ms·

This is a good attempt to kickstart the discussion of some quantitative metric about bugs and the time one needs to deal with it, but I wonder if there wasn't

This is a good attempt to kickstart the discussion of some quantitative metric about bugs and the time one needs to deal with it, but I wonder if there wasn't any software engineering theories or practices to address this. If anyone know other related topics, please share them, thanks!

jakear 8y ago
This is somewhat similar to the concept of “Cyclomatic Complexity”, which is the number of independent paths though a piece of code, and can be statically analyzed. https://en.m.wikipedia.org/wiki/Cyclomatic_complexity https://en.m.wikipedia.org/wiki/Cyclomatic_complexity
- danabramov 8y ago
  Note one difference is that I'm talking about characteristics of an API, not particular code snippet. Although it's relevant.
- amelius 8y ago
  This makes me wonder, are software metrics still a thing?
  - rbanffy 8y ago
    Yes. At least for me.
- jerf 8y ago
  I like this concept better than cyclomatic complexity, precisely because this is what cyclomatic complexity is trying to poorly approximate. But it can't be automatically derived in the general case, or even terribly easily in non-general cases, either, unlike cyclomatic complexity. I use a descendant of gometalinter on all my Go code, and one of the things it ships with is cyclomatic complexity, and I always turn it off. Precisely because I tend to think like this, it often gives my code terribly wrong ratings because it can't see that there's actually only a finite set of paths through the code that are possible, and also as the blog post demonstrates, sometimes adding more code actually simplifies the conceptual flow. But cyclomatic complexity will generally say the "more code" is even more cyclomatically complex. There's also some Go-specific elements to my distaste for the measure, though; since in Go right now handling an error is automatically an if statement, that'll hit you right in the metrics. But in many cases, something, err := GetSomething() if err != nil { return errwrap.Wrapf("while trying to get something: {{err}}", err) } Officially that's an if statement; unofficially, it ought to be a cyclomatic complexity of zero. Now, it isn't technically a free if clause, in the sense that a bug could live there, but if you're going to count the exception-based equivalent as zero (and it has the same callout; bugs can lie in your exception handling exactly the same way), then this ought to be practically zero too. Consequently, Go functions tend to get smacked with much higher complexity numbers than exception-equivalent code. Yeah, it's probably not a one-to-one, but it's certainly not as lopsided as the metric makes it seem.
  - sigil 8y ago
    Does gometalinter report absolute cyclomatic complexity numbers, or does it report density (cyclomatic complexity / sloc)?
l0b0 8y ago
I've found TDD to be a good (but not perfect, as some people complain) way to keep complexity down - when you force yourself to test every state you very quickly start focussing on how to keep the number of possible states down.
- AllegedAlec 8y ago
  I think that a more important reason is that TDD forces you to write code in small modules which can be tested independently. Because of this, you have small to medium size modules to reason about, rather than one huge class where you wouldn't know where to start.
jtolmar 8y ago
There are various studies of defect density (bugs per 1000 lines of code) versus whatever factor someone thought would influence bugs. "Software defect density" seem to be the magic search terms for finding actual studies instead of blog posts. I unfortunately can't find it, but I remember there being a study (by Microsoft?) of defect density for projects with different methodologies (scrum, TDD, whatever), which found lines of code is more correlated with defect rate than everything else they tried. They took that as a failure; I've always taken that to mean that reducing lines of code is the most important first step (or probably more accurately reducing tokens of executable code; one liners don't help and type annotations don't hurt). While attempting to find that study, I found a study [1] that claims to have found an empirical sweet spot in defect density by module size - apparently 400 lines for assembly modules or 200 lines for ADA modules. Those are some weird languages to have numbers on, but maybe something else in that paper's family tree has something more relatable. [1] http://www.cs.colostate.edu/~malaiya/p/denton_2000.pdf http://www.cs.colostate.edu/~malaiya/p/denton_2000.pdf
- beaconstudios 8y ago
  even reducing tokens isn't a very good measure - lines of code is a proxy for complexity of implementation. I think a good measure (albeit a heuristic one) is how concise the implementation is relative to the complexity of the problem - or in other words, the ratio of accidental to essential complexity.
biggio 8y ago
It's a broad topic and called "code metrics" or "software metrics".

jakear 8y ago

This is somewhat similar to the concept of “Cyclomatic Complexity”, which is the number of independent paths though a piece of code, and can be statically analyzed. https://en.m.wikipedia.org/wiki/Cyclomatic_complexity https://en.m.wikipedia.org/wiki/Cyclomatic_complexity

danabramov 8y ago
Note one difference is that I'm talking about characteristics of an API, not particular code snippet. Although it's relevant.
amelius 8y ago
This makes me wonder, are software metrics still a thing?
- rbanffy 8y ago
  Yes. At least for me.
jerf 8y ago
I like this concept better than cyclomatic complexity, precisely because this is what cyclomatic complexity is trying to poorly approximate. But it can't be automatically derived in the general case, or even terribly easily in non-general cases, either, unlike cyclomatic complexity. I use a descendant of gometalinter on all my Go code, and one of the things it ships with is cyclomatic complexity, and I always turn it off. Precisely because I tend to think like this, it often gives my code terribly wrong ratings because it can't see that there's actually only a finite set of paths through the code that are possible, and also as the blog post demonstrates, sometimes adding more code actually simplifies the conceptual flow. But cyclomatic complexity will generally say the "more code" is even more cyclomatically complex. There's also some Go-specific elements to my distaste for the measure, though; since in Go right now handling an error is automatically an if statement, that'll hit you right in the metrics. But in many cases, something, err := GetSomething() if err != nil { return errwrap.Wrapf("while trying to get something: {{err}}", err) } Officially that's an if statement; unofficially, it ought to be a cyclomatic complexity of zero. Now, it isn't technically a free if clause, in the sense that a bug could live there, but if you're going to count the exception-based equivalent as zero (and it has the same callout; bugs can lie in your exception handling exactly the same way), then this ought to be practically zero too. Consequently, Go functions tend to get smacked with much higher complexity numbers than exception-equivalent code. Yeah, it's probably not a one-to-one, but it's certainly not as lopsided as the metric makes it seem.
- sigil 8y ago
  Does gometalinter report absolute cyclomatic complexity numbers, or does it report density (cyclomatic complexity / sloc)?

l0b0 8y ago

I've found TDD to be a good (but not perfect, as some people complain) way to keep complexity down - when you force yourself to test every state you very quickly start focussing on how to keep the number of possible states down.

AllegedAlec 8y ago
I think that a more important reason is that TDD forces you to write code in small modules which can be tested independently. Because of this, you have small to medium size modules to reason about, rather than one huge class where you wouldn't know where to start.

jtolmar 8y ago

There are various studies of defect density (bugs per 1000 lines of code) versus whatever factor someone thought would influence bugs. "Software defect density" seem to be the magic search terms for finding actual studies instead of blog posts. I unfortunately can't find it, but I remember there being a study (by Microsoft?) of defect density for projects with different methodologies (scrum, TDD, whatever), which found lines of code is more correlated with defect rate than everything else they tried. They took that as a failure; I've always taken that to mean that reducing lines of code is the most important first step (or probably more accurately reducing tokens of executable code; one liners don't help and type annotations don't hurt). While attempting to find that study, I found a study [1] that claims to have found an empirical sweet spot in defect density by module size - apparently 400 lines for assembly modules or 200 lines for ADA modules. Those are some weird languages to have numbers on, but maybe something else in that paper's family tree has something more relatable. [1] http://www.cs.colostate.edu/~malaiya/p/denton_2000.pdf http://www.cs.colostate.edu/~malaiya/p/denton_2000.pdf

beaconstudios 8y ago
even reducing tokens isn't a very good measure - lines of code is a proxy for complexity of implementation. I think a good measure (albeit a heuristic one) is how concise the implementation is relative to the complexity of the problem - or in other words, the ratio of accidental to essential complexity.

biggio 8y ago

It's a broad topic and called "code metrics" or "software metrics".